Evaluating Patient Experience when using Digital Healthcare Services : surveys and Natural Language Processing-based methods

Tese de mestrado, Engenharia Biomédica e Biofísica , 2023, Universidade de Lisboa, Faculdade de Ciências ; Evaluating patients' experiences has become a standard measure to judge the quality of care. However, studying patient experience data may be a time-consuming task. Efficient analysis techniques to examine such feedback have not been frequently applied in European Portuguese, especially for digital healthcare services. To fill this gap, two approaches were considered. Firstly, for structured data, we compiled and translated items from validated questionnaires for digital healthcare services in English, resulting in a 13-item questionnaire in European Portuguese. Secondly, a Natural Language Processing (NLP) pipeline was developed to analyse unstructured data. The pipeline was applied to 20 patient interview transcripts from a digital healthcare service. The data was pre-processed, vectorized, and each word was assigned to its morphosyntactic category. Posteriorly, a dictionary-based approach was implemented to return the sentiment of a comment based on the number of positive, negative, or neutral words. Two machine learning algorithms were trained and tested with a previously labelled dataset to classify comments according to their sentiment. Finally, a topic model unsupervised algorithm, Latent Dirichlet Allocation, was created to predict the topics of the text. The sentiment of each topic was calculated using the dictionary-based approach. The most common adjectives gave valuable insights into the text. The obtained accuracy for the sentiment dictionary-based approach was 68% when compared to manual labelling, and 59% and 78% for each machine learning model. The results made us conclude that the overall satisfaction with the project was positive. However, a larger dataset would be necessary to examine this model's feasibility, as the study's main limitation is the dataset's size and quality. Still, by applying text analytics tools, we could demonstrate how NLP could be used on a larger scale for European ...