Background: Regulatory bodies such as health care inspectorates can identify potential patient safety problems in health care providers by analyzing patient complaints. However, it is challenging to analyze the large number of complaints. Text mining techniques may help identify signals of problems with patient safety at health care providers. Objective: The aim of this study was to explore whether employing text mining techniques on patient complaint databases can help identify potential problems with patient safety at health care providers and automatically predict the severity of patient complaints. Methods: We performed an exploratory study on the complaints database of the Dutch Health and Youth Care Inspectorate with more than 22,000 written complaints. Severe complaints are defined as those cases where the inspectorate contact point experts deemed it worthy of a triage by the inspectorate, or complaints that led to direct action by the inspectorate. We investigated a range of supervised machine learning techniques to assign a severity label to complaints that can be used to prioritize which incoming complaints need the most attention. We studied several features based on the complaints’ written content, including sentiment analysis, to decide which were helpful for severity prediction. Finally, we showcased how we could combine these severity predictions and automatic keyword analysis on the complaints database and listed health care providers and their organization-specific complaints to determine the average severity of complaints per organization. Results: A straightforward text classification approach using a bag-of-words feature representation worked best for the severity prediction of complaints. We obtained an accuracy of 87%-93% (2658-2990 of 3319 complaints) on the held-out test set and an F1 score of 45%-51% on the severe complaints. The skewed class distribution led to only reasonable recall (47%-54%) and precision (44%-49%) scores. The use of sentiment analysis for severity prediction was not helpful. By combining the predicted severity outcomes with an automatic keyword analysis, we identified several health care providers that could have patient safety problems. Conclusions: Text mining techniques for analyzing complaints by civilians can support inspectorates. They can automatically predict the severity of the complaints, or they can be used for keyword analysis. This can help the inspectorate detect potential patient safety problems, or support prioritizing follow-up supervision activities by sorting complaints based on the severity per organization or per sector.

This is the abstract only. Read the full article on the JMIR site. JMIR is the leading open access journal for eHealth and healthcare in the Internet age.

中文翻译：

---

使用文本挖掘技术识别存在患者安全问题的医疗保健提供者：探索性研究

背景：医疗保健检查员等监管机构可以通过分析患者投诉来识别医疗保健提供者中潜在的患者安全问题。然而，分析大量的投诉是具有挑战性的。文本挖掘技术可能有助于识别医疗保健提供者的患者安全问题信号。目的：本研究的目的是探讨在患者投诉数据库上采用文本挖掘技术是否有助于识别医疗保健提供者的患者安全潜在问题，并自动预测患者投诉的严重程度。方法：我们对荷兰健康和青年护理监察局的投诉数据库进行了探索性研究，其中包含 22,000 多份书面投诉。严重投诉被定义为监察员联络点专家认为值得监察员分类的情况，或导致监察员直接采取行动的投诉。我们研究了一系列受监督的机器学习技术，为投诉分配严重性标签，可用于确定哪些收到的投诉最需要关注的优先级。我们根据投诉的书面内容研究了几个特征，包括情绪分析，以确定哪些对严重性预测有帮助。最后，我们展示了如何将这些严重性预测和对投诉数据库和列出的医疗保健提供者及其组织特定投诉的自动关键字分析结合起来，以确定每个组织的平均投诉严重程度。结果：使用词袋特征表示的直接文本分类方法最适用于投诉的严重程度预测。我们在保留的测试集上获得了 87%-93%（3319 个投诉中的 2658-2990 个）的准确率，在严重投诉上获得了 45%-51% 的 F1 分数。偏斜的类分布导致只有合理的召回率 (47%-54%) 和准确率 (44%-49%) 分数。使用情绪分析进行严重性预测没有帮助。通过将预测的严重程度结果与自动关键字分析相结合，我们确定了几个可能存在患者安全问题的医疗保健提供者。结论：用于分析平民投诉的文本挖掘技术可以支持检查员。它们可以自动预测投诉的严重程度，也可以用于关键字分析。

这只是摘要。阅读 JMIR 网站上的完整文章。JMIR 是互联网时代电子健康和医疗保健领域领先的开放获取期刊。