BACKGROUND The interest in information extraction from clinical reports for secondary data use is increasing. But experience with the productive use of information extraction processes over time is scarce. A clinical data warehouse has been in use at our university hospital for several years, which also provides an information extraction of echocardiography reports developed for general use. OBJECTIVES This study aims to illustrate the difficulties encountered, while using data from a preexisting information extraction process for a large clinical study. To compare the data from the preexisting process with the data obtained from a specially developed process designed to improve the quality and completeness of the study data. METHODS We extracted the echocardiography variables for 440 patients from the general-use information extraction of the data warehouse (678 reports). Then we developed an information extraction process for the same variables but specifically for this study, with the aim to extract as much information as possible from the text. The extracted data of both processes were compared with a newly created gold standard defined by a cardiologist with long-standing experience in heart failure. RESULTS Among 57 echocardiography variables considered relevant for the study, 50 were documented in the routine text reports and could be extracted. Twenty of the required variables were not provided by the general-use extraction process, some others were not provided correctly. The median macro F1-score (precision, recall) across the 30 variables for which values were extracted was 0.81 (0.94, 0.77). Across all 50 variables, as relevant for the study, median macro F1-score was only 0.49 (0.56, 0.46). Employing the study-specific approach considerably improved the quality and completeness of the variables, resulting in F1-scores of 0.97 (0.98, 0.96) across all variables. CONCLUSION Data from information extractions can be used for large clinical studies. However, preexisting information extraction processes should be treated with caution, as the time and effort spent defining each variable in the information extraction process may not be clear.

中文翻译：

---

从超声心动图报告中提取信息以进行临床随访研究-将用于数据仓库的通用变量与专门用于研究的变量进行比较。

背景技术从临床报告中提取信息以用于二次数据的兴趣正在增加。但是，随着时间的流逝，缺乏有效使用信息提取过程的经验。临床数据仓库已经在我们的大学医院使用了几年，它还提供了开发用于一般用途的超声心动图报告的信息。目的本研究旨在说明在使用大型临床研究中现有信息提取过程中的数据时遇到的困难。将现有过程中的数据与从专门开发的过程中获得的数据进行比较，该过程旨在提高研究数据的质量和完整性。方法我们从数据仓库的通用信息提取中提取了440名患者的超声心动图变量（678个报告）。然后，我们针对相同的变量（特别是针对本研究）开发了一种信息提取过程，旨在从文本中提取尽可能多的信息。将这两个过程的提取数据与具有长期心衰经验的心脏病专家定义的新创建的金标准进行比较。结果在与该研究相关的57个超声心动图变量中，有50个被记录在常规文本报告中并可以提取。通用提取过程未提供20个所需变量，而其他一些变量未正确提供。中值宏F1得分（精度，回忆）在30个变量中提取的值为0.81（0.94，0.77）。在与研究相关的所有50个变量中，宏F1得分的中位数仅为0.49（0.56，0.46）。采用针对特定研究的方法，可以显着提高变量的质量和完整性，所有变量的F1得分均为0.97（0.98、0.96）。结论信息提取中的数据可用于大型临床研究。但是，应谨慎对待先前存在的信息提取过程，因为在信息提取过程中定义每个变量所花费的时间和精力可能并不明确。采用针对特定研究的方法，可以显着提高变量的质量和完整性，所有变量的F1得分均为0.97（0.98、0.96）。结论信息提取中的数据可用于大型临床研究。但是，应谨慎对待先前存在的信息提取过程，因为在信息提取过程中定义每个变量所花费的时间和精力可能并不明确。采用针对特定研究的方法，可以显着提高变量的质量和完整性，所有变量的F1得分均为0.97（0.98、0.96）。结论信息提取中的数据可用于大型临床研究。但是，应谨慎对待先前存在的信息提取过程，因为在信息提取过程中定义每个变量所花费的时间和精力可能并不明确。