Objective

Evaluating Health Information Technologies (HITs) can be challenging, but studies are necessary so that the most beneficial interventions can be identified. Our objective was to systematically review the available recommendations for improving the methods used in HIT evaluations.

Methods

HIT evaluation frameworks were identified from database (MEDLINE, EMBASE, CINAHL) and grey literature searches. Outcome measures included framework recommendations and characteristics. Recommendations were coded and organised using thematic analysis methods. A scoring instrument was used to measure framework quality.

Results

The search identified 23 frameworks and 272 recommendations. These were organised into five evaluation domains and 42 themes. The themes included recommendations for improving the evaluation of technical aspects of HITs (e.g. describing aspects of HIT functionality) and suggestions for improving the evaluation of complex factors that may influence the overall effects of HITs (e.g. careful reporting of whether the HIT became integrated into existing working patterns). The frameworks were not generally developed in association with healthcare professionals, or with input from patients. The frameworks tended not to have been developed using systematic methods designed to reduce the risk of bias.

Discussion

HIT evaluations are important but they are challenging to conduct and appraise. This review was conducted using systematic methods enabling the organisation of framework recommendations into key themes. These findings may help investigators to successfully plan, conduct and appraise HIT evaluations. The quality appraisal demonstrated that HIT evaluation research may be improved by using more systematic methods and the involvement of participants from a range of differing backgrounds.

中文翻译：

---

评估健康信息技术：对框架建议的系统审查。

目的

评估健康信息技术（HIT）可能具有挑战性，但是必须进行研究才能确定出最有益的干预措施。我们的目标是系统地审查可用于改进HIT评估方法的可用建议。

方法

从数据库（MEDLINE，EMBASE，CINAHL）和灰色文献搜索中确定了HIT评估框架。结果措施包括框架建议和特征。使用主题分析方法对建议进行编码和组织。使用评分工具来衡量框架质量。

结果

搜索确定了23个框架和272条建议。这些活动分为五个评估领域和42个主题。主题包括关于改进对HIT技术方面的评估的建议（例如，描述HIT功能的各个方面）和对改进可能影响HIT总体影响的复杂因素的评估的建议（例如，仔细报告HIT是否已整合到现有技术中）工作模式）。这些框架通常没有与医疗保健专业人员或患者的投入共同开发。往往没有使用旨在减少偏见风险的系统方法来开发框架。

讨论区

HIT评估很重要，但进行和评估却充满挑战。这项审查是使用系统化方法进行的，能够将框架建议组织成关键主题。这些发现可能有助于调查人员成功地计划，实施和评估HIT评估。质量评估表明，通过使用更系统的方法以及来自不同背景的参与者的参与，HIT评估研究可能会得到改善。