Abstract

Objective

Toolkits are an important knowledge translation strategy for implementing digital health. We studied how toolkits for the implementation and evaluation of digital health were developed, tested, and reported.

Materials and Methods

We conducted a systematic review of toolkits that had been used, field tested or evaluated in practice, and published in the English language from 2009 to July 2019. We searched several electronic literature sources to identify both peer-reviewed and gray literature, and records were screened as per systematic review conventions.

Results

Thirteen toolkits were eventually identified, all of which were developed in North America, Europe, or Australia. All reported their intended purpose, as well as their development process. Eight of the 13 toolkits involved a literature review, 3 did not, and 2 were unclear. Twelve reported an underlying conceptual framework, theory, or model: 3 cited the normalization process theory and 3 others cited the World Health Organization and International Telecommunication Union eHealth Strategy. Seven toolkits were reportedly evaluated, but details were unavailable. Forty-three toolkits were excluded for lack of field-testing.

Discussion

Despite a plethora of published toolkits, few were tested, and even fewer were evaluated. Methodological rigor was of concern, as several did not include an underlying conceptual framework, literature review, or evaluation and refinement in real-world settings. Reporting was often inconsistent and unclear, and toolkits rarely reported being evaluated.

Conclusion

Greater attention needs to be paid to rigor and reporting when developing, evaluating, and reporting toolkits for implementing and evaluating digital health so that they can effectively function as a knowledge translation strategy.

中文翻译：

---

实施和评估数字健康的工具包：对严谨性和报告的系统评价

摘要

客观的

工具包是实施数字健康的重要知识转化策略。我们研究了如何开发、测试和报告用于实施和评估数字健康的工具包。

材料和方法

我们对 2009 年至 2019 年 7 月期间在实践中使用、现场测试或评估并以英语出版的工具包进行了系统审查。我们搜索了多个电子文献资源，以识别同行评议和灰色文献，记录被根据系统评价公约筛选。

结果

最终确定了 13 个工具包，所有这些工具包都是在北美、欧洲或澳大利亚开发的。所有人都报告了他们的预期目的以及他们的开发过程。13 个工具包中有 8 个涉及文献综述，3 个没有，2 个不清楚。12 份报告了基本的概念框架、理论或模型：3 份引用了正常化过程理论，另外 3 份引用了世界卫生组织和国际电信联盟电子卫生战略。据报道，评估了七个工具包，但无法获得详细信息。由于缺乏现场测试，43 个工具包被排除在外。

讨论

尽管有大量已发布的工具包，但测试的很少，评估的更少。方法的严谨性令人担忧，因为其中一些不包括潜在的概念框架、文献综述或现实世界环境中的评估和改进。报告往往不一致和不清楚，工具包很少报告被评估。

结论

在开发、评估和报告用于实施和评估数字健康的工具包时，需要更加注意严谨性和报告，以便它们能够有效地作为知识转化策略发挥作用。