As patient health information is highly regulated due to privacy concerns, most machine learning (ML)-based healthcare studies are unable to test on external patient cohorts, resulting in a gap between locally reported model performance and cross-site generalizability. Different approaches have been introduced for developing models across multiple clinical sites, however less attention has been given to adopting ready-made models in new settings. We introduce three methods to do this—(1) applying a ready-made model “as-is” (2); readjusting the decision threshold on the model’s output using site-specific data and (3); finetuning the model using site-specific data via transfer learning. Using a case study of COVID-19 diagnosis across four NHS Hospital Trusts, we show that all methods achieve clinically-effective performances (NPV > 0.959), with transfer learning achieving the best results (mean AUROCs between 0.870 and 0.925). Our models demonstrate that site-specific customization improves predictive performance when compared to other ready-made approaches.

由于隐私问题，患者健康信息受到高度监管，大多数基于机器学习 (ML) 的医疗保健研究无法对外部患者群组进行测试，从而导致本地报告的模型性能与跨站点通用性之间存在差距。已经引入了不同的方法来开发跨多个临床站点的模型，但是对在新环境中采用现成模型的关注较少。我们介绍了三种方法来做到这一点——（1）“按原样”应用现成的模型（2）；使用特定地点的数据和 (3) 重新调整模型输出的决策阈值；通过迁移学习使用特定于站点的数据对模型进行微调。通过四个 NHS 医院信托基金的 COVID-19 诊断案例研究，我们表明所有方法都达到了临床有效的性能（NPV > 0.959），迁移学习取得最佳结果（平均 AUROC 介于 0.870 和 0.925 之间）。我们的模型表明，与其他现成方法相比，特定于站点的定制可以提高预测性能。