BACKGROUND Automated analysis of cardiac structure and function using machine learning (ML) has great potential, but is currently hindered by poor generalizability. Comparison is traditionally against clinicians as a reference, ignoring inherent human inter- and intraobserver error, and ensuring that ML cannot demonstrate superiority. Measuring precision (scan:rescan reproducibility) addresses this. We compared precision of ML and humans using a multicenter, multi-disease, scan:rescan cardiovascular magnetic resonance data set. METHODS One hundred ten patients (5 disease categories, 5 institutions, 2 scanner manufacturers, and 2 field strengths) underwent scan:rescan cardiovascular magnetic resonance (96% within one week). After identification of the most precise human technique, left ventricular chamber volumes, mass, and ejection fraction were measured by an expert, a trained junior clinician, and a fully automated convolutional neural network trained on 599 independent multicenter disease cases. Scan:rescan coefficient of variation and 1000 bootstrapped 95% CIs were calculated and compared using mixed linear effects models. RESULTS Clinicians can be confident in detecting a 9% change in left ventricular ejection fraction, with greater than half of coefficient of variation attributable to intraobserver variation. Expert, trained junior, and automated scan:rescan precision were similar (for left ventricular ejection fraction, coefficient of variation 6.1 [5.2%-7.1%], P=0.2581; 8.3 [5.6%-10.3%], P=0.3653; 8.8 [6.1%-11.1%], P=0.8620). Automated analysis was 186× faster than humans (0.07 versus 13 minutes). CONCLUSIONS Automated ML analysis is faster with similar precision to the most precise human techniques, even when challenged with real-world scan:rescan data. Assessment of multicenter, multi-vendor, multi-field strength scan:rescan data (available at www.thevolumesresource.com) permits a generalizable assessment of ML precision and may facilitate direct translation of ML to clinical practice.

中文翻译：

---

一项多中心、扫描-重新扫描、人类和机器学习 CMR 研究，用于测试成像生物标志物分析的普遍性和精度。


背景技术使用机器学习（ML）对心脏结构和功能进行自动分析具有巨大的潜力，但目前由于通用性差而受到阻碍。传统上，比较是以临床医生为参考，忽略了固有的人类观察者间和观察者内的错误，并确保机器学习无法表现出优越性。测量精度（扫描：重新扫描再现性）解决了这个问题。我们使用多中心、多疾病、扫描：重新扫描心血管磁共振数据集比较了机器学习和人类的精度。方法 110 名患者（5 个疾病类别、5 个机构、2 个扫描仪制造商和 2 个场强）接受了扫描：重新扫描心血管磁共振（96% 在一周内）。在确定了最精确的人体技术后，由专家、经过培训的初级临床医生和经过 599 个独立多中心疾病病例训练的全自动卷积神经网络测量左心室容积、质量和射血分数。使用混合线性效应模型计算并比较扫描：重新扫描变异系数和 1000 个自举 95% CI。结果 临床医生可以自信地检测到左心室射血分数 9% 的变化，其中一半以上的变异系数归因于观察者内部的变异。专家、训练有素的初级扫描和自动扫描：重新扫描精度相似（左心室射血分数，变异系数 6.1 [5.2%-7.1%]，P=0.2581；8.3 [5.6%-10.3%]，P=0.3653；8.8 [6.1%-11.1%]，P=0.8620）。自动分析比人类快 186 倍（0.07 分钟对 13 分钟）。结论 即使面临现实世界扫描：重新扫描数据的挑战，自动化 ML 分析速度更快，精度与最精确的人类技术相似。 多中心、多供应商、多场强扫描的评估：重新扫描数据（可在 www.thevolumesresource.com 上获取）允许对 ML 精度进行普遍评估，并可能有助于将 ML 直接转化为临床实践。