The purpose of this study was to evaluate and compare the diagnostic performances of the deep convolutional neural network (CNN) and expert radiologists for differentiating thyroid nodules on ultrasonography (US), and to validate the results in multicenter data sets. This multicenter retrospective study collected 15,375 US images of thyroid nodules for algorithm development (n = 13,560, Severance Hospital, SH training set), the internal test (n = 634, SH test set), and the external test (n = 781, Samsung Medical Center, SMC set; n = 200, CHA Bundang Medical Center, CBMC set; n = 200, Kyung Hee University Hospital, KUH set). Two individual CNNs and two classification ensembles (CNNE1 and CNNE2) were tested to differentiate malignant and benign thyroid nodules. CNNs demonstrated high area under the curves (AUCs) to diagnose malignant thyroid nodules (0.898–0.937 for the internal test set and 0.821–0.885 for the external test sets). AUC was significantly higher for CNNE2 than radiologists in the SH test set (0.932 vs. 0.840, P < 0.001). AUC was not significantly different between CNNE2 and radiologists in the external test sets (P = 0.113, 0.126, and 0.690). CNN showed diagnostic performances comparable to expert radiologists for differentiating thyroid nodules on US in both the internal and external test sets.

本研究的目的是评估和比较深度卷积神经网络 (CNN) 和放射科专家在超声检查 (US) 上区分甲状腺结节的诊断性能，并在多中心数据集中验证结果。这项多中心回顾性研究收集了 15,375 张美国甲状腺结节图像，用于算法开发（n = 13,560，Severance 医院，SH 训练集）、内部测试（n = 634，SH 测试集）和外部测试（n = 781，三星医疗中心，SMC 设置；n = 200，CHA Bundang 医疗中心，CBMC 设置；n = 200，庆熙大学医院，KUH 设置）。测试了两个单独的 CNN 和两个分类集合（CNNE1 和 CNNE2）以区分恶性和良性甲状腺结节。CNN 显示出诊断恶性甲状腺结节的高曲线下面积 (AUC)（内部测试集为 0.898-0.937，外部测试集为 0.821-0.885）。在 SH 测试集中，CNNE2 的 AUC 显着高于放射科医生（0.932 对 0.840，P  < 0.001)。在外部测试集中，CNNE2 和放射科医生之间的 AUC 没有显着差异（P  = 0.113、0.126和 0.690）。CNN 在内部和外部测试集上显示出可与专家放射科医师相媲美的诊断性能，可在 US 上区分甲状腺结节。