While many deep-learning-based computer-aided detection systems (CAD) have been developed and commercialized for abnormality detection in chest radiographs (CXR), their ability to localize a target abnormality is rarely reported. Localization accuracy is important in terms of model interpretability, which is crucial in clinical settings. Moreover, diagnostic performances are likely to vary depending on thresholds which define an accurate localization. In a multi-center, stand-alone clinical trial using temporal and external validation datasets of 1,050 CXRs, we evaluated localization accuracy, localization-adjusted discrimination, and calibration of a commercially available deep-learning-based CAD for detecting consolidation and pneumothorax. The CAD achieved image-level AUROC (95% CI) of 0.960 (0.945, 0.975), sensitivity of 0.933 (0.899, 0.959), specificity of 0.948 (0.930, 0.963), dice of 0.691 (0.664, 0.718), moderate calibration for consolidation, and image-level AUROC of 0.978 (0.965, 0.991), sensitivity of 0.956 (0.923, 0.978), specificity of 0.996 (0.989, 0.999), dice of 0.798 (0.770, 0.826), moderate calibration for pneumothorax. Diagnostic performances varied substantially when localization accuracy was accounted for but remained high at the minimum threshold of clinical relevance. In a separate trial for diagnostic impact using 461 CXRs, the causal effect of the CAD assistance on clinicians’ diagnostic performances was estimated. After adjusting for age, sex, dataset, and abnormality type, the CAD improved clinicians’ diagnostic performances on average (OR [95% CI] = 1.73 [1.30, 2.32]; p < 0.001), although the effects varied substantially by clinical backgrounds. The CAD was found to have high stand-alone diagnostic performances and may beneficially impact clinicians’ diagnostic performances when used in clinical settings.

虽然许多基于深度学习的计算机辅助检测系统 (CAD) 已被开发并商业化，用于胸部 X 光片 (CXR) 的异常检测，但它们定位目标异常的能力却很少被报道。定位精度对于模型可解释性非常重要，这在临床环境中至关重要。此外，诊断性能可能会根据定义准确定位的阈值而变化。在一项使用 1,050 个 CXR 的时间和外部验证数据集的多中心独立临床试验中，我们评估了用于检测实变和气胸的商用基于深度学习的 CAD 的定位精度、定位调整辨别力和校准。 CAD 的图像级 AUROC (95% CI) 为 0.960 (0.945, 0.975)，灵敏度为 0.933 (0.899, 0.959)，特异性为 0.948 (0.930, 0.963)，骰子为 0.691 (0.664, 0.718)，中等校准合并，图像级 AUROC 为 0.978 (0.965, 0.991)，敏感性为 0.956 (0.923, 0.978)，特异性为 0.996 (0.989, 0.999)，骰子为 0.798 (0.770, 0.826)，对气胸进行中等校准。当考虑定位精度时，诊断性能差异很大，但在临床相关性的最低阈值下仍然很高。在使用 461 个 CXR 进行诊断影响的单独试验中，评估了 CAD 辅助对临床医生诊断表现的因果影响。在调整年龄、性别、数据集和异常类型后，CAD 平均提高了临床医生的诊断表现（OR [95% CI] = 1.73 [1.30, 2.32]； p < 0.001），尽管效果因临床而异背景。 研究发现 CAD 具有较高的独立诊断性能，并且在临床环境中使用时可能会对临床医生的诊断性能产生有益的影响。