Artificial intelligence (AI) algorithms continue to rival human performance on a variety of clinical tasks, while their actual impact on human diagnosticians, when incorporated into clinical workflows, remains relatively unexplored. In this study, we developed a deep learning-based assistant to help pathologists differentiate between two subtypes of primary liver cancer, hepatocellular carcinoma and cholangiocarcinoma, on hematoxylin and eosin-stained whole-slide images (WSI), and evaluated its effect on the diagnostic performance of 11 pathologists with varying levels of expertise. Our model achieved accuracies of 0.885 on a validation set of 26 WSI, and 0.842 on an independent test set of 80 WSI. Although use of the assistant did not change the mean accuracy of the 11 pathologists (p = 0.184, OR = 1.281), it significantly improved the accuracy (p = 0.045, OR = 1.499) of a subset of nine pathologists who fell within well-defined experience levels (GI subspecialists, non-GI subspecialists, and trainees). In the assisted state, model accuracy significantly impacted the diagnostic decisions of all 11 pathologists. As expected, when the model’s prediction was correct, assistance significantly improved accuracy (p = 0.000, OR = 4.289), whereas when the model’s prediction was incorrect, assistance significantly decreased accuracy (p = 0.000, OR = 0.253), with both effects holding across all pathologist experience levels and case difficulty levels. Our results highlight the challenges of translating AI models into the clinical setting, and emphasize the importance of taking into account potential unintended negative consequences of model assistance when designing and testing medical AI-assistance tools.

人工智能 (AI) 算法在各种临床任务上继续与人类表现相媲美，而当它们纳入临床工作流程时，它们对人类诊断医生的实际影响仍然相对未经探索。在这项研究中，我们开发了一种基于深度学习的助手，帮助病理学家在苏木精和伊红染色的全切片图像 (WSI) 上区分原发性肝癌的两种亚型：肝细胞癌和胆管癌，并评估其对诊断的影响。 11 名具有不同专业水平的病理学家的表现。我们的模型在 26 WSI 的验证集上实现了 0.885 的准确率，在 80 WSI 的独立测试集上实现了 0.842 的准确率。虽然助手的使用并没有改变 11 名病理学家的平均准确度（p = 0.184，OR = 1.281），但它显着提高了 9 名病理学家子集的 准确度（p = 0.045，OR = 1.499）。明确的经验水平（GI 亚专家、非 GI 亚专家和实习生）。在辅助状态下，模型准确性显着影响所有 11 名病理学家的诊断决策。正如预期的那样，当模型预测正确时，辅助显着提高了准确性（p  = 0.000，OR = 4.289），而当模型预测错误时，辅助显着降低了准确性（p  = 0.000，OR = 0.253），两种效应均成立涵盖所有病理学家经验水平和病例难度水平。我们的结果强调了将人工智能模型转化为临床环境的挑战，并强调在设计和测试医疗人工智能辅助工具时考虑模型辅助潜在的意外负面后果的重要性。