Robustness is an important aspect when evaluating a method of medical image analysis. In this study, we investigated the robustness of a deep learning (DL)-based lung-nodule classification model for CT images with respect to noise perturbations. A deep neural network (DNN) was established to classify 3D CT images of lung nodules into malignant or benign groups. The established DNN was able to predict malignancy rate of lung nodules based on CT images, achieving the area under the curve of 0.91 for the testing dataset in a tenfold cross validation as compared to radiologists’ prediction. We then evaluated its robustness against noise perturbations. We added to the input CT images noise signals generated randomly or via an optimization scheme using a realistic noise model based on a noise power spectrum for a given mAs level, and monitored the DNN’s output. The results showed that the CT noise was able to affect the prediction results of the established DNN model. With random noise perturbations at 100 mAs, DNN’s predictions for 11.2% of training data and 17.4% of testing data were successfully altered by at least once. The percentage increased to 23.4% and 34.3%, respectively, for optimization-based perturbations. We further evaluated robustness of models with different architectures, parameters, number of output labels, etc, and robustness concern was found in these models to different degrees. To improve model robustness, we empirically proposed an adaptive training scheme. It fine-tuned the DNN model by including perturbations in the training dataset that successfully altered the DNN’s perturbations. The adaptive scheme was repeatedly performed to gradually improve DNN’s robustness. The numbers of perturbations at 100 mAs affecting DNN’s predictions were reduced to 10.8% for training and 21.1% for testing by the adaptive training scheme after two iterations. Our study illustrated that robustness may potentially be a concern for an exemplary DL-based lung-nodule classification model for CT images, indicating the needs for evaluating and ensuring model robustness when developing similar models. The proposed adaptive training scheme may be able to improve model robustness.

在评估医学图像分析方法时，鲁棒性是一个重要方面。在这项研究中，我们研究了基于深度学习 (DL) 的 CT 图像肺结节分类模型在噪声扰动方面的稳健性。建立深度神经网络（DNN）将肺结节的 3D CT 图像分类为恶性或良性组。建立的 DNN 能够根据 CT 图像预测肺结节的恶性率，与放射科医生的预测相比，在十倍交叉验证中测试数据集的曲线下面积达到 0.91。然后我们评估了它对噪声扰动的鲁棒性。我们将随机生成的噪声信号添加到输入 CT 图像中，或者通过使用基于给定 mAs 级别的噪声功率谱的真实噪声模型的优化方案来添加噪声信号，并监控 DNN 的输出。结果表明，CT噪声能够影响所建立的DNN模型的预测结果。在 100 mA 的随机噪声扰动下，DNN 对 11.2% 训练数据和 17.4% 测试数据的预测至少成功改变了一次。对于基于优化的扰动，该百分比分别增加到 23.4% 和 34.3%。我们进一步评估了具有不同架构、参数、输出标签数量等的模型的鲁棒性，发现这些模型不同程度地存在鲁棒性问题。为了提高模型的鲁棒性，我们根据经验提出了一种自适应训练方案。它通过在训练数据集中包含扰动来微调 DNN 模型，从而成功改变 DNN 的扰动。反复执行自适应方案以逐渐提高DNN的鲁棒性。经过两次迭代后，通过自适应训练方案，影响 DNN 预测的 100 mA 扰动数量在训练时减少至 10.8%，在测试时减少至 21.1%。我们的研究表明，稳健性可能是基于深度学习的 CT 图像肺结节分类模型的一个潜在问题，这表明在开发类似模型时需要评估和确保模型的稳健性。所提出的自适应训练方案可能能够提高模型的鲁棒性。