Abstract
Early and automatic diagnosis of Solitary Pulmonary Nodules (SPN) in Computed Tomography (CT) chest scans can provide early treatment for patients with lung cancer, as well as doctor liberation from time-consuming procedures. The purpose of this study is the automatic and reliable characterization of SPNs in CT scans extracted from Positron Emission Tomography and Computer Tomography (PET/CT) system. To achieve the aforementioned task, Deep Learning with Convolutional Neural Networks (CNN) is applied. The strategy of training specific CNN architectures from scratch and the strategy of transfer learning, by utilizing state-of-the-art pre-trained CNNs, are compared and evaluated. To enhance the training sets, data augmentation is performed. The publicly available database of CT scans, named as Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI), is also utilized to further expand the training set and is added to the PET/CT dataset. The results highlight the effectiveness of transfer learning and data augmentation for the classification task of small datasets. The best accuracy obtained on the PET/CT dataset reached 94%, utilizing a modification proposal of a state-of-the-art CNN, called VGG16, and enhancing the training set with LIDC-IDRI dataset. Besides, the proposed modification outperforms in terms of sensitivity several similar researches, which exploit the benefits of transfer learning.
Graphical abstract
Overview of the experiment setup. The two datasets containing nodule representations are combined to evaluate the effectiveness of transfer learning over the traditional approach of training Convolutional Neural Networks from scratch.
Similar content being viewed by others
References
Zia ur Rehman M, Javaid M, Shah SIA, Gilani SO, Jamil M, Butt SI, (2018) An appraisal of nodules detection techniques for lung cancer in CT images. Biomed Signal Process Control 41:140–151
Bruno MA, Walker EA, Abujudeh HH (2015) Understanding and confronting our mistakes: the epidemiology of error in radiology and strategies for error reduction. Radiographics 35:1668–1676
Wang X, Mao K, Wang L, Yang P, Lu D, He P (2019) An appraisal of lung nodules automatic classification algorithms for CT images. Sensors 19:194
Wu B, Zhou Z, Wang J, Wang Y (2018) Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction. arXiv preprint arXiv.1802.03584
LeCun Y, Kavukcuoglu K, Farabet C (2010) Convolutional networks and applications in vision. In proceedings of 2010 IEEE International Symposium on Circuits and Systems, Paris, France. p. 253–6
Zhu W, Liu C, Fan W, Xie X (2018) DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification. arXiv preprint arXiv.1801.09555
Russakovsky O, Deng J, et al (2015) ImageNet large scale visual recognition challenge. arXiv preprint arXiv.14090.575
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv.1409.1556
Howard AG, Zhu M, et al (2017) MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv.1704.04861
Zhang G, Yang Z et al (2019) An appraisal of nodule diagnosis for lung cancer in CT images. J Med Syst 43:181
Armato SG, McLennan G et al (2011) The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans: the LIDC/IDRI thoracic CT database of lung nodules. Med Phys 38:915–931
Chen G, Zhang J et al (2019) Identification of pulmonary nodules via CT images with hierarchical fully convolutional networks. Med Biol Eng Compu 57(7):1567–1580
Pang S, Du A et al (2019) A novel fused convolutional neural network for biomedical image classification. Med Biol Eng Compu 57(1):107–121
Shin H-C, Roth HR et al (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35:1285–1298
Zhao X, Liu L et al (2018) Agile convolutional neural network for pulmonary nodule classification using CT images. Int J CARS 13:585–595
Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359
Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60:84–90
Shen W, Zhou M et al (2017) Multi-crop Convolutional Neural Networks for lung nodule malignancy suspiciousness classification. Pattern Recogn 61:663–673
Song Q, Zhao L et al (2017) Using deep learning for classification of lung nodules on computed tomography images. J Healthc Eng 2017:1–7
Causey JL, Zhang J et al (2018) Highly accurate model for prediction of lung nodule malignancy with CT scans. Sci Rep 8:9286
Dey R, Lu Z, Hong Y (2018) Diagnostic classification of lung nodules using 3D neural networks. arXiv preprint arXiv.1803.07192
Huang G, Liu Z, et al (2018) Densely Connected Convolutional Networks. arXiv preprint arXiv.1608.06993
Nobrega RV, Peixoto SA et al (2018) Lung Nodule Classification via Deep Transfer Learning in CT Lung Images. IEEE 31st International Symposium on Computer-Based Medical Systems (CBMS). Karlstad, Sweden, pp 244–249
Zhao X, Qi S et al (2019) Deep CNN models for pulmonary nodule classification: model modification, model integration, and transfer learning. XST 27:615–629
Xie Y, Xia Y, et al (2017) Transferable Multi-model Ensemble for Benign-Malignant Lung Nodule Classification on Chest CT. In Descoteaux M, Maier-Hein L, Franz A, Jannin P, Collins DL, Duchesne S (ed) Medical Image Computing and Computer Assisted Intervention − MICCAI 2017, vol. 10435, Springer International Publishing, p. 656–64
Pintelas E, Livieris IE, Pintelas P (2020) A Grey-Box Ensemble Model Exploiting Black-Box Accuracy and White-Box Intrinsic Interpretability. Algorithms 13(1):17
Pintelas E, Liaskos M, Livieris IE, Kotsiantis S, Pintelas P (2020) Explainable machine learning framework for image classification problems: case study on Glioma cancer prediction. J Imaging, 6(6), 37, ID: jimaging-805312, https://www.mdpi.com/journal/jimaging/special_issues/dlmia
Oquab M, Bottou L, Laptev I, Sivic J (2014) Learning and transferring mid-level image representations using convolutional neural networks. In proceedings of the 2014 IEEE conference on computer vision and pattern recognition, Columbus, OH, USA, p. 1717–1724
Kwasigroch A, Mikolajczyk A, Grochowski M (2017) Deep neural networks approach to skin lesions classification — a comparative analysis. In proceedings of the 22nd International Conference on Methods and Models in Automation and Robotics (MMAR), Miedzyzdroje, Poland, p. 1069–74
Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF, (ed) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, vol. 9351, Springer International Publishing, p. 234–41
Nibali A, He Z, Wollersheim D (2017) Pulmonary nodule classification with deep residual networks. Int J CARS 12:1799–1808
Clark K, Vendt B et al (2013) (2013) The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 26:1045–1057
Chollet F (2017) Xception: Deep Learning with Depthwise Separable Convolutions. arXiv preprint arXiv.1610.02357
Cheng JZ, Ni D et al (2016) Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans. Sci Rep 6:244–254
Wei G, Ma H et al (2018) Lung nodule classification using local kernel regression models with out-of-sample extension. Biomed Signal Process Control 40:1–9
Bayrakdar ME (2019) Priority based health data monitoring with IEEE 802 11af technology in wireless medical sensor networks. Med Biol Eng Comput 57(12):2757–2769
Michail CM, Agavanakis KN, Karpetas GE et al (2019) Information content in nuclear medicine imaging. Energy Procedia 157:1517–1524. https://doi.org/10.1016/j.egypro.2018.11.317
Bayrakdar ME (2019) Fuzzy logic based coordinator node selection approach in wireless medical sensor networks. In 2019 4th International Conference on Computer Science and Engineering (UBMK) (pp. 340–343). Presented at the 2019 4th International Conference on Computer Science and Engineering (UBMK), Samsun, Turkey: IEEE.
Mohammed Z, Abdulla A (2020) Thresholding-based white blood cells segmentation from microscopic blood images. UHD J Sci Technol 4(1):9
Goodfellow I, Pouget-Abadie J, et al (2014) Generative adversarial nets. In Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ, (ed) Advances in neural information processing systems 27, Curran Associates, p. 2672–2680
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Apostolopoulos, I.D., Pintelas, E.G., Livieris, I.E. et al. Automatic classification of solitary pulmonary nodules in PET/CT imaging employing transfer learning techniques. Med Biol Eng Comput 59, 1299–1310 (2021). https://doi.org/10.1007/s11517-021-02378-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11517-021-02378-y