Skip to main content
Log in

Automatic classification of solitary pulmonary nodules in PET/CT imaging employing transfer learning techniques

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Early and automatic diagnosis of Solitary Pulmonary Nodules (SPN) in Computed Tomography (CT) chest scans can provide early treatment for patients with lung cancer, as well as doctor liberation from time-consuming procedures. The purpose of this study is the automatic and reliable characterization of SPNs in CT scans extracted from Positron Emission Tomography and Computer Tomography (PET/CT) system. To achieve the aforementioned task, Deep Learning with Convolutional Neural Networks (CNN) is applied. The strategy of training specific CNN architectures from scratch and the strategy of transfer learning, by utilizing state-of-the-art pre-trained CNNs, are compared and evaluated. To enhance the training sets, data augmentation is performed. The publicly available database of CT scans, named as Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI), is also utilized to further expand the training set and is added to the PET/CT dataset. The results highlight the effectiveness of transfer learning and data augmentation for the classification task of small datasets. The best accuracy obtained on the PET/CT dataset reached 94%, utilizing a modification proposal of a state-of-the-art CNN, called VGG16, and enhancing the training set with LIDC-IDRI dataset. Besides, the proposed modification outperforms in terms of sensitivity several similar researches, which exploit the benefits of transfer learning.

Graphical abstract

Overview of the experiment setup. The two datasets containing nodule representations are combined to evaluate the effectiveness of transfer learning over the traditional approach of training Convolutional Neural Networks from scratch.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Zia ur Rehman M, Javaid M, Shah SIA, Gilani SO, Jamil M, Butt SI, (2018) An appraisal of nodules detection techniques for lung cancer in CT images. Biomed Signal Process Control 41:140–151

    Article  Google Scholar 

  2. Bruno MA, Walker EA, Abujudeh HH (2015) Understanding and confronting our mistakes: the epidemiology of error in radiology and strategies for error reduction. Radiographics 35:1668–1676

    Article  Google Scholar 

  3. Wang X, Mao K, Wang L, Yang P, Lu D, He P (2019) An appraisal of lung nodules automatic classification algorithms for CT images. Sensors 19:194

    Article  Google Scholar 

  4. Wu B, Zhou Z, Wang J, Wang Y (2018) Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction. arXiv preprint arXiv.1802.03584

  5. LeCun Y, Kavukcuoglu K, Farabet C (2010) Convolutional networks and applications in vision. In proceedings of 2010 IEEE International Symposium on Circuits and Systems, Paris, France. p. 253–6

  6. Zhu W, Liu C, Fan W, Xie X (2018) DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification. arXiv preprint arXiv.1801.09555

  7. Russakovsky O, Deng J, et al (2015) ImageNet large scale visual recognition challenge. arXiv preprint arXiv.14090.575

  8. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv.1409.1556

  9. Howard AG, Zhu M, et al (2017) MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv.1704.04861

  10. Zhang G, Yang Z et al (2019) An appraisal of nodule diagnosis for lung cancer in CT images. J Med Syst 43:181

    Article  Google Scholar 

  11. Armato SG, McLennan G et al (2011) The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans: the LIDC/IDRI thoracic CT database of lung nodules. Med Phys 38:915–931

    Article  Google Scholar 

  12. Chen G, Zhang J et al (2019) Identification of pulmonary nodules via CT images with hierarchical fully convolutional networks. Med Biol Eng Compu 57(7):1567–1580

    Article  Google Scholar 

  13. Pang S, Du A et al (2019) A novel fused convolutional neural network for biomedical image classification. Med Biol Eng Compu 57(1):107–121

    Article  Google Scholar 

  14. Shin H-C, Roth HR et al (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35:1285–1298

    Article  Google Scholar 

  15. Zhao X, Liu L et al (2018) Agile convolutional neural network for pulmonary nodule classification using CT images. Int J CARS 13:585–595

    Article  Google Scholar 

  16. Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359

    Article  Google Scholar 

  17. Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60:84–90

    Article  Google Scholar 

  18. Shen W, Zhou M et al (2017) Multi-crop Convolutional Neural Networks for lung nodule malignancy suspiciousness classification. Pattern Recogn 61:663–673

    Article  Google Scholar 

  19. Song Q, Zhao L et al (2017) Using deep learning for classification of lung nodules on computed tomography images. J Healthc Eng 2017:1–7

    Article  Google Scholar 

  20. Causey JL, Zhang J et al (2018) Highly accurate model for prediction of lung nodule malignancy with CT scans. Sci Rep 8:9286

    Article  Google Scholar 

  21. Dey R, Lu Z, Hong Y (2018) Diagnostic classification of lung nodules using 3D neural networks. arXiv preprint arXiv.1803.07192

  22. Huang G, Liu Z, et al (2018) Densely Connected Convolutional Networks. arXiv preprint arXiv.1608.06993

  23. Nobrega RV, Peixoto SA et al (2018) Lung Nodule Classification via Deep Transfer Learning in CT Lung Images. IEEE 31st International Symposium on Computer-Based Medical Systems (CBMS). Karlstad, Sweden, pp 244–249

    Chapter  Google Scholar 

  24. Zhao X, Qi S et al (2019) Deep CNN models for pulmonary nodule classification: model modification, model integration, and transfer learning. XST 27:615–629

    Article  Google Scholar 

  25. Xie Y, Xia Y, et al (2017) Transferable Multi-model Ensemble for Benign-Malignant Lung Nodule Classification on Chest CT. In Descoteaux M, Maier-Hein L, Franz A, Jannin P, Collins DL, Duchesne S (ed) Medical Image Computing and Computer Assisted Intervention − MICCAI 2017, vol. 10435, Springer International Publishing, p. 656–64

  26. Pintelas E, Livieris IE, Pintelas P (2020) A Grey-Box Ensemble Model Exploiting Black-Box Accuracy and White-Box Intrinsic Interpretability. Algorithms 13(1):17

    Article  Google Scholar 

  27. Pintelas E, Liaskos M, Livieris IE, Kotsiantis S, Pintelas P (2020) Explainable machine learning framework for image classification problems: case study on Glioma cancer prediction. J Imaging, 6(6), 37, ID: jimaging-805312, https://www.mdpi.com/journal/jimaging/special_issues/dlmia

  28. Oquab M, Bottou L, Laptev I, Sivic J (2014) Learning and transferring mid-level image representations using convolutional neural networks. In proceedings of the 2014 IEEE conference on computer vision and pattern recognition, Columbus, OH, USA, p. 1717–1724

  29. Kwasigroch A, Mikolajczyk A, Grochowski M (2017) Deep neural networks approach to skin lesions classification — a comparative analysis. In proceedings of the 22nd International Conference on Methods and Models in Automation and Robotics (MMAR), Miedzyzdroje, Poland, p. 1069–74

  30. Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF, (ed) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, vol. 9351, Springer International Publishing, p. 234–41

  31. Nibali A, He Z, Wollersheim D (2017) Pulmonary nodule classification with deep residual networks. Int J CARS 12:1799–1808

    Article  Google Scholar 

  32. Clark K, Vendt B et al (2013) (2013) The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 26:1045–1057

    Article  Google Scholar 

  33. Chollet F (2017) Xception: Deep Learning with Depthwise Separable Convolutions. arXiv preprint arXiv.1610.02357

  34. Cheng JZ, Ni D et al (2016) Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans. Sci Rep 6:244–254

    Google Scholar 

  35. Wei G, Ma H et al (2018) Lung nodule classification using local kernel regression models with out-of-sample extension. Biomed Signal Process Control 40:1–9

    Article  CAS  Google Scholar 

  36. Bayrakdar ME (2019) Priority based health data monitoring with IEEE 802 11af technology in wireless medical sensor networks. Med Biol Eng Comput 57(12):2757–2769

    Article  Google Scholar 

  37. Michail CM, Agavanakis KN, Karpetas GE et al (2019) Information content in nuclear medicine imaging. Energy Procedia 157:1517–1524. https://doi.org/10.1016/j.egypro.2018.11.317

    Article  CAS  Google Scholar 

  38. Bayrakdar ME (2019) Fuzzy logic based coordinator node selection approach in wireless medical sensor networks. In 2019 4th International Conference on Computer Science and Engineering (UBMK) (pp. 340–343). Presented at the 2019 4th International Conference on Computer Science and Engineering (UBMK), Samsun, Turkey: IEEE.

  39. Mohammed Z, Abdulla A (2020) Thresholding-based white blood cells segmentation from microscopic blood images. UHD J Sci Technol 4(1):9

    Article  Google Scholar 

  40. Goodfellow I, Pouget-Abadie J, et al (2014) Generative adversarial nets. In Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ, (ed) Advances in neural information processing systems 27, Curran Associates, p. 2672–2680

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ioannis D. Apostolopoulos.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Apostolopoulos, I.D., Pintelas, E.G., Livieris, I.E. et al. Automatic classification of solitary pulmonary nodules in PET/CT imaging employing transfer learning techniques. Med Biol Eng Comput 59, 1299–1310 (2021). https://doi.org/10.1007/s11517-021-02378-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-021-02378-y

Keywords

Navigation