Skip to main content
SpringerLink
Log in

Predicting Unnecessary Nodule Biopsies from a Small, Unbalanced, and Pathologically Proven Dataset by Transfer Learning

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

This study explores an automatic diagnosis method to predict unnecessary nodule biopsy from a small, unbalanced, and pathologically proven database. The automatic diagnosis method is based on a convolutional neural network (CNN) model. Because of the small and unbalanced samples, the presented method aims to improve the transfer learning capability via the VGG16 architecture and optimize the related transfer learning parameters. For comparison purpose, a traditional machine learning method is implemented, which extracts the texture features and classifies the features by support vector machine (SVM). The database includes 68 biopsied nodules, 16 are pathologically proven benign and the remaining 52 are malignant. To consider the volumetric data by the CNN model, each image slice from each nodule volume is selected randomly until all image slices of each nodule are utilized. The leave-one-out and 10-folder cross validations are applied to train and test the randomly selected 68 image slices (one image slice from one nodule) in each experiment, respectively. The averages over all the experimental outcomes are the final results. The experiments revealed that the features from both the medical and the natural images share the similarity of focusing on simpler and less-abstract objects, leading to the conclusion that not the more the transfer convolutional layers, the better the classification results. Transfer learning from other larger datasets can supply additional information to small and unbalanced datasets to improve the classification performance. The presented method has shown the potential to adapt CNN architecture to improve the prediction of unnecessary nodule biopsy from small, unbalanced, and pathologically proven volumetric dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Siegel RL, Miller KD, Jemal A: Cancer statistics, 2016. CA: A Cancer Journal for Clinicians 67(1):7–30, 2016. https://doi.org/10.3322/caac.21332

    Google Scholar 

  2. Trama A, Botta L, Foschi R, Ferrari A, Stiller C, Desandes E, Maule MM, Merletti F, Gatta G, the EUROCARE-5 working group: Survival of European adolescents and young adults diagnosed with cancer in 2000–07: Population-based data from EUROCARE-5. The Lancet Oncology 17(7):896–906, 2016. https://doi.org/10.1016/S1470-2045(16)00162-5

    Article  Google Scholar 

  3. Allemani C, Matsuda T, Carlo VD, Harewood R, Matz M, Nikšić M, Bonaventure A, Valkov M, Johnson CJ, Estève J, Ogunbiyi OJ, Silva GAE, Chen WQ, Eser S, Engholm G, Stiller CA, Monnereau A, Woods RR, Visser O, Lim GH, Aitken J, Weir HK, Coleman MP, the CONCORD working group: Global surveillance of trends in cancer survival 2000-14 (CONCORD-3): Analysis of individual records for 37513025 patients diagnosed with one of 18 cancers from 322 population-based registries in 71 countries. The Lancet 391(10125):1023–1075, 2018. https://doi.org/10.1016/S0140-6736(17)33326-3

    Article  Google Scholar 

  4. American Cancer Society: Cancer facts & figures 2019. Atlanta: American Cancer Society, 2019. Available at https://www.cancer.org/research/cancer-facts-statistics/all-cancer-facts-figures/cancer-facts-figures-2019.html

  5. Siegel RL, Miller KD, Jemal A: Cancer statistics, 2019. CA: A Cancer Journal for Clinicians 69(1):7–34, 2019. https://doi.org/10.3322/caac.21551

    Google Scholar 

  6. Gary Clayman. Thyroid nodules: Hyperthyroidism and thyroid Cancer. Endocrineweb, November 27th, 2018. Available at https://www.endocrineweb.com/conditions/thyroid/thyroid-nodules

  7. Iwano S, Nakamura T, Kamioka Y, Ishigaki T: Computer-aided diagnosis: A shape classification of pulmonary nodules imaged by high-resolution CT. Computerized Medical Imaging and Graphics 29(7):565–570, 2005. https://doi.org/10.1016/j.compmedimag.2008.04.001

    Article  Google Scholar 

  8. Saito H, Minamiya Y, Kawai H, Nakagawa T, Ito M, Hosono Y, Motoyama S, Hashimoto M, Ishiyama K, Ogawa JI. Usefulness of circumference difference for estimating the likelihood of malignancy in small solitary pulmonary nodules on CT. Lung Cancer 58(3): 348-354, 2007. https://doi.org/10.1016/j.lungcan.2007.06.018

    Article  Google Scholar 

  9. El–Baz A, Nitzken M, Khalifa F, Elnakib A, Gimel’farb G, Falk R, and El-Ghar MA. 3D shape analysis for early diagnosis of malignant lung nodules. Information Processing in Medical Imaging 2011;22:772–783. https://doi.org/10.1007/978-3-642-22092-0_63

    Google Scholar 

  10. El–Baz A, Nitzken M, Vanbogaert E, Gimel’farb G, Falk R, El-Ghar MA: A novel shape-based diagnostic approach for early diagnosis of lung nodules. 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Chicago, IL, USA, 30 March-2 April, 2011. https://doi.org/10.1109/ISBI.2011.5872373

  11. Yankelevitz DF, Reeves AP, Kostis WJ, Zhao B, Henschke CI: Small pulmonary nodules: Volumetrically determined growth rates based on CT evaluation. Radiology 217(1):251–256, 2000. https://doi.org/10.1148/radiology.217.1.r00oc33251

    Article  CAS  Google Scholar 

  12. Kostis WJ, Reeves AP, Yankelevitz DF, Henschke CI: Three-dimensional segmentation and growth-rate estimation of small pulmonary nodules in helical CT images. IEEE Transactions on Medical Imaging 22(10):1259–1274, 2003. https://doi.org/10.1109/TMI.2003.817785

    Article  Google Scholar 

  13. Prasanna P, Tiwari P, Madabhushi A: Co-occurrence of local anisotropic gradient orientations (CoL1AGe): A new radiomics descriptor. Scientific Reports 6:37241, 2016. https://doi.org/10.1038/srep37241

  14. Dalal N, Triggs B: Histograms of oriented gradients for human detection. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, 20-25 June, 2005. https://doi.org/10.1109/CVPR.2005.177

  15. Ojala T, Pietikäinen M, Mäenpää T: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and. Machine Intelligence 24(7):971–987, 2002. https://doi.org/10.1109/TPAMI.2002.1017623

    Article  Google Scholar 

  16. Han F, Wang H, Han H, Song B, Li L, Moore W, Lu H, Zhao H, Liang Z: Texture feature analysis for computer-aided diagnosis on pulmonary nodules. Journal of Digital Imaging 28(1):99–115, 2015. https://doi.org/10.1007/s10278-014-9718-8

    Article  Google Scholar 

  17. Wang H, Zhao T, Li LC, Pan H, Liu W, Gao H, Han F, Wang Y, Qi Y, Liang Z: A hybrid CNN feature model for pulmonary nodule malignancy risk differentiation. Journal of X-Ray Science and Technology 26(2):171–187, 2018. https://doi.org/10.3233/XST-17302

    Article  Google Scholar 

  18. Keshani M, Azimifa Z, Tajeripour F, Boostani R: Lung nodule segmentation and recognition using SVM classifier and active contour modeling: A complete intelligent system. Computers in Biology and Medicine 43(4):287–300, 2013. https://doi.org/10.1016/j.compbiomed.2012.12.004

    Article  Google Scholar 

  19. Tartar A, Kilic N, Akan A: A new method for pulmonary nodule detection using decision trees. 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Osaka, Japan, 3–7 July 2013. https://doi.org/10.1109/EMBC.2013.6611257

  20. Lee SLA, Kouzani A, Hu EJ: Random forest based lung nodule classification aided by clustering. Computerized Medical Imaging and Graphics 34(7):535–542, 2010. https://doi.org/10.1016/j.compmedimag.2010.03.006

    Article  CAS  Google Scholar 

  21. Yann L: LeNet-5, convolutional neural networks . NY, USA. [Online], 2013 Available at https://yann.lecun.com/exdb/lenet/

  22. Zhang T, Zhao J, Luo J, Qiang Y: Deep belief network for lung nodules diagnosed in CT imaging. International Journal of Performability Engineering 13(8):1358–1370, 2017. https://doi.org/10.23940/ijpe.17.08.p17.13581370

  23. Cheng J, Ni D, Chou Y, Qin J, Tiu C, Chang Y, Huang C, Shen D, Chen C: Computer-aided diagnosis with deep learning architecture: Applications to breast lesions in US images and pulmonary nodules in CT scans. Scientific Reports 6:24454, 2016. https://doi.org/10.1038/srep24454

  24. Song Q, Zhao L, Luo X, Dou X: Using deep learning for classification of lung nodules on computed tomography images. Journal of Healthcare Engineering 2017(1):1–7, 2017. https://doi.org/10.1155/2017/8314740

    Article  Google Scholar 

  25. Sun W, Zheng B, Qian W: Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis. Computers in Biology and Medicine 89:530–539, 2017. https://doi.org/10.1016/j.compbiomed.2017.04.006

    Article  Google Scholar 

  26. Armato, III SG, Mclennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, Zhao B, Henschke CI, Hoffman EA, Kazerooni EA, MacMahon H, van Beek EJR, Yankelevitz D, Biancardi AM, Bland PH, Brown MS, Engelmann RM, Laderach GE, Max D, Pais RC, Qing DP-Y, Roberts RY, Smith AR, Starkey A, Batra P, Caligiuri P, Farooqi A, Gladish GW, Jude CM, Munden RF, Petkovska I, Quint LE, Schwartz LH, Sundaram B, Dodd LE, Fenimore C, Gur D, Petrick N, Freymann J, Kirby J, Hughes B, Casteele AV, Gupte S, Sallam M, Heath MD, Kuhn MH, Dharaiya E, Burns R, Fryd DS, Salganicoff M, Anand V, Shreter U, Vastagh S, Croft BY, Clarke LP: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Medical Physics 38(2):915–931, 2011. https://doi.org/10.1118/1.3528204

    Article  Google Scholar 

  27. Jacobs C, Setio AAA, Traverso A, Ginneken BV: Lung nodule analysis 2016. [Online], 2016. Available at https://luna16.grand-challenge.org/home/

  28. Shin H-C, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM: Deep donvolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging 35(5):1285–1298, 2016. https://doi.org/10.1109/TMI.2016.2528162

    Article  Google Scholar 

  29. Hosny KM, Kassem MA, Foaud MM: Skin cancer classification using deep learning and transfer learning. 9th Cairo International Biomedical Engineering Conference (CIBEC2018), Cairo, Egypt, December 20–22, 2018. https://doi.org/10.1109/CIBEC.2018.8641762

  30. Hosny KM, Kassem MA, Foaud MM: Classification of skin lesions using transfer learning and augmentation with Alex-net. PLOS ONE 14(5):e0217293, 2019. https://doi.org/10.1371/journal.pone.0217293

    Article  CAS  Google Scholar 

  31. Krizhevsky A, Sutskever I, Hinton GE: ImageNet classification with deep convolutional neural networks. Communications of the ACM 60(6):84–90, June 2017. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  32. Simonyan K and Zisserman A: Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations 2015, San Diego, CA, May 7–9, 2015. Available at https://arxiv.org/pdf/1409.1556.pdf

  33. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, and Rabinovich A: Going deeper with convolutions. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, June 7-12, 2015:1–9. https://doi.org/10.1109/cvpr.2015.7298594

  34. He K, Zhang X, Ren S, Sun J: Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, June 27-30, 2016. https://doi.org/10.1109/CVPR.2016.90

  35. Pan SJ, Yang Q: A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22(10):1345–1359, 2010. https://doi.org/10.1109/TKDE.2009.191

    Article  Google Scholar 

  36. Glorot X, Bengio Y: Understanding the difficulty of training deep feedforward neural networks. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, PMLR 9:249–256, 2010. Available at http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf

  37. Hinton GE, Srivastava N, Krizhevsky A, Sutskever I and Salakhutdinov RR: Improving neural networks by preventing co-adaptation of feature detectors. [Online]. arXiv: 1207. 0580 [cs. NE], 2012. Available at https://arxiv.org/pdf/1207.0580.pdf

  38. Ioffe S, Szegedy C: Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference on Machine Learning, July 2015, 37:448–456. Available at http://arxiv.org/abs/1502.03167.pdf

  39. Yosinski J, Clune J, Bengio Y, Lipson H: How transferable are features in deep neural networks? Proceedings of the 27th International Conference on Neural Information Processing Systems, December 2014, 2:3320-3328. Available at https://papers.nips.cc/paper/5347-how-transferable-are-features-in-deep-neural-networks.pdf

  40. Zeiler MD, Fergus R: Visualizing and understanding convolutional networks. European Conference on Computer Vision, Zurich, Switzerland 2014:818–833. https://doi.org/10.1007/978-3-319-10590-1-53

Download references

Funding

This work was supported by the National Natural Science Foundation of China under Grant No. 81671773, 61672146, and 61802055, the Fundamental Research Funds for the Central Universities of China under grant N151903001, N171902001 and N171903003, and Natural Science Foundation of Liaoning Province of China 20180550182. Zhengrong Liang was supported by NIH Grant #CA206171 of the National Cancer Institute, USA.

Author information

Authors and Affiliations

  1. School of Biomedical Engineering, Southern Medical University, Guangzhou, Guangdong, 510515, People’s Republic of China

    Fangfang Han

  2. College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, Liaoning, 110169, People’s Republic of China

    Fangfang Han, Linkai Yan, Junxin Chen, Yueyang Teng, Shuo Chen & Shouliang Qi

  3. College of Engineering, University of Texas at El Paso, El Paso, TX, 79968, USA

    Wei Qian

  4. Department of Family, Population and Preventive Medicine, Stony Brook University, Stony Brook, NY, 11794, USA

    Jie Yang

  5. Department of Radiology, Stony Brook University, Stony Brook, NY, 11794, USA

    William Moore, Shu Zhang & Zhengrong Liang

Authors
  1. Fangfang Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Linkai Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junxin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yueyang Teng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shuo Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shouliang Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Wei Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jie Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. William Moore
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Shu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Zhengrong Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Wei Qian or Zhengrong Liang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, F., Yan, L., Chen, J. et al. Predicting Unnecessary Nodule Biopsies from a Small, Unbalanced, and Pathologically Proven Dataset by Transfer Learning. J Digit Imaging 33, 685–696 (2020). https://doi.org/10.1007/s10278-019-00306-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-019-00306-z

Keywords

Search

Navigation