Skip to main content

Advertisement

SpringerLink
Log in

A transfer learning-based novel fusion convolutional neural network for breast cancer histology classification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

To train a convolutional neural network (CNN) from scratch is not suitable for medical image tasks with insufficient data. Benefiting from the transfer learning, the pre-trained CNN model can provide a reliable initial solution for model optimization of medical image classification. A key concern in breast cancer histology classification is that the model should cover the multi-scale features including nuclei-scale, nuclei organization, and structure-scale features. Inspired by these conjectures, we proposed a novel fusion convolutional neural network (FCNN) based on pre-trained VGG19. The FCNN fuses the shallow, intermediate abstract, and abstract layers to approximately cover the multi-scale features. In order to improve the sensitivity of carcinoma classes, the prediction priority is introduced to enable the lesions can be detected as early as possible. Experimental results show that the proposed FCNN can approximately cover the nuclei-scale, nuclei organization, and structure-scale features. Accuracies of 85%, 75%, and 80.56% are achieved in Initial, Extended, and Overall test set, respectively. The source code for this research is available at https://github.com/yxchspring/breasthistolgoy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Abadi M, Barham P, Chen J, et al. (2016) Tensorflow: A system for large-scale machine learning[C]. 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16). 265–283

  2. Albarqouni S, Baur C, Achilles F, Belagiannis V, Demirci S, Navab N (2016) AggNet: deep Learning from crowds for mitosis detection in breast Cancer histology images[J]. IEEE Trans Med Imaging 35(5):1313–1321

    Article  Google Scholar 

  3. Araújo T, Aresta G, Castro E, Rouco J, Aguiar P, Eloy C, Polónia A, Campilho A (2017) Classification of breast cancer histology images using convolutional neural networks[J]. PLoS One 12(6):e0177544

    Article  Google Scholar 

  4. Beam AL, Kohane IS (2018) Big data and machine learning in health care[J]. JAMA 319(13):1317–1318

    Article  Google Scholar 

  5. Berg E (2015) Breast cancer[J]. CMAJ Can Med Assoc J 75(S1):257–269

    Google Scholar 

  6. Bioimaging Challenge (2015) Breast Histology Dataset: Available at https://rdm.inesctec.pt/dataset/nis-2017-003. Accessed 18 July 2019

  7. Che D, Pan W, Ma Y et al (2019) Tire tread pattern recognition based on non-linear activated aggregation residual neural network[J]. J Jiangxi Univ Sci Technol 40(05):80–85

    Google Scholar 

  8. Ciresan DC, Giusti A, Gambardella LM et al (2013) Mitosis detection in breast Cancer histology images with deep neural networks[J]. Med Image Comput Comput Assist Interv 16(Pt 2):411–418

    Google Scholar 

  9. DeSantis CE, Ma J, Goding Sauer A et al (2017) Breast cancer statistics, 2017, racial disparity in mortality by state[J]. CA Cancer J Clin 67(6):439–448

    Article  Google Scholar 

  10. Doi K (2007) Computer-aided diagnosis in medical imaging: historical review, current status and future potential[J]. Comput Med Imaging Graph 31(4–5):198–211

    Article  Google Scholar 

  11. Golatkar A, Anand D, Sethi A (2018) Classification of breast cancer histology using deep learning[C]. International Conference Image Analysis and Recognition. Springer: Cham, 837–844

  12. Gudmundsson E, Straus CM, Armato SG III (2019) Pre-trained deep convolutional neural networks for the segmentation of malignant pleural mesothelioma tumor on CT scans[C]. Medical Imaging 2019: computer-aided diagnosis. Int Soc Opt Photon 10950:109503J

    Google Scholar 

  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition[C]. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778

  14. Huang JB, Yang MH (2010) Fast sparse representation with prototypes[C]. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 3618–3625

  15. Jaworek-Korjakowska J, Kleczek P, Gorgon M (2019) Melanoma thickness prediction based on convolutional neural Network with VGG-19 model Transfer Learning[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops: 0–0

  16. Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories[C]. 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). IEEE, 2:2169–2178

  17. LeCun Y, Bengio Y, Hinton G (2015) Deep learning[J]. Nature 521(7553):436–444

  18. Li B, Sun F, Zhang Y (2019) Building recognition using Gist feature based on locality sensitive histograms of oriented gradients[J]. Pattern Recognit Image Anal 29(2):258–267

  19. McGuire S (2015) World cancer report 2014. Geneva, Switzerland: World Health Organization, international agency for research on cancer, WHO Press, [J]. 2016:418–419

  20. Nawaz W, Ahmed S, Tahir A, et al. (2018) Classification of breast cancer histology images using alexnet[C]. International Conference Image Analysis and Recognition. Springer: Cham, 869–876

  21. Oliva A, Torralba A (2001) Modeling the shape of the scene: A holistic representation of the spatial envelope[J]. Int J Comput Vis 42(3):145–175

    Article  Google Scholar 

  22. Rajaraman S, Antani SK, Poostchi M, Silamut K, Hossain MA, Maude RJ, Jaeger S, Thoma GR (2018) Pre-trained convolutional neural networks as feature extractors toward improved malaria parasite detection in thin blood smear images[J]. PeerJ 6:e4568

    Article  Google Scholar 

  23. Reinhard E, Adhikhmin M, Gooch B, Shirley P (2001) Color transfer between images[J]. IEEE Comput Graph Appl 21(5):34–41

    Article  Google Scholar 

  24. Shima Y, Nakashima Y, Yasuda M (2018) Classifying for a mixture of object images and character patterns by using CNN pre-trained for large-scale object image dataset[C]. 2018 13th IEEE Conference on Industrial Electronics and Applications (ICIEA). IEEE, 2360–2365

  25. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556

  26. Sivic J, Zisserman A (2003) Video Google: A text retrieval approach to object matching in videos[C]. Proceedings Ninth IEEE International Conference on Computer Vision, Nice, France, 1470–1477

  27. Stain Normalisation Toolbox (n.d.): Available at https://warwick.ac.uk/fac/sci/dcs/research/tia/software/sntoolbox. Accessed 17 July 2019

  28. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision[C]. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2818–2826

  29. Szeliski R (2010) Computer vision: algorithms and applications[M]. Springer Science & Business Media, Washington

    MATH  Google Scholar 

  30. Upadhyay PK, Chandra S (2019) Salient bag of feature for skin lesion recognition[J]. Int J Performability Eng 15(4):1083–1093

    Google Scholar 

  31. Wen L, Li X, Li X, et al. (2019) A New Transfer Learning Based on VGG-19 Network for Fault Diagnosis[C]. 2019 IEEE 23rd international conference on computer supported cooperative work in design (CSCWD). IEEE, 205–209

  32. Yang J, Ma J (2019) Feed-forward neural network training using sparse representation[J]. Expert Syst Appl 116:255–264

    Article  Google Scholar 

  33. Yang X, Yeo SY, Hong JM et al (2016) A deep learning approach for tumor tissue image classification[J]. IASTED Biomedical Engineering

  34. Yang G, Xu N, Li F et al (2018) Research on deep learning classification for nonlinear activation function[J]. J Jiangxi Univ Sci Technol 39(03):76–83

    Google Scholar 

  35. Yu X, Yu Z, Pang W et al (2018) An improved EMD-Based dissimilarity metric for unsupervised linear subspace Learning[J]. Complexity 2018:1–24

    Google Scholar 

  36. Yu X, Yu Z, Wu L et al (2019) Data-driven two-layer visual dictionary structure learning[J]. J Electron Imaging 28(2):023006

  37. Yu X, Cao S, Zhou Y, Yu Z, Xu Y (2020) Co-expression based cancer staging and application[J]. Sci Rep 10(1):10624

    Article  Google Scholar 

  38. Yu X, Zhang Z, Wu L et al (2020) Deep ensemble Learning for human action recognition in still images[J]. Complexity 2020(1):1–23

    Google Scholar 

  39. Zhang J, Wang L, Wen X (2019) Combination of GIST and PHOG features for calligraphy styles classification[C]. In Proceedings of the 2019 4th International Conference on Multimedia Systems and Signal Processing, 21–24

  40. Zhang Y, Zhang H, Yu M et al (2019) Sparse representation-based video quality assessment for synthesized 3D videos[J]. IEEE Transactions on Image Processing 29:509–524

Download references

Availability of data and material

The breast histology image data used to support the findings of this study are included within the article.

Funding

This research was funded by 1) Doctoral Scientific Research Foundation of Jiangxi University of Science and Technology, grant number jxxjbs19029, jxxjbs19006, jxxjbs19012. 2) National Natural Science Foundation of China, grant number 61901198, 61902145.

Author information

Authors and Affiliations

  1. School of Information Engineering, Jiangxi University of Science and Technology, Ganzhou, 341000, People’s Republic of China

    Xiangchun Yu, Miaomiao Liang & Qing Xu

  2. School of Artificial Intelligence, Jilin University, Changchun, 130012, People’s Republic of China

    Hechang Chen

  3. Department of Computer Science and Engineering, Lehigh University, Bethlehem, PA 18015, USA

    Lifang He

Authors
  1. Xiangchun Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hechang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Miaomiao Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qing Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lifang He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Miaomiao Liang or Qing Xu.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Code availability

The source code for this research is available at https://github.com/yxchspring/breasthistolgoy.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, X., Chen, H., Liang, M. et al. A transfer learning-based novel fusion convolutional neural network for breast cancer histology classification. Multimed Tools Appl 81, 11949–11963 (2022). https://doi.org/10.1007/s11042-020-09977-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09977-1

Keywords

Search

Navigation