Abstract
Colorectal cancer (CRC) is the second most diagnosed cancer in the United States. It is identified by histopathological evaluations of microscopic images of the cancerous region, relying on a subjective interpretation. The Colorectal Histology dataset used in this study contains 5000 images, made available by the University Medical Center Mannheim. This approach proposes the automatic identification of eight types of tissues found in CRC histopathological evaluation. We apply Transfer Learning from architectures of Convolutional Neural Networks (CNNs). We modify the structures of CNNs to extract features from the images and input them to well-known machine learning methods: Naive Bayes, Multilayer Perceptron, k-Nearest Neighbors, Random Forest, and Support Vector Machine (SVM). We evaluated 108 extractor–classifier combinations. The one that achieved the best results is DenseNet169 with SVM (RBF), reaching an Accuracy of 92.083% and F1-Score of 92.117%. Therefore, our approach is capable of distinguishing tissues found in CRC histopathological evaluation.
Similar content being viewed by others
Notes
We considered the SVM classifier with different kernels as different classifiers.
References
Abir F, Alva S, Longo WE, Audiso R, Virgo KS, Johnson FE (2006) The postoperative surveillance of patients with colon cancer and rectal cancer. Am J Surg 192(1):100–108. https://doi.org/10.1016/j.amjsurg.2006.01.053
Alhindi TJ, Kalra S, Ng KH, Afrin A, Tizhoosh HR (2018) Comparing LBP, HOG and deep features for classification of histopathology images. In: 2018 International Joint Conference on Neural Networks (IJCNN), IEEE, pp 1–7, https://doi.org/10.1109/IJCNN.2018.8489329
Araújo T, Aresta G, Castro E, Rouco J, Aguiar P, Eloy C, Polónia A, Campilho A (2017) Classification of breast cancer histology images using convolutional neural networks. PLoS ONE 12(6):e0177544. https://doi.org/10.1371/journal.pone.0177544
Arnold M, Sierra MS, Laversanne M, Soerjomataram I, Jemal A, Bray F (2017) Global patterns and trends in colorectal cancer incidence and mortality. Gut 66(4):683–691. https://doi.org/10.1136/gutjnl-2015-310912
Bayramoglu N, Heikkilä J (2016) Transfer learning for cell nuclei classification in histopathology images. In: European Conference on Computer Vision, Springer, pp 532–539
Breiman L (2001) Random forests. Mach Learn 45(1):5–32
Chatterjee R, Maitra T, Islam SH, Hassan MM, Alamri A, Fortino G (2019) A novel machine learning based feature selection for motor imagery EEG signal classification in internet of medical things environment. Future Gener Comput Syst 98:419–434. https://doi.org/10.1016/j.future.2019.01.048
Choi JY, Yoo TK, Seo JG, Kwak J, Um TT, Rim TH (2017) Multi-categorical deep learning neural network to classify retinal images: a pilot study employing small database. PLoS ONE. https://doi.org/10.1371/journal.pone.0187336
Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1251–1258. https://doi.org/10.1109/CVPR.2017.195
Coudray N, Ocampo PS, Sakellaropoulos T, Narula N, Snuderl M, Fenyö D, Moreira AL, Razavian N, Tsirigos A (2018) Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med 24(10):1559–1567. https://doi.org/10.1038/s41591-018-0177-5
da Nóbrega RVM, Filho PPR, Rodrigues MB, da Silva SPP, Júnior CMJMD, de Albuquerque VHC (2018) Lung nodule malignancy classification in chest computed tomography images using transfer learning and convolutional neural networks. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3895-1
Fukunaga K, Narendra PM (1975) A branch and bound algorithm for computing k-nearest neighbors. IEEE Trans Comput 24(7):750–753. https://doi.org/10.1109/T-C.1975.224297
Gecer B, Aksoy S, Mercan E, Shapiro LG, Weaver DL, Elmore JG (2018) Detection and classification of cancer in whole slide breast histopathology images using deep convolutional networks. Pattern Recogn 84:345–356. https://doi.org/10.1016/j.patcog.2018.07.022
Girard L, Rodriguez-Canales J, Behrens C, Thompson DM, Botros IW, Tang H, Xie Y, Rekhtman N, Travis WD, Wistuba II et al (2016) An expression signature as an aid to the histologic classification of non-small cell lung cancer. Clin Cancer Res 22(19):4880–4889. https://doi.org/10.1158/1078-0432
Han D, Liu Q, Fan W (2018) A new image classification method using CNN transfer learning and web data augmentation. Expert Syst Appl 95:43–56. https://doi.org/10.1016/j.eswa.2017.11.028
Han T, Nunes VX, De Freitas Souza LF, Marques AG, Silva ICL, Junior MAAF, Sun J, Filho PPR (2020) Internet of medical things-based on deep learning techniques for segmentation of lung and stroke regions in CT scans. IEEE Access 8:71117–71135. https://doi.org/10.1109/ACCESS.2020.2987932
Hassan MM, Ullah S, Hossain MS, Alelaiwi A (2020) An end-to-end deep learning model for human activity recognition from highly sparse body sensor data in internet of medical things environment. J Supercomput. https://doi.org/10.1007/s11227-020-03361-4
Haykin S (2009) Neural networks and learning machines, vol 3. Pearson Upper Saddle River, NJ, USA
He K, Zhang X, Ren S, Sun J (2016a) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778. https://doi.org/10.1109/CVPR.2016.90
He K, Zhang X, Ren S, Sun J (2016b) Identity mappings in deep residual networks. In: European Conference on Computer Vision, Springer, pp 630–645
Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:170404861
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4700–4708. https://doi.org/10.1109/CVPR.2017.243
Hussein S, Cao K, Song Q, Bagci U (2017) Risk stratification of lung nodules using 3d CNN-based multi-task learning. Lecture Notes in Computer Science. Springer, Cham, pp 249–260
Jr Dourado CM, da Silva SP, da Nóbrega RV, Barros AC, Reboucas Filho PP, de Albuquerque VHC (2019) Deep learning IoT system for online stroke detection in skull computed tomography images. Comput Netw 152:25–39. https://doi.org/10.1016/j.comnet.2019.01.019
Kather J, Weis CA, Bianconi F, Melchers S, Schad L, Gaiser T, Marx A, Zöllner F (2016a) Multi-class texture analysis in colorectal cancer histology. Sci Rep 6:27988. https://doi.org/10.1038/srep27988
Kather JN, Zöllner FG, Bianconi F, Melchers SM, Schad LR, Gaiser T, Marx A, Weis CA (2016b) Collection of textures in colorectal cancer histology. https://doi.org/10.5281/ZENODO.53169
Kather JN, Krisam J, Charoentong P, Luedde T, Herpel E, Weis CA, Gaiser T, Marx A, Valous NA, Ferber D et al (2019) Predicting survival from colorectal cancer histology slides using deep learning: a retrospective multicenter study. PLoS Med. https://doi.org/10.1371/journal.pmed.1002730
Kermany DS, Goldbaum M, Cai W, Valentim CC, Liang H, Baxter SL, McKeown A, Yang G, Wu X, Yan F, Dong J, Prasadha MK, Pei J, Ting MY, Zhu J, Li C, Hewett S, Dong J, Ziyar I, Shi A, Zhang R, Zheng L, Hou R, Shi W, Fu X, Duan Y, Huu VA, Wen C, Zhang ED, Zhang CL, Li O, Wang X, Singer MA, Sun X, Xu J, Tafreshi A, Lewis MA, Xia H, Zhang K (2018) Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5):1122–1131.e9. https://doi.org/10.1016/j.cell.2018.02.010
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:14126980
Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791
Loh BCS, Then PHH (2017) Deep learning for cardiac computer-aided diagnosis: benefits, issues & solutions. mHealth 3:45. https://doi.org/10.21037/mhealth.2017.09.01
Marques JAL, Cortez PC, Madeiro JPDV, Fong SJ, Schlindwein FS, De Albuquerque VHC (2019) Automatic cardiotocography diagnostic system based on hilbert transform and adaptive threshold technique. IEEE Access 7:73085–73094
Marques JAL, Cortez PC, Madeiro JP, de Albuquerque VHC, Fong SJ, Schlindwein FS (2020) Nonlinear characterization and complexity analysis of cardiotocographic examinations using entropy measures. J Supercomput 76(2):1305–1320
Olivares R, Munoz R, Soto R, Crawford B, Cárdenas D, Ponce A, Taramasco C (2020) An optimized brain-based algorithm for classifying parkinson’s disease. Appl Sci 10(5):1827. https://doi.org/10.3390/app10051827
Orenstein EC, Beijbom O (2017) Transfer learning and deep feature extraction for planktonic image data sets. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, pp 1082–1088. https://doi.org/10.1109/WACV.2017.125
Paul R, Hawkins SH, Balagurunathan Y, Schabath MB, Gillies RJ, Hall LO, Goldgof DB (2016) Deep feature transfer learning in combination with traditional features predicts survival among patients with lung adenocarcinoma. Tomography 2(4):388. https://doi.org/10.18383/j.tom.2016.00211
Rachapudi V, Devi GL (2020) Improved convolutional neural network based histopathological image classification. Evolut Intell 284:1–7
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252. https://doi.org/10.1007/s11263-015-0816-y
Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4510–4520
Santos MA, Munoz R, Olivares R, Rebouças Filho PP, Del Ser J, de Albuquerque VHC (2020) Online heart monitoring systems on the internet of health things environments: a survey, a reference model and an outlook. Inf Fusion 53:222–239. https://doi.org/10.1016/j.inffus.2019.06.004
Sarmento RM, Vasconcelos FFX, Rebouças Filho PP, Wu W, De Albuquerque VHC (2019) Automatic neuroimage processing and analysis in stroke—a systematic review. IEEE Rev Biomed Eng 13:130–155
Sarmento RM, Vasconcelos FF, Filho PPR, de Albuquerque VHC (2020) An IoT platform for the analysis of brain CT images based on parzen analysis. Future Gener Comput Syst 105:135–147. https://doi.org/10.1016/j.future.2019.11.033
Shaban M, Awan R, Fraz MM, Azam A, Tsang YW, Snead D, Rajpoot NM (2020) Context-aware convolutional neural network for grading of colorectal cancer histology images. IEEE Trans Med Imaging. https://doi.org/10.1109/tmi.2020.2971006
Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298. https://doi.org/10.1109/TMI.2016.2528162
Siegel RL, Miller KD, Fedewa SA, Ahnen DJ, Meester RG, Barzi A, Jemal A (2017) Colorectal cancer statistics, 2017. CA Cancer J Clin 67(3):177–193. https://doi.org/10.3322/caac.21395
Siegel RL, Miller KD, Sauer AG, Fedewa SA, Butterly LF, Anderson JC, Cercek A, Smith RA, Jemal A (2020) Colorectal cancer statistics, 2020. Cancer J Clin CA. https://doi.org/10.3322/caac.21601
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556
Slaoui M, Fiette L (2010) Histopathology procedures: from tissue sampling to histopathological evaluation. Methods in molecular biology. Humana Press, Totowa, pp 69–82
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1–9. https://doi.org/10.1109/CVPR.2015.7298594
Theodoridis S, Koutroumbas K (2008) Pattern recognition, 4th edn. Academic Press, USA
Tseng KK, Zhang R, Chen CM, Hassan MM (2020) DNetUnet: a semi-supervised CNN of medical image segmentation for super-computing AI service. J Supercomput. https://doi.org/10.1007/s11227-020-03407-7
Vapnik VN (1998) Statistical learning theory. Wiley-Interscience, Hoboken
Wang C, Shi J, Zhang Q, Ying S (2017) Histopathological image classification with bilinear convolutional neural networks. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, pp 4050–4053
Wang EK, Chen CM, Hassan MM, Almogren A (2020) A deep learning based medical image segmentation technique in internet-of-medical-things domain. Future Gener Comput Syst 108:135–144. https://doi.org/10.1016/j.future.2020.02.054
Wang P, Hu X, Li Y, Liu Q, Zhu X (2016) Automatic cell nuclei segmentation and classification of breast cancer histopathology images. Signal Process 122:1–13. https://doi.org/10.1016/j.sigpro.2015.11.011
Xu J, Xiang L, Liu Q, Gilmore H, Wu J, Tang J, Madabhushi A (2015) Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images. IEEE Trans Med Imaging 35(1):119–130. https://doi.org/10.1109/TMI.2015.2458702
Xu Y, Jia Z, Wang LB, Ai Y, Zhang F, Lai M, Eric I, Chang C (2017) Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features. BMC Bioinform 18(1):281. https://doi.org/10.1186/s12859-017-1685-x
Yu KH, Zhang C, Berry GJ, Altman RB, Ré C, Rubin DL, Snyder M (2016) Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nat Commun 7:12474. https://doi.org/10.1038/ncomms12474
Yu KH, Wang F, Berry GJ, Ré C, Altman RB, Snyder M, Kohane IS (2020) Classifying non-small cell lung cancer types and transcriptomic subtypes using convolutional neural networks. J Am Med Inform Assoc 27(5):757–769. https://doi.org/10.1093/jamia/ocz230
Zoph B, Le QV (2016) Neural architecture search with reinforcement learning. arXiv preprint arXiv:161101578
Acknowledgements
The authors are grateful to the Deanship of Scientific Research at King Saud University for funding this work through Vice Deanship of Scientific Research Chairs: Chair of Pervasive and Mobile Computing. This study was also supported in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Also, Pedro Pedrosa Rebouças Filho acknowledges the sponsorship from the Brazilian National Council for Research and Development (CNPq) via Grants Nos. 431709/2018-1 and 311973/2018-3.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ohata, E.F., Chagas, J.V.S.d., Bezerra, G.M. et al. A novel transfer learning approach for the classification of histological images of colorectal cancer. J Supercomput 77, 9494–9519 (2021). https://doi.org/10.1007/s11227-020-03575-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-020-03575-6