Abstract
MRI is a broadly used imaging method to determine glioma-based tumors. During image processing, MRI provides large image information, and therefore, an accurate image processing must be carried out in clinical practices. Therefore, automatic and consistent methods are requisite for knowing the precise details of the image. The automated segmentation method inheres obstacles like inconsistency in tracing out the large spatial and structural inconsistency of brain tumors. In this work, a semantic-based U-NET-convolutional neural networks exploring a 3∗3 kernel’s size is proposed. Small kernels have an effect against overfitting in the deeper architecture and provide only a smaller number of weights in this network. Multiscale multimodal convolutional neural network (MSMCNN) with long short-term memory (LSTM)-based deep learning semantic segmentation technique is used for multimodalities of magnetic resonance images (MRI). The proposed methodology aims to identify and segregate the classes of tumors by analyzing every pixel in the image. Further, the performance of semantic segmentation is enhanced by applying a patch-wise classification technique. In this work, multiscale U-NET-based deep convolution network is used for classifying the multimodal convolutions into three different scale patches based on a pixel level. In order to identify the tumor classes, all three pathways are combined in the LSTM network. The proposed methodology is validated by a fivefold cross-validation scheme from MRI BRATS’15 dataset. The experiment outcomes show that the MSMCNN model outperforms the CNN-based models over the Dice coefficient and positive predictive value and obtains 0.9214 sensitivity and 0.9636 accuracy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Jin L et al (2014) A survey of MRI-based brain tumor segmentation methods. Tsinghua Sci Technol 19(6):578–595
Whittle IR (2004) The dilemma of low grade glioma. J Neurol Neurosurg Psychiatry 75(suppl 2):31
Kang E et al (2018) Deep convolutional framelet denosing for low-dose CT via wavelet residual network. IEEE Trans Med Imaging 37(6):1358–1369
Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35(8):1798–1828
Fan-Hui, K (2012) Image retrieval based on Gaussian Mixture Model. In: 2012 international conference on machine learning and cybernetics
Savitha R, Suresh S, Sundararajan N (2013) Projection-based fast learning fully complex-valued relaxation neural network. IEEE Trans Neural Netw Learn Syst 24(4):529–541
Zhang Y, Brady M, Smith S (2001) Segmentation of brain MR images through a hidden Markov random field model and the expectation–maximization algorithm. IEEE Trans Med Imaging 20(1):45–57
Shahzadi I, Tang TB, Meriadeau F, Quyyum A (2018) CNN-LSTM: cascaded framework for brain tumour classification. In: 2018 IEEE-EMBS conference on biomedical engineering and sciences (IECBES), Dec 2018. IEEE, pp 633–637
Akkus Z et al (2017) Deep learning for brain MRI segmentation: state of the art and future directions. J Digit Imaging 30(4):449–459
Moon WK et al (2013) Computer-aided tumor detection based on Multi-scale blob detection algorithm in automated breast ultrasound images. IEEE Trans Med Imaging 32(7):1191–1200
Gonçalves VM, Delamaro ME, Nunes FdLdS (2014) A systematic review on the evaluation and characteristics of computer-aided diagnosis systems. Revista Brasileira de Engenharia Biomédica 30:355–383
Brosch T et al (2016) Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation. IEEE Trans Med Imaging 35(5):1229–1239
Zaharchuk G et al (2018) Deep learning in neuroradiology. Am J Neuroradiol 39(10):1776–1784
Pereira S et al (2016) Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 35(5):1240–1251
Zhao L, Jia K (2016) Multiscale CNNs for brain tumor segmentation and diagnosis. Comput Math Methods Med 2016:7
Soltaninejad M et al (2017) Automated brain tumour detection and segmentation using superpixel-based extremely randomized trees in FLAIR MRI. Int J Comput Assist Radiol Surg 12(2):183–203
Dong H et al (2017) Automatic brain tumor detection and segmentation using U-Net based fully convolutional networks. In: Medical image understanding and analysis 2017. pp 506–517
Hu Y, Xia Y (2018) 3D Deep neural network-based brain tumor segmentation using multimodality magnetic resonance sequences. In: Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries. Springer, Cham
Wang X et al (2017) Beyond frame-level CNN: saliency-aware 3-D CNN With LSTM for video action recognition. IEEE Signal Process Lett 24(4):510–514
Li S et al (2017) Generating image descriptions with multidirectional 2D long short-term memory. IET Comput Vision 11(1):104–111
Tsironi E et al (2017) An analysis of convolutional long short-term memory recurrent neural networks for gesture recognition. Neurocomputing 268:76–86
Bauer S et al (2013) A survey of MRI-based medical image analysis for brain tumor studies. Phys Med Biol 58(13):0031–9155
Menze BH et al (2015) The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging 34(10):1993–2024
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declared that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Rajasree, R., Columbus, C.C. & Shilaja, C. Multiscale-based multimodal image classification of brain tumor using deep learning method. Neural Comput & Applic 33, 5543–5553 (2021). https://doi.org/10.1007/s00521-020-05332-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-020-05332-5