Skip to main content
SpringerLink
Log in

Multi-branch sharing network for real-time 3D brain tumor segmentation

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Brain tumors are one of the most lethal diseases in the world. The segmentation of brain tumor is of great significance for physician in formulating appropriate diagnostic and treatment plans, not only accurate but also efficient 3D segmentation algorithms are urgently demanded in clinical practice. Nowadays, several 3D convolution neural networks have achieved impressive segmentation performance. However, these architectures come with extremely high computational overheads due to the extra depth dimensionality in 3D convolution, which may make these models prohibitive from practical large-scale clinic application. In this work, we aim at designing a more efficient and lightweight network without accuracy reduction for real-time segmentation of magnetic resonance images. To this end, we propose a multi-branch sharing network which consists of novel multi-branch sharing units. Different from other works, our proposed multi-branch sharing units focus the information sharing and communication between grouped layers by leveraging a Multiplexer operation, which can reduce the computational cost significantly while maintaining decent performance. Extensive experimental results on the BraTS2018 challenge dataset show that the proposed architecture achieve real-time inference while maintaining high accuracy for 3D brain magnetic resonance image segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Zhou, C., Ding, C., Wang, X., Lu, Z., Tao, D.: One-pass multi-task networks with crosstask guided attention for brain tumor segmentation. arXiv preprint arXiv:1906.01796 (2019)

  2. Moghbel, M., Mashohor, S., Mahmud, R., Saripan, M.I.B.: Review of liver segmentation and computer assisted detection/diagnosis methods in computed tomography. Artif. Intell. Rev. 50(4), 497–537 (2018)

    Article  Google Scholar 

  3. Tripathi, S., Anand, R., Fernandez, E.: A review of brain mr image segmentation techniques. In: Proceedings of International Conference on Recent Innovations in Applied Science, Engineering & Technology, pp 16–17 (2018)

  4. Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3d u-net: learning dense volumetric segmentation from sparse annotation. In: International Conference on Medical Image Computing and Computer-assisted Intervention, pp. 424–432. Springer (2016)

  5. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention, pp. 234–241. Springer (2015)

  6. Milletari, F., Navab, N., Ahmadi, S.A.: V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565—571. IEEE (2016)

  7. Nuechterlein, N., Mehta, S.: 3d-espnet with pyramidal refinement for volumetric brain tumor image segmentation. In: International MICCAI Brainlesion Workshop, pp. 245–253. Springer (2018)

  8. Mehta, S., Rastegari, M., Caspi, A., Shapiro, L., Hajishirzi, H.: Espnet: Efficient spatial pyramid of dilated convolutions for semantic segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 552–568 (2018)

  9. Chen, W., Liu, B., Peng, S., Sun, J., Qiao, X.: S3d-unet: Separable 3d u-net for brain tumor segmentation. In: International MICCAI Brainlesion Workshop, pp. 358–368. Springer (2018)

  10. Mehta, S., Rastegari, M., Shapiro, L., Hajishirzi, H.: Espnetv2: A light-weight, power efficient, and general purpose convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9190–9200 (2019)

  11. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  12. Pereira, S., Alves, V., Silva, C.A.: Adaptive feature recombination and recalibration for semantic segmentation: application to brain tumor segmentation in MRI. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 706–714. Springer (2018)

  13. Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.M., Larochelle, H.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017)

    Article  Google Scholar 

  14. Zikic, D., Ioannou, Y., Brown, M., Criminisi, A.: Segmentation of brain tumor tissues with convolutional neural networks. In: Proceedings MICCAI-BRATS, pp. 36–39 (2014)

  15. Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3d CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017)

    Article  Google Scholar 

  16. Han, X.: Automatic liver lesion segmentation using a deep convolutional neural network method. arXiv preprint arXiv:1704.07239 (2017)

  17. Chen, W., Wilson, J., Tyree, S., Weinberger, K., Chen, Y.: Compressing neural networks with the hashing trick. In: International Conference on Machine Learning, pp. 2285–2294 (2015)

  18. Han, S., Mao, H., Dally, W.J.: Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 (2015)

  19. Rastegari, M., Ordonez, V., Redmon, J., Farhadi, A.: Xnor-net: Imagenet classification using binary convolutional neural networks. In: European Conference on Computer Vision, pp. 525–542. Springer (2016)

  20. Zhang, X., Zhou, X., Lin, M., Sun, J.: Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6848–6856 (2018)

  21. Wang, Y., Zhou, Q., Liu, J., Xiong, J., Gao, G., Wu, X., Latecki, L.J.: Lednet: a lightweight encoder-decoder network for real-time semantic segmentation. arXiv preprint arXiv:1905.02423 (2019)

  22. Arredondo-Velzquez, M., Diaz-Carmona, J., Torres-Huitzil, C., Padilla-Medina, A., Prado-Olivarez, J.: A streaming architecture for convolutional neural networks based on layer operations chaining. J. Real Time Image Process. 1–19 (2020)

  23. Cheng, G., Cheng, J., Luo, M., He, L., Tian, Y., Wang, R.: Effective and efficient multitask learning for brain tumor segmentation. J. Real Time Image Process. 1–10 (2020)

  24. Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: Bisenet: bilateral segmentation network for real-time semantic segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 325–341 (2018)

  25. Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: a deep convolutional encoderdecoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)

    Article  Google Scholar 

  26. Romera, E., Alvarez, J.M., Bergasa, L.M., Arroyo, R.: Erfnet: efficient residual factorized convnet for real-time semantic segmentation. IEEE Trans. Intell. Transp. Syst. 19(1), 263–272 (2017)

    Article  Google Scholar 

  27. Chen, Y., Kalantidis, Y., Li, J., Yan, S., Feng, J.: Multi-fiber networks for video recognition. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 352–367 (2018)

  28. Gao, S.H., Cheng, M.M., Zhao, K., Zhang, X.Y., Yang, M.H., Torr, P.: Res2net: a new multi-scale backbone architecture. arXiv preprint arXiv:1904.01169 (2019)

  29. Huang, Y., Wang, Q., Jia, W., He, X.: See more than oncekernel-sharing atrous convolution for semantic segmentation. arXiv preprint arXiv:1908.09443 (2019)

  30. Simi, V.R., Edla, D.R., Joseph, J., Kuppili, V.: Analysis of controversies in the formulation and evaluation of restoration algorithms for MR Images. Expert Syst. Appl. 135, 39–59 (2019)

    Article  Google Scholar 

  31. Kuppusamy, P.G., Joseph, J., Jayaraman, S.: A customized nonlocal restoration schemes with adaptive strength of smoothening for magnetic resonance images. Biomed. Signal Process. Control 49, 160–172 (2019)

    Article  Google Scholar 

  32. Simi, V.R., Edla, D.R., Joseph, J., Kuppili, V.: Parameter-free fuzzy histogram equalisation with illumination preserving characteristics dedicated for contrast enhancement of magnetic resonance images. Appl. Soft Comput. 106364 (2020)

  33. Joseph, J., Anoop, B.N., Williams, J.: A modified unsharp masking with adaptive threshold and objectively defined amount based on saturation constraints. Multimed. Tools Appl. 78(8), 11073–11089 (2019)

    Article  Google Scholar 

  34. Joseph, J., Periyasamy, R.: Nonlinear sharpening of MR images using a locally adaptive sharpness gain and a noise reduction parameter. Pattern Anal. Appl. 22(1), 273–283 (2019)

    Article  MathSciNet  Google Scholar 

  35. Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J.S., Freymann, J.B., Farahani, K., Davatzikos, C.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 4, 170117 (2017)

    Article  Google Scholar 

  36. Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al.: The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2014)

    Article  Google Scholar 

  37. Kao, P.Y., Ngo, T., Zhang, A., Chen, J.W., Manjunath, B.: Brain tumor segmentation and tractographic feature extraction from structural MR images for overall survival prediction. In: International MICCAI Brainlesion Workshop, pp. 128–141. Springer (2018)

  38. Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., Maier-Hein, K.H.: No new-net. In: International MICCAI brainlesion workshop, pp. 234–244. Springer (2018)

  39. Myronenko, A.: 3d MRI brain tumor segmentation using autoencoder regularization. In: International MICCAI brainlesion workshop, pp. 311–320. Springer (2018)

Download references

Acknowledgements

This work was supported by the Natural Science Foundation of Beijing Municipality (no. 4182038), the National Natural Science Foundation of China, Major Research Plan (no. 61671054) and the Fundamental Research Funds for the China Central Universities of USTB (FRF-DF-19-002).

Author information

Authors and Affiliations

  1. School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, 100083, China

    Jiangyun Li, Junfeng Zheng & Hong Yu

  2. Thermo Fisher Scientific, Waltham, USA

    Meng Ding

Authors
  1. Jiangyun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junfeng Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meng Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiangyun Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Zheng, J., Ding, M. et al. Multi-branch sharing network for real-time 3D brain tumor segmentation. J Real-Time Image Proc 18, 1409–1419 (2021). https://doi.org/10.1007/s11554-020-01049-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-01049-9

Keywords

Search

Navigation