Skip to main content
SpringerLink
Log in

Medical image super-resolution via deep residual neural network in the shearlet domain

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper proposes aconvolutional neural network (CNN)-based efficient medical image super-resolution (SR) method in the shearlet domain. Because of differences between imaging mechanisms optimized for natural images and medical images, the design begins with building a medical image dataset for medical image SR and extracting effective areas to remarkably enhance the training effects of the CNN-based method. Then, a new medical image SR network structure—deep medical super-resolution network (DMSRN)—has been designed in which local residual learning is implemented through a recursive network and combined with global residual learning to heighten the depth of the network on the ground with no parameter increase. This effectively fixes the long-term dependency problem, which causes the prior state layers to barely have any effect on the following state layers. Last, the design addresses the problem of too-smooth reconstruction effects in the CNN-based method in the image space domain; shearlet transform is introduced to DMSRN to restore global topology through low-frequency sub-bands and restore local edge detail information through high-frequency sub-bands. Experimental results show that the proposed method is better than other state-of-the-art methods for medical image SR, which significantly promotes the restoration ability of texture structure and edge details.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Ahn N, Kang B, Sohn KA (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In: Eur. Conf. Comput. Vis. (ECCV), Munich

  2. Aly HA, Dubois E (2005) Image up-sampling using total-variation regularization with a new observation model. IEEE Trans Image Process 14(10):1647–1659

    Article  Google Scholar 

  3. Cao F, Liu H (2019) Single image super-resolution via multi-scale residual channel attention network. Neurocomputing 358:424–436

    Article  Google Scholar 

  4. Chantas GK, Galatsanos NP, Woods NA (2007) Super-resolution based on fast registration and maximum a posteriori reconstruction. IEEE Trans Image Process 16(7):1821–1830

    Article  MathSciNet  Google Scholar 

  5. Chen X, Wang X, Lu Y, Li W, Wang Z, Huang Z (2020) RBPNET: An Asymptotic Residual Back-Projection Network for super-resolution of very low-resolution face image. Neurocomputing 376:119–127

    Article  Google Scholar 

  6. Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M (2013) The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 26(6):1045–1057

    Article  Google Scholar 

  7. Deeba F, Kun S, Wang W, Ahmed J, Qadi B (2019) Wavelet integrated residual dictionary training for single image super-resolution. Multimed Tools Appl 78:27683–27701

    Article  Google Scholar 

  8. Deng X, Yang R, Xu M, Dragotti PL (2019) Wavelet Domain style transfer for an effective perception-distortion tradeoff in single image superresolution. Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Seoul, pp 3076–3085

  9. Dong C, Loy CC, He K, Tang X (2014) “Learning a deep convolutional network for image super-resolution. In: Eur. Conf. Comput. Vis. (ECCV), Zurich, pp 184–199

  10. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: Eur. Conf. Comput. Vis. (ECCV), Amsterdam, pp 391–407

  11. Easley G, Labate D, Lim WQ (2008) Sparse directional image representation using the discrete shearlet transform. Appl Comput Harmon Anal 25:25–46

    Article  MathSciNet  Google Scholar 

  12. Fan DP, Ji GP, Sun G, Cheng MM, Shen J, Shao L (2020) Camouflaged object detection. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Seattle, pp 2777–2787

  13. Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65

    Article  Google Scholar 

  14. Fu X, Huang J, Zeng D, Huang Y, Ding X, Paisley J (2017) Removing rain from single images via a deep detail network. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Honolulu, pp 3855–3863

  15. Gao M, Han X, Li J, Ji H, Zhang H, Sun J (2020) Image super-resolution based on two-level residual learning CNN. Multimed Tools Appl 79:4831–4846

    Article  Google Scholar 

  16. Gu YC, Zeng ZT, Chen HB, Wei J, Zhang Y, Chen BH, Li YQ, Qin YJ, Xie Q, Jiang ZR, Lu Y (2020) MedSRGAN: medical images super-resolution using generative adversarial networks,. Multimed Tools Appl 79:21815–21840

    Article  Google Scholar 

  17. Guo T, Mousavi HS, Vu TH, Monga V (2017) Deep wavelet prediction for image super-resolution. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Honolulu, pp 624–632

  18. Guo T, Mousavi HS, Monga V (2019) Adaptive Transform Domain Image Super-Resolution via Orthogonally Regularized Deep Networks. IEEE Trans Image Process 28(9):4685–4700

  19. Haris M, Shakhnarovich G, Ukita N (2018) Deep backprojection networks for super-resolution. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Salt Lake City, pp 1664–1673

  20. Harris JL (1964) Diffraction and resolving power. J Opt Soc Am 54(7):931–936

    Article  Google Scholar 

  21. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Santiago, pp 1026– 1034

  22. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Las Vegas, pp 770–778

  23. He J, Zheng J, Shen Y, Guo Y, Zhou H (2020) Facial image synthesis and Super-Resolution with stacked generative adversarial network. Neurocomputing 402:359–365

    Article  Google Scholar 

  24. Huang H, He R, Sun Z, Tan T (2017) Wavelet-srnet: A wavelet-based cnn for multi-scale face super resolution. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Honolulu, pp 1689–1697

  25. Huang H, He R, Sun Z, Tan T (2019) Wavelet domain generative adversarial network for multi-scale face hallucination. Int J Comput Vis 127:763–784

    Article  Google Scholar 

  26. Hui Z, Wang X, Gao X (2018) Fast and accurate single image super-resolution via information distillation network. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Salt Lake City, pp 723–731

  27. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proc. Int. Conf. Mach. Learn. (ICML), Lille, pp 448–456

  28. Jin X, Xiong Q, Xiong C, Li Z, Gao Z (2019) Single image super-resolution with multi-level feature fusion recursive network. Neurocomputing 370:166–173

    Article  Google Scholar 

  29. Kim J, Kwon Lee J, Mu Lee, K (2016) Accurate image super-resolution using very deep convolutional networks. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Las Vegas, pp 1646– 1654

  30. Kim J, Kwon Lee J, Mu Lee, K (2016) Deeply-recursive convolutional network for image super-resolution. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Las Vegas, pp 1637–1645

  31. Lai W-S, Huang J-B, Ahuja N, Yang M-H (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Honolulu, pp 624–632

  32. Le TN, Nguyen TV, Nie Z, Tran MT, Sugimoto A (2019) Anabranch network for camouflaged object segmentation. Comput Vis Image Underst 184:45–56

  33. Ledig C, Theis L, Huszr F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Honolulu, pp 4681–4690

  34. Li X, Lam KM, Qiu G, Shen L, Wang S (2009) Example-based image super-resolution with class-specific predictors. J Vis Commun Image Represent 20(5):312–322

    Article  Google Scholar 

  35. Lin D, Xu G, Xu W, Wang YX, Fu K (2020) SCRSR: An efficient recursive convolutional neural network for fast and accurate image super-resolution. Neurocomputing 398:399–407

    Article  Google Scholar 

  36. Liu YN, Zhang SS, Sang Y, Wang SM (2019) Improving image retrieval by integrating shape and texture features. Multimed Tools Appl 78:2525–2550

    Article  Google Scholar 

  37. Liu YN, Zhao L, Zhang SS, Yang J (2020) Hybrid resolution network using edge guided region mutual information loss for human parsing. ACM Multimedia, Seattle, pp 1670–1678

  38. Liu Y, Zhang S, Xu J, Yang J, Tai Y-W (2021) An accurate and lightweight method for human body image super-resolution. IEEE Trans Image Process 30:2888–2897

    Article  Google Scholar 

  39. Ma B, Shi YQ (2016) A reversible data hiding scheme based on code division multiplexing. IEEE Trans Inf Forensic Secur 11(9):1914–1927

    Article  Google Scholar 

  40. Mao X, Shen C, Yang Y-B (2016) Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Adv. Neural Inf. Process. Syst. (NIPS), Barcelona, pp 2802–2810

  41. Park SC, Park MK, Kang MG (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag 20(3):21–36

    Article  Google Scholar 

  42. Qiu Y, Wang R, Tao D, Cheng J (2019) Embedded block residual network: a recursive restoration model for Single-Image Super-Resolution. Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Seoul, pp 4179– 4188

  43. Sanchez-Beato A, Pajares G (2008) Noniterative interpolation-based super-resolution minimizing aliasing in the reconstructed image. IEEE Trans Image Process 17(10):1817–1826

    Article  MathSciNet  Google Scholar 

  44. Sang Y, Sun J, Wang S, Li K, Qi H (2020) Medical image Super-Resolution via granular Multi-Scale network in NSCT domain. IEEE Int. Conf. on Multimedia and Expo (ICME), London, pp

  45. Sang Y, Sun J, Wang S, Peng Y, Zhang X, Yang Z (2019) Multi-scale information distillation network for image super resolution in NSCT domain. Int. Conf. on Neural Information Processing (ICONIP), Sydney, pp 50–59

  46. Sha F, Zandavi SM, Chung YY (2019) Fast deep parallel residual network for accurate super resolution image processing. Expert Syst Appl 128:157–168

    Article  Google Scholar 

  47. Shi W, Caballero J, Huszr F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Las Vegas, pp 1874–1883

  48. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Honolulu, pp 3147–3155

  49. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: A persistent memory network for image restoration. In: Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Venice, pp 4539–4547

  50. Umehara K, Ota J, Ishida T (2018) Application of super-resolution convolutional neural network for enhancing image resolution in chest CT. J Digit Imaging 31(4):441–450

    Article  Google Scholar 

  51. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  52. Wang C, Wang S, Ma B, Li J, Dong X, Xia Z (2019) Transform domain based medical image super-resolution via deep multi-scale network. Proc. of IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), Brighton, pp 2387–2391

  53. Wang C, Wang X, Xia Z, Ma B, Shi Y (2020) Image description with polar harmonic fourier moments. IEEE Trans Circ Syst Video Technol 30 (12):4440–4452

    Article  Google Scholar 

  54. Wang C, Wang X, Xia Z, Zhang C (2019) Ternary radial harmonic Fourier moments based robust stereo image zero-watermarking algorithm. Inf Sci 470:109–120

    Article  Google Scholar 

  55. Xu W, Song H, Zhang K, Liu Q, Liu J (2020) Learning lightweight Multi-Scale Feedback Residual network for single image super-resolution. Comput Vis Image Underst 197-198:103005

  56. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Eur. Conf. Comput. Vis. (ECCV), Munich, pp 294–310

  57. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Salt Lake City, pp 2472–2481

  58. Zhong Z, Shen T, Yang Y, Zhang C, Lin Z (2018) Joint sub-bands learning with clique structures for wavelet domain super-resolution. Neural Information Processing Systems (NeurIPS), Montral, pp 165–175

  59. Zhou F, Yang W, Liao Q (2012) Interpolation-based image super-resolution using multisurface fitting. IEEE Trans Image Process 21(7):3312–3318

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Nos: 61802212 and 61872203), the Shandong Provincial Natural Science Foundation (No: ZR2019BF017), the Project of Shandong Province Higher Educational Science and Technology Program (No: J18KA331), China Postdoctoral Science Foundation (No: 2020M670728), Major Scientific and Technological Innovation Projects of Shandong Province (Nos: 2019JZZY010127, 2019JZZY010132, and 2019JZZY010201), Plan of Youth Innovation Team Development of Colleges and Universities in Shandong Province (No: SD2019-161) and Jinan City 20 universities Funding Projects Introducing Innovation Team Program (No: 2019GXRC031).

Author information

Authors and Affiliations

  1. School of Cyber Security, Qilu University of Technology (Shandong Academy of Sciences), Jinan, 250353, China

    Chunpeng Wang, Zhiqiu Xia, Qi Li, Bin Ma, Jian Li & Meihong Yang

  2. Shandong Provincial Key Laboratory of Computer Networks, Qilu University of Technology (Shandong Academy of Sciences), Shandong Computer Science Center (National Supercomputer Center in Jinan), Jinan, 250014, China

    Chunpeng Wang, Qi Li, Jian Li & Meihong Yang

  3. School of Electronic and Information Engineering, Liaoning Technical University, Huludao, 125105, China

    Simiao Wang

  4. Department of Electrical and Computer Engineering, New Jersey Institute of Technology, Newark, NJ, 07102, USA

    Yun-Qing Shi

Authors
  1. Chunpeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simiao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiqiu Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bin Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Meihong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yun-Qing Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Ma.

Ethics declarations

Competing interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C., Wang, S., Xia, Z. et al. Medical image super-resolution via deep residual neural network in the shearlet domain. Multimed Tools Appl 80, 26637–26655 (2021). https://doi.org/10.1007/s11042-021-10894-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10894-0

Keywords

Search

Navigation