Skip to main content
SpringerLink
Log in

Efficient local cascading residual network for real-time single image super-resolution

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

In the past decade, single image super-resolution (SISR) based on convolutional neural networks (CNNs) has been represented remarkable performance. Powerful characterization of CNN is important for recent methods to learn an intricate non-linear mapping between high-resolution and corresponding low-resolution images. However, a deeper and wider network structure brings superior performance while increasing the number of network parameters and calculations so that it is difficult to handle the real-time information. Hence, it can be embedded in mobile devices with difficulty. Inspired by the above motivation, a lightweight network for the real-time SISR is proposed by stacking efficient cascading residual blocks, which consist of several concatenate effective modules with wide activation. To further improve the network performance, with the increase of a slight number of parameters, the proposed network cooperates with a lightweight residual efficient channel attention module to capture feature interaction between channels. Extensive experiments provide significant demonstrations that the proposed network obtains the superior trade-off between performance and parameters compared with other current methods. The lightweight trait of our method allows it to implement real-time image processing and can be embedded in mobile devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Agustsson, E., Timofte, R.: Ntire 2017 challenge on single image super-resolution: dataset and study. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 126–135 (2017)

  2. Ahn, N., Kang, B., Sohn, K.A.: Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 252–268 (2018)

  3. Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding (2012)

  4. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: European Conference on Computer Vision, pp. 184–199. Springer (2014)

  5. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: European Conference on Computer Vision, pp. 391–407. Springer (2016)

  6. Gao, S., Cheng, M.M., Zhao, K., Zhang, X.Y., Yang, M.H., Torr, P.H.: Res2net: a new multi-scale backbone architecture. IEEE Trans. Pattern Anal. Mach. Intell. (2019)

  7. Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., Xu, C.: Ghostnet: more features from cheap operations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1580–1589 (2020)

  8. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  9. Howard, A., Zhmoginov, A., Chen, L.C., Sandler, M., Zhu, M.: Inverted residuals and linear bottlenecks: mobile networks for classification, detection and segmentation (2018)

  10. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861 (2017)

  11. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  12. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5197–5206 (2015)

  13. Hui, Z., Wang, X., Gao, X.: Fast and accurate single image super-resolution via information distillation network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 723–731 (2018)

  14. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K.: Squeezenet: Alexnet-level accuracy with 50x fewer parameters and< 0.5 mb model size. arXiv:1602.07360 (2016)

  15. Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. Adv. Neural Inf. Process. Syst. 28, 2017–2025 (2015)

    Google Scholar 

  16. Jin, X., Xiong, Q., Xiong, C., Li, Z., Gao, Z.: Single image super-resolution with multi-level feature fusion recursive network. Neurocomputing 370, 166–173 (2019)

    Article  Google Scholar 

  17. Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)

  18. Kim, J., Kwon Lee, J., Mu Lee, K.: Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1637–1645 (2016)

  19. Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Deep laplacian pyramid networks for fast and accurate super-resolution, pp. 5835–5843 (2017)

  20. Li, Q., Li, Z., Lu, L., Jeon, G., Liu, K., Yang, X.: Gated multiple feedback network for image super-resolution. arXiv:1907.04253 (2019)

  21. Li, Y., Hou, X., Koch, C., Rehg, J.M., Yuille, A.L.: The secrets of salient object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 280–287 (2014)

  22. Li, Z., Li, Q., Wu, W., Wu, Z., Lu, L., Yang, X.: Clustering based multiple branches deep networks for single image super-resolution. Multimed. Tools Appl. 79, 9019–9035 (2020)

    Article  Google Scholar 

  23. Li, Z., Li, Q., Wu, W., Yang, J., Li, Z., Yang, X.: Deep recursive up-down sampling networks for single image super-resolution. Neurocomputing 398, 377–388 (2020)

    Article  Google Scholar 

  24. Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 136–144 (2017)

  25. Ma, N., Zhang, X., Zheng, H.T., Sun, J.: Shufflenet v2: Practical guidelines for efficient CNN architecture design. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 116–131 (2018)

  26. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol. 2, pp. 416–423. IEEE (2001)

  27. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: IEEE Conference on Computer Vision and Pattern Recognition (2017)

  28. Tai, Y., Yang, J., Liu, X., Xu, C.: [IEEE 2017 IEEE International Conference on Computer Vision (ICCV) - venice (2017.10.22–2017.10.29)] 2017 IEEE International Conference on Computer Vision (ICCV)—memnet: A Persistent Memory Network for Image Restoration, pp. 4549–4557 (2017)

  29. Tian, C., Zhuge, R., Wu, Z., Xu, Y., Zuo, W., Chen, C., Lin, C.W.: Lightweight image super-resolution with enhanced CNN. Knowl. Based Syst. 205, 106235 (2020)

    Article  Google Scholar 

  30. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, pp. I–I. IEEE (2001)

  31. Wang, Q., Wu, B., Zhu, P., Li, P., Hu, Q.: Eca-net: efficient channel attention for deep convolutional neural networks. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

  32. Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7794–7803 (2018)

  33. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  34. Woo, S., Park, J., Lee, J.Y., So Kweon, I.: Cbam: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  35. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: International Conference on Curves and Surfaces, pp. 711–730. Springer (2010)

  36. Zhang, K., Zuo, W., Zhang, L.: Learning a single convolutional super-resolution network for multiple degradations. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018)

  37. Zhang, X., Zhou, X., Lin, M., Sun, J.: Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6848–6856 (2018)

  38. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 286–301 (2018)

  39. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2472–2481 (2018)

Download references

Acknowledgements

The research in our paper is sponsored by National Key R and D Program of China, under Grant No. 2020AAA0-104500. The funding is from Sichuan University under grant 2020SCUNG205.

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Sichuan University, Chengdu, 610064, Sichuan, China

    Haoran Yang & Xiaomin Yang

  2. School of Aeronautics and Astronautics, Sichuan University, Chengdu, China

    Haoran Yang & Xiaomin Yang

  3. College of Electrical Engineering, Sichuan University, Chengdu, 610065, Sichuan, China

    Kai Liu

  4. The Center of Gerontology and Geriatrics, West China Hospital, Sichuan University, Chengdu, China

    Qingyu Dou

  5. TAL Education Group, Beijing, China

    Zitao Liu

  6. Department of Computer Science, University of Salerno, Via Giovanni Paolo II, 13284084, Fisciano, SA, Italy

    Rita Francese

Authors
  1. Haoran Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qingyu Dou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zitao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rita Francese
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiaomin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Qingyu Dou or Kai Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, H., Dou, Q., Liu, K. et al. Efficient local cascading residual network for real-time single image super-resolution. J Real-Time Image Proc 18, 1235–1246 (2021). https://doi.org/10.1007/s11554-021-01134-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-021-01134-7

Keywords

Search

Navigation