Skip to main content
SpringerLink
Log in

Progressive residual networks for image super-resolution

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The recent advances in deep convolutional neural networks (DCNNs) have convincingly demonstrated high-capability reconstruction for single image super-resolution (SR). However, it is a big challenge for most DCNNs-based SR models when the scaling factor increases. In this paper, we propose a novel Progressive Residual Network (PRNet) to integrate hierarchical and scale features for single image SR, which works well for both small and large scaling factors. Specifically, we introduce a Progressive Residual Module (PRM) to extract local multi-scale features through dense connected up-sampling convolution layers. Meanwhile, by embedding residual learning into each module, the relative information between high-resolution and low-resolution multi-scale features is fully exploited to boost reconstruction performance. Finally, the scale-specific features are fused to the reconstruction module for restoring the high-quality image. Extensive quantitative and qualitative evaluations on benchmark datasets illustrate that our PRNet achieves superior performance and in particular obtains new state-of-the-art results for large scaling factors such as 4 × and 8 ×.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Agustsson E, Timofte R (2017) Ntire 2017 challenge on single image super-resolution: Dataset and study. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 126–135

  2. Bevilacqua M, Roumy A, Guillemot C, AlberiMorel ML (2012) Low-complexity singleimage super-resolution based on nonnegative neighbor embedding. In: Proceedings of the 23rd British Machine Vision Conference, pp 1–10

  3. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: Proceedings of European conference on computer vision, pp 184– 199

  4. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: Proceedings of European conference on computer vision, pp 391–407

  5. Facebook (2017) Pytorch. https://pytorch.org/

  6. Gao X, Zhang K, Tao D, Li X (2012) Image super-resolution with sparse neighbor embedding. IEEE Trans Image Process 21(7):3194–3205

    Article  MathSciNet  Google Scholar 

  7. Haris M, Shakhnarovich G, Ukita N (2018) Deep back-projection networks for super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 1664–1673

  8. Huang G, Liu Z, Maaten LVD, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4700–4708

  9. Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 5197–5206

  10. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 1646–1654

  11. Kim J, Kwon Lee J, Mu Lee K (2016) Deeply-recursive convolutional network for image super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 1637–1645

  12. Kingma D, Ba J (2014) Adam: a method for stochastic optimization. In: Proceedings of International Conference on Learning Representations

  13. Kuang P, Ma T, Chen Z, Li F (2019) Image super-resolution with densely connected convolutional networks. Appl Intell 49(1):125–136

    Article  Google Scholar 

  14. Lai W, Huang J, Ahuja N, Yang M (2018) Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp 1–1

  15. Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 624–632

  16. Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 4681– 4690

  17. Li J, Fang F, Mei K, Zhang G (2018) Multi-scale residual network for image super-resolution. In: Proceedings of European Conference on Computer Vision, pp 517–532

  18. Lim B, Son S, Kim H, Nah S, Mu Lee K (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 136–144

  19. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of IEEE International conference on computer vision, pp 416–423

  20. MatsuiY IK, Aramaki Y, Fujimoto A, Ogawa T, Yamasaki T, Aizawa K (2017) Sketch-based manga retrieval using manga109 dataset. Multimedia Tools and Applications, pp 21811–21838

  21. Schulter S, Leistner C, Bischof H (2015) Fast and accurate image upscaling with super-resolution forests. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp 3791–3799

  22. Shi W, Caballero J, Huszar F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2015) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 1874–1883

  23. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 3147–3155

  24. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: A persistent memory network for image restoration. In: Proceedings of IEEE International Conference on Computer Vision, pp 4539–4547

  25. Timofte R, De Smet V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. In: Proceedings of IEEE International Conference on Computer Vision, pp 1920–1927

  26. Timofte R, Rothe R, Van Gool L (2016) Seven ways to improve example-based single image super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 1865–1873

  27. Tong T, Li G, Liu X, Gao Q (2017) Image super-resolution using dense skip connections. In: Proceedings of IEEE International Conference on Computer Vision, pp 4799–4807

  28. Zagoruyko S, Komodakis N (2017) Diracnets: Training very deep neural networks without skip-connections. arXiv:170600388

  29. Zeyde R, Elad M, Protter M (2012) On single image scale-up using sparse-representations. In: Curves and Surfaces, pp 711–730

  30. Zhang K, Gao X, Tao D, Li X (2012) Single image super-resolution with non-local means and steering kernel regression. IEEE Trans Image Process, pp 4544–4556

  31. Zhang L, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process, pp 2226–2238

  32. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 2472–2481

  33. Zhao L, Li M, Meng D, Li X, Zhang Z, Zhuang Y, Tu Z, Wang J (2018) Deep convolutional neural networks with merge-and-run mappings. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp 3170–3176

Download references

Acknowledgments

This work is supported by National Nature Science Foundation of China (61472029, 51827813, 61473031), National Key R&D Program of China (2017YFB1201104, 2016YFB1200100), and Scientific Research Project of Beijing Educational Committee (SM20191001107; PXM 2019_014213_000007).

Author information

Authors and Affiliations

  1. Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing Jiaotong University, Beijing, 100044, China

    Jin Wan, Hui Yin & Zhi-Hao Liu

  2. Key Laboratory of Beijing for Railway Engineering, Beijing Jiaotong University, Beijing, 100044, China

    Ai-Xin Chong

Authors
  1. Jin Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ai-Xin Chong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhi-Hao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hui Yin.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wan, J., Yin, H., Chong, AX. et al. Progressive residual networks for image super-resolution. Appl Intell 50, 1620–1632 (2020). https://doi.org/10.1007/s10489-019-01548-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01548-8

Keywords

Search

Navigation