Skip to main content
SpringerLink
Log in

Image super-resolution by learning weighted convolutional sparse coding

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Single image super-resolution (SISR) has witnessed substantial progress recently by deep learning-based methods, due to the data-driven end-to-end training. However, most existing DL-based models are built intuitively, with little thought on priors. And the lack of interpretability limits their further improvements. To avoid this, this paper presents an end-to-end trainable unfolding network which leverages both DL- and prior-based methods. Specifically, we introduce the reweighted algorithm into CSC model and solve it by learning weighted iterative soft thresholding algorithm in a convolutional manner. Based on this, we present a SISR model by learning weighted convolutional sparse coding, in which the channel attention is resorted to learn the weight. Extensive experiments demonstrate the superiority of our method to recent state-of-the-art SISR methods, in terms of both quantitative and qualitative results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Agustsson, E., Timofte, R.: NTIRE 2017 challenge on single image super-resolution: Dataset and study. In: CVPRW, pp. 126–135 (2017)

  2. Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: BMVC (2012)

  3. Cai, J., Gu, S., Timofte, R., Zhang, L.: Ntire 2019 challenge on real image super-resolution: methods and results. In: CVPRW (2019)

  4. Cai, J., Zeng, H., Yong, H., Cao, Z., Zhang, L.: Toward real-world single image super-resolution: a new benchmark and a new model. In: ICCV (2019)

  5. Candes, E.J., Wakin, M.B., Boyd, S.P.: Enhancing sparsity by reweighted \(\ell _{1}\) minimization. J. Fourier Anal. Appl. 14(5–6), 877–905 (2008)

    Article  MathSciNet  Google Scholar 

  6. Chartrand, R.: Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Process. Lett. 14(10), 707–710 (2007)

    Article  Google Scholar 

  7. Dai, S., Han, M., Xu, W., Wu, Y., Gong, Y.: Soft edge smoothness prior for alpha channel super resolution. CVPR 7, 1–8 (2007)

    Google Scholar 

  8. Dai, T., Cai, J., Zhang, Y.B., Xia, S., Zhang, L.: Second-order attention network for single image super-resolution. In: CVPR, pp. 11057–11066 (2019)

  9. Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. A J. Issued Courant Inst. Math. Sci. 57(11), 1413–1457 (2004)

    Article  MathSciNet  Google Scholar 

  10. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. TPAMI 38, 295–307 (2016)

    Article  Google Scholar 

  11. Dong, W., Wang, P., Yin, W., Shi, G., Wu, F., Lu, X.: Denoising prior driven deep neural network for image restoration. TPAMI 41, 2305–2318 (2019)

    Article  Google Scholar 

  12. Donoho, D.L., et al.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  Google Scholar 

  13. Foucart, S., Lai, M.J.: Sparsest solutions of underdetermined linear systems via \(\ell _{q}\)-minimization for 0\(<\)q\(\le \)1. Appl. Comput. Harmon. Anal. 26(3), 395–407 (2009)

    Article  MathSciNet  Google Scholar 

  14. Fu, X., Zha, Z., Wu, F., Ding, X., Paisley, J.W.: Jpeg artifacts reduction via deep convolutional sparse coding. In: ICCV, pp. 2501–2510 (2019)

  15. Gregor, K., LeCun, Y.: Learning fast approximations of sparse coding. In: ICML (2010)

  16. Guo, Z., Chen, Z., Yu, T., Chen, J., Liu, S.: Progressive image inpainting with full-resolution residual network. In: ACM multimedia, pp. 2496–2504 (2019)

  17. Haris, M., Shakhnarovich, G., Ukita, N.: Deep back-projection networks for super-resolution. In: CVPR (2018)

  18. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: ICCV, pp. 1026–1034 (2015)

  19. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)

  20. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: CVPR, pp. 7132–7141 (2018)

  21. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: CVPR, pp. 5197–5206 (2015)

  22. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: CVPR, pp. 1646–1654 (2016)

  23. Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: CVPR, pp. 1637–1645 (2016)

  24. Ledig, C., Theis, L., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: CVPR (2016)

  25. Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: CVPRW, pp. 1132–1140 (2017)

  26. Liu, R., Cheng, S., He, Y., Fan, X., Lin, Z., Luo, Z.: On the convergence of learning-based iterative methods for nonconvex inverse problems. In: TPAMI (2019)

  27. Liu, Z., Yu, L., Sun, H.: Image denoising via nonlocal low rank approximation with local structure preserving. IEEE Access 7, 7117–7132 (2019)

    Article  Google Scholar 

  28. Lyu, Q., Lin, Z., She, Y., Zhang, C.: A comparison of typical \(\ell _{p}\) minimization algorithms. Neurocomputing 119, 413–424 (2013)

    Article  Google Scholar 

  29. Mairal, J., Bach, F.R., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: ICCV, pp. 2272–2279 (2009)

  30. Martin, D.R., Fowlkes, C.C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV (2001)

  31. Papyan, V., Romano, Y., Elad, M.: Convolutional neural networks analyzed via convolutional sparse coding. J. Mach. Learn. Res. 18, 83:1–83:52 (2017)

    MathSciNet  MATH  Google Scholar 

  32. Pérez-Pellitero, E., Salvador, J., Hidalgo, J.R., Rosenhahn, B.: PSyCo: manifold span reduction for super resolution. In: CVPR, pp. 1837–1845 (2016)

  33. Peyré, G., Bougleux, S., Cohen, L.: Non-local regularization of inverse problems. In: ECCV (2008)

  34. Rabaud, V., Belongie, S.: Big little icons. In: CVPRW, p. 24 (2005)

  35. Simon, D., Elad, M.: Rethinking the csc model for natural images. In: NeurIPS (2019)

  36. Sreter, H., Giryes, R.: Learned convolutional sparse coding. In: ICASSP, pp. 2191–2195 (2018)

  37. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: CVPR, pp. 2790–2798 (2017)

  38. Tai, Y., Yang, J.X., Liu, X., Xu, C.: MemNet: a persistent memory network for image restoration. In: ICCV, pp. 4549–4557 (2017)

  39. Wang, F., Jiang, M., Qian, C., Yang, S., Li, C.C., Zhang, H., Wang, X., Tang, X.: Residual attention network for image classification. In: CVPR, pp. 6450–6458 (2017)

  40. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. TIP 13, 600–612 (2004)

    Google Scholar 

  41. Wang, Z., Liu, D., Yang, J., Han, W., Huang, T.: Deep networks for image super-resolution with sparse prior. In: ICCV, pp. 370–378 (2015)

  42. Xie, C., Liu, Y., Zeng, W., Lu, X.: An improved method for single image super-resolution based on deep learning. SIVP 13(3), 557–565 (2019)

    Google Scholar 

  43. Xie, Z., Hu, J.: Rewighted \(\ell _{1}\)-minimization for sparse solutions to underdetermined linear systems. In: CISP, vol. 3, pp. 1660–1664. IEEE (2013)

  44. Yang, J., Wright, J.N., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. TIP 19, 2861–2873 (2010)

    MathSciNet  MATH  Google Scholar 

  45. Zeiler, M.D., Krishnan, D., Taylor, G.W., Fergus, R.: Deconvolutional networks. In: CVPR, pp. 2528–2535 (2010)

  46. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Curves and Surfaces (2010)

  47. Zha, Z., Liu, X., Huang, X., Shi, H., Xu, Y., Wang, Q., Tang, L., Zhang, X.: Analyzing the group sparsity based on the rank minimization methods. In: ICME, pp. 883–888 (2017)

  48. Zhang, J., Cao, Y., Wang, Z.: A new image filtering method: nonlocal image guided averaging, pp. 2460–2464 (2014)

  49. Zhang, J., Cao, Y., Zha, Z., Zheng, Z., Chen, C.W., Wang, Z.: A unified scheme for super-resolution and depth estimation from asymmetric stereoscopic video. TCSVT 26, 479–493 (2016)

    Google Scholar 

  50. Zhang, J., Tao, D.: Famed-net: a fast and accurate multi-scale end-to-end dehazing network. TIP 29, 72–84 (2020)

    MathSciNet  Google Scholar 

  51. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. TIP 26(7), 3142–3155 (2017)

    MathSciNet  MATH  Google Scholar 

  52. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: ECCV (2018)

  53. Zuo, W., Meng, D., Zhang, L., Feng, X., Zhang, D.: A generalized iterated shrinkage algorithm for non-convex sparse coding. In: ICCV, pp. 217–224 (2013)

Download references

Acknowledgements

This work is supported by NSFC (Grant No. 61871297).

Author information

Authors and Affiliations

  1. Electronic Information School, Wuhan University, Wuhan, 430072, China

    Jingwei He, Lei Yu, Zhou Liu & Wen Yang

Authors
  1. Jingwei He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhou Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wen Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Yu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, J., Yu, L., Liu, Z. et al. Image super-resolution by learning weighted convolutional sparse coding. SIViP 15, 967–975 (2021). https://doi.org/10.1007/s11760-020-01821-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-020-01821-1

Keywords

Search

Navigation