Skip to main content
SpringerLink
Log in

Paradigm shifts in super-resolution techniques for remote sensing applications

  • Survey
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Super-resolution (SR) algorithms have now become a bottleneck for several remote sensing applications. SR is a technique that enhances minute details of the image by increasing spatial resolution of imaging systems. SR overcomes the problems of conventional resolution enhancement techniques such as introduction of noise, spectral distortion, and lack of clarity in the details of the image. In this paper, a survey has been conducted since the inception of SR algorithm till the latest state-of-the-art SR techniques to elucidate the importance of the SR algorithms that lead to paradigm shifts in the last two decades revolutionizing toward visually pleasing high-resolution image. Inspired from the natural images, the algorithms addressing the SR problems such as ill-posed, prior and regularization problem, inverse problem, multi-frame problem and illumination and shadow problem in remote sensing applications are analyzed. For an intuitive understanding of the paradigm shifts, publicly available images are tested with representative paradigm shift SR algorithms. The result of this paradigm shift analysis is done both qualitatively and quantitatively in terms of blurs in the image, pattern clarity, edge strength, and super-resolving capability. The convergence of the natural image to the remote sensed image is critically analyzed. The challenges with possible solutions for super-resolving the remote sensed image are recommended. On experimentation, it is found that deep learning-based SR algorithms produces visually pleasing images retaining sharp edges, enhanced spatial data, and clarity in feature representation while zooming at a certain level beyond interest.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20(3), 21–36 (2003). https://doi.org/10.1109/MSP.2003.1203207

    Article  Google Scholar 

  2. Nasrollahi, K., Moeslund, T.B.: Super-resolution: a comprehensive survey. Mach. Vis. Appl. 25, 1423–1468 (2014). https://doi.org/10.1007/s00138-014-0623-4

    Article  Google Scholar 

  3. Tsai, R., Huang, T.: Multiframe image restoration and registration. In: Tsai, R.Y., Huang, T.S. (eds.) Advances in Computer Vision and Image Processing, vol. 1, pp. 317–339 (1984)

  4. Demirel, H., Anbarjafari, G.: IMAGE resolution enhancement by using discrete and stationary wavelet decomposition. IEEE Trans. Image Process. 20(5), 1458–1460 (2011). https://doi.org/10.1109/TIP.2010.2087767

    Article  MathSciNet  MATH  Google Scholar 

  5. Li, F., Jia, X., Fraser, D.: Universal HMT based super resolution for remote sensing images. In: 15th IEEE International Conference on Image Processing, San Diego, CA, pp. 333–336 (2008). https://doi.org/10.1109/icip.2008.4711759

  6. Tom, B.C., Galatsanos, N.P., Katsaggelos, A.K.: Reconstruction of a high resolution image from multiple low resolution images. In: Chaudhuri, S. (ed.) Super-Resolution Imaging, vol. 4, pp. 73–105. Kluwer, Norwell (2002)

    Google Scholar 

  7. Gotoh, T., Okutomi, M.: Direct super-resolution and registration using raw CFA images. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2004, Washington, DC, USA, pp. II–II (2004). https://doi.org/10.1109/cvpr.2004.1315219

  8. Yang, C.Y., Ma, C., Yang, M.H.: Single-image super-resolution: a benchmark. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) Computer Vision–ECCV 2014. Lecture Notes in Computer Science, vol. 8692. Springer, Cham (2014)

    Google Scholar 

  9. Singh, A., Porikli, F., Ahuja, N.: Super-resolving noisy images. In: IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, pp. 2846–2853 (2014). https://doi.org/10.1109/cvpr.2014.364

  10. Peleg, S., Keren, D., Schweitzer, L.: Improving image resolution using subpixel motion. Pattern Recogn. Lett. 5(3), 223–226 (1987). https://doi.org/10.1016/0167-8655(87)90067-5

    Article  Google Scholar 

  11. Keren, D., Peleg, S., Brada, R.: Image sequence enhancement using sub-pixel displacements. In: Proceedings CVPR ‘88: The Computer Society Conference on Computer Vision and Pattern Recognition, Ann Arbor, MI, USA, pp. 742–746 (1988). https://doi.org/10.1109/cvpr.1988.196317

  12. Shukla, K.K., Tiwari, S.K., Arvind, K.: Efficient Algorithms for Discrete Wavelet Transform, With Applications to Denoising and Fuzzy Inference Systems. Springer, Berlin (2014). https://doi.org/10.1007/978-1-4471-4941-5

    Book  MATH  Google Scholar 

  13. Milanfar, P.: Super Resolution Imaging. CRC Press, Boca Raton (2010). https://doi.org/10.1201/9781439819319

    Book  Google Scholar 

  14. Yue, L., Shen, H., Li, J., Yuan, Q., Zhang, H., Zhang, L.: Image super-resolution: the techniques, applications, and future. Signal Process. (2016). https://doi.org/10.1016/j.sigpro.2016.05.002

    Article  Google Scholar 

  15. Ha, V.K., Ren, J., Xu, X., et al.: Deep learning based single image super-resolution: a survey. Int. J. Autom. Comput. 16, 413–426 (2019)

    Google Scholar 

  16. Kumar, N., Verma, R., Sethi, A.: Convolutional neural networks for wavelet domain super resolution. Pattern Recogn. Lett. 90, 65–71 (2017). https://doi.org/10.1016/j.patrec.2017.03.014

    Article  Google Scholar 

  17. Hayat, K.: Multimedia super-resolution via deep learning: a survey. Digit. Signal Process. 81, 198–217 (2018)

    Google Scholar 

  18. Tsagkatakis, G., Aidini, A., Fotiadou, K., Giannopoulos, M., Pentari, A., Tsakalides, P.: Survey of deep-learning approaches for remote sensing observation enhancement. Sensors 19(18), 3929 (2019)

    Google Scholar 

  19. Nguyena, K., et al.: Super resolution super-resolution for biometrics: a comprehensive survey. Pattern Recognit. 78, 23–42 (2018)

    Google Scholar 

  20. Wang, J., Gong, Y.: Image and video super resolution techniques. In: Furht, B. (ed.) Encyclopedia of Multimedia. Springer, Boston (2008). https://doi.org/10.1007/978-0-387-78414-4

    Chapter  Google Scholar 

  21. Abd El-Samie, F.E., Hadhoud, M.M., El-Khamy, S.E.: Image Super-Resolution and Applications. CRC Press, Boca Raton (2012)

    MATH  Google Scholar 

  22. Wang, Y., Fevig, R., Schultz, R.R.: Super-resolution mosaicking of UAV surveillance video. In: 15th IEEE International Conference on Image Processing, San Diego, CA, 2008, pp. 345–348 (2008). https://doi.org/10.1109/icip.2008.4711762

  23. Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2004, Washington, DC, USA, pp. I–I (2004). https://doi.org/10.1109/cvpr.2004.1315043

  24. Lanaras, C., Bioucas-Dias, J., Galliani, S., Baltsavias, E., Schindler, K.: Super-resolution of sentinel-2 images: learning a globally applicable deep neural network. ISPRS J. Photogrammet. Remote Sens. 146, 305–319 (2018)

    Google Scholar 

  25. Rubert, C., Fonseca, L., Velho, L.: Learning based super- resolution using YUV model for remote sensing images. In: Proceedings of Workshop of Theses and Dissertations in Computer Graphics and Image Processing (2005)

  26. Keshk, H.M., Yin, X.: Satellite super-resolution images depending on deep learning methods: a comparative study. In: IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), Xiamen, pp. 1–7. (2017). https://doi.org/10.1109/icspcc.2017.8242625

  27. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) Computer Vision. ECCV 2014. Lecture Notes in Computer Science, vol. 8692. Springer, Cham (2014)

    Google Scholar 

  28. Pandey, G., Ghanekar, U.: Classification of priors and regularization techniques appurtenant to single image super-resolution. Vis. Comput. (2019). https://doi.org/10.1007/s00371-019-01729-z

    Article  Google Scholar 

  29. Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced Deep Residual Networks for Single Image Super-Resolution, pp. 1132–1140 (2017). https://doi.org/10.1109/cvprw.2017.151

  30. Dong, C., Loy, C.C., Tang, X.: Accelerating the Super-Resolution Convolutional Neural Network. 9906, pp. 391–407 (2016). https://doi.org/10.1007/978-3-319-46475-6_25

  31. Kim, J., Lee, J.K., Lee, K.M.: Accurate Image Super-Resolution Using Very Deep Convolutional Networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp. 1646–1654 (2016)

  32. Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp. 1637–1645 (2016)

  33. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Google Scholar 

  34. AbduRahiman, V., George, S.N.: Single image super resolution using neighbor embedding and statistical prediction model. Comput. Electr. Eng. 62, 281–292 (2017). https://doi.org/10.1016/j.compeleceng.2016.12.018

    Article  Google Scholar 

  35. Abdu Rahiman, V., Rohit, U., George, S.N.: Modified dictionary learning method for sparsity based single image super-resolution. In: 3rd International Conference on Recent Advances in Information Technology (RAIT), Dhanbad, pp. 473–477 (2016). https://doi.org/10.1109/rait.2016.7507947

  36. Zhang, D., He, J., Du, M.: Morphable model space based face super-resolution reconstruction and recognition. Image Vis. Comput. 30, 100–108 (2012). https://doi.org/10.1016/j.imavis.2012.01.005

    Article  Google Scholar 

  37. Yuan, Q., Zhang, L., Shen, H.: Multiframe super-resolution employing a spatially weighted total variation model. IEEE Trans. Circuits Syst. Video Technol. 22(3), 379–392 (2012). https://doi.org/10.1109/tcsvt.2011.2163447

    Article  Google Scholar 

  38. Panda, S.S., Prasad, M.S.R., Jena, G.: POCS based super-resolution image reconstruction using an adaptive regularization parameter. IJCSI Int. J. Comput. Sci. 8 (2011)

  39. Nguyen, N., Milanfar, P., Golub, G.: A computationally efficient super resolution image reconstruction algorithm. IEEE Trans. Image Process. 10, 4 (2001)

    MATH  Google Scholar 

  40. Yang, S., Hu, Y.H., Nguyen, T.Q., Tull, D.L.: Maximum-likelihood parameter estimation for image ringing-artifact removal. IEEE Trans. Circuits Syst. Video Technol. 11(8), 963–973 (2001)

    Google Scholar 

  41. Kim, S.P., Bose, N.K., Valenzuela, H.M.: Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Trans. Acoust. Speech Signal Process. 38(6), 1013–1027 (1990). https://doi.org/10.1109/29.56062

    Article  Google Scholar 

  42. Jiji, C.V., Chaudhuri, S., Chatterjee, P.: Single frame image super-resolution: should we process locally or globally? Multidimen. Syst. Sign. Process. 18, 123–152 (2007). https://doi.org/10.1007/s11045-007-0024-1

    Article  MathSciNet  MATH  Google Scholar 

  43. Sakurai, M., Sakuta, Y., Watanabe, M., Goto, T., Hirano, S.: Super-resolution through non-linear enhancement filters. In: IEEE International Conference on Image Processing, Melbourne, VIC, pp. 854–858 (2013)

  44. Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)

    Google Scholar 

  45. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, S., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680(2014)

  46. Aymaz, S., Köse, C.: A novel image decomposition-based hybrid technique with super-resolution method for multi-focus image fusion. Infor. Fusion 45, 113–127 (2019)

    Google Scholar 

  47. Rajaram, S., Gupta, M.D., Petrovic, N., Huang, T.S.: Learning-based nonparametric image super-resolution. EURASIP J. Adv. Signal Process. 206(1), 051306 (2006)

    Google Scholar 

  48. Cheeseman, P., Kanefsky, B., Kraft, R., Stutz, J., Hanson, R.: Super-resolved surface reconstruction from multiple images. In: Heidbreder, G.R. (ed.) Maximum Entropy and Bayesian Methods. Fundamental Theories of Physics. An International Book Series on The Fundamental Theories of Physics: Their Clarification, Development and Application, vol. 62. Springer, Dordrecht (1996)

    Google Scholar 

  49. Pohl, C., Van Genderen, J.L.: Multisensor image fusion in remote sensing: concepts, methods and applications. Int. J. Remote Sens. 19(5), 823–854 (1998)

    Google Scholar 

  50. Li, L., Wang, W., Luo, H., Ying, S.: Super-resolution reconstruction of high-resolution satellite ZY-3 TLC images. Sensors 17(5), 1062 (2017)

    Google Scholar 

  51. Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Fast and robust multi-frame super-resolution. IEEE Trans. Image Process. 13(10), 1327–1344 (2004)

    Google Scholar 

  52. Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010). https://doi.org/10.1109/TIP.2010.2050625

    Article  MathSciNet  MATH  Google Scholar 

  53. Abdu Rahiman, V., George, S.N.: Robust single image super resolution using neighbor embedding and fusion in wavelet domain. Comput. Electric. Eng. 70, 674–689 (2018)

    Google Scholar 

  54. Venkatanath, N., Praneeth, D., Chandrasekhar, B.M., Channappayya, S.S., Medasani, S.S.: Blind image quality evaluation using perception based features. In: Proceedings of the 21st National Conference on Communications (NCC). IEEE, Piscataway

  55. Yang, J., Wang, Z., Lin, Z., Cohen, S., Huang, T.: Coupled dictionary training for image super-resolution. IEEE Trans. Image Process. 21(8), 3467–3478 (2012). https://doi.org/10.1109/TIP.2012.2192127

    Article  MathSciNet  MATH  Google Scholar 

  56. Wang, H., Feng, L., Yu, L., Zhang, J.: Multi-view sparsity preserving projection for dimension reduction. Neurocomputing 216, 286–295 (2016). https://doi.org/10.1016/j.neucom.2016.07.044

    Article  Google Scholar 

  57. Li, B., Zhou, Y., Zhang, Y., Wang, A.: Depth image super-resolution based on joint sparse coding. Pattern Recogn. Lett. 130, 21–29 (2020)

    Google Scholar 

  58. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, pp. 3431–3440 (2015)

  59. Miravet, C., Rodriguez, F.B.: A two step neural network based algorithm for fast image super-resolution. Image Vis. Comput. 25, 1473–1499 (2007)

    Google Scholar 

  60. Gajjar, P.P., Joshi, M.: Zoom based super-resolution a fast approach using particle swarm optimization. In: Image and Signal Processing. Lecture Notes in Computer Science, vol. 6134, pp. 63–70 (2010)

  61. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533 (1986)

    MATH  Google Scholar 

  62. Stark, H., Oskoui, P.: High-resolution image recovery from image-plane arrays, using convex projections. JOSA A 6, 1715–1726 (1989)

    Google Scholar 

  63. Bose, N.K., Kim, H.C., Valenzuela, H.M.: Recursive implementation of total least squares algorithm for image reconstruction from noisy, undersampled multiframes. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, Minneapolis, MN, USA, vol. 5, pp. 269–272 (1993). https://doi.org/10.1109/icassp.1993.319799

  64. Lu, J., Hu, W., Sun, Y.: A deep learning method for image super-resolution based on geometric similarity. Signal Process. Image Commun. 70, 210–219 (2019)

    Google Scholar 

  65. Märtens, M., Izzo, D., Krzic, A., et al.: Super-resolution of PROBA-V images using convolutional neural networks. Astrodynamics 3, 387–402 (2019). https://doi.org/10.1007/s42064-019-0059-8

    Article  Google Scholar 

  66. Shi, W., Caballero, J., Husar, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874–1883 (2016)

  67. Witwit, W., Zhao, Y., Jenkins, K., Addepalli, S.: Global motion based video super-resolution reconstruction using discrete wavelet transform. Multimed. Tools Appl. 77(20), 27641–27660 (2018)

    Google Scholar 

  68. Freeman, W.T., Pasztor, E.C.: Learning to estimate scenes from images. In: Kearns, M.S., Solla, S.A., Cohn, D.A. (eds.) Advances in Neural Information Processing Systems, vol. 11 (1999)

  69. Freeman, W.T., Pasztor, E.: Markov networks for low-level vision. Mitsubishi Electric Research Laboratory Technical, Report TR99 (1999)

  70. Gunturk, B.K., Altunbasak, Y., Mersereau, R.M.: Multiframe resolution-enhancement methods for compressed video. IEEE Signal Process. Lett. 9(6), 170–174 (2002). https://doi.org/10.1109/LSP.2002.800503

    Article  Google Scholar 

  71. Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006). https://doi.org/10.1109/TSP.2006.881199

    Article  MATH  Google Scholar 

  72. Schultz, R.R., Meng, L., Stevenson, R.L.: Subpixel motion estimation for super-resolution image sequence enhancement. J. Vis. Commun. Image Represent. 9(1), 38–50 (1998)

    Google Scholar 

  73. Li, K., Cao, F.: Super-resolution using neighbourhood regression with local structure prior. Signal Process. Image Commun. 72, 58–68 (2019)

    Google Scholar 

  74. Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012)

    MathSciNet  MATH  Google Scholar 

  75. Efrat, N., Glasner, D., Apartsin, A., Nadler, B., Levin, A.: Accurate blur models vs. image priors in single image super-resolution. In: IEEE International Conference on Computer Vision (ICCV), pp. 2832–2839 (2013)

  76. Ledig, C., Theis, L., Huszar, F., et al. Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE Conference on Computer Vision and Pattern Recognition (2017)

  77. Zhu, J.Y, Park, T., Isola, P., et al.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

  78. Zeng, K., Ding, S.F.: Single image super-resolution using a polymorphic parallel CNN. Appl. Intell. 49(1), 292–300 (2019)

    Google Scholar 

  79. Wu, J., Yue, T., Shen, Q., Cao, X., Ma, Z.: Multiple-image super resolution using both reconstruction optimization and deep neural network. In: IEEE Global Conference on Signal and Information Processing (GlobalSIP) (2018)

  80. Xu, K., Wang, X., Yang, X., et al.: Efficient image super-resolution integration. Vis. Comput. 34, 1065–1076 (2018). https://doi.org/10.1007/s00371-018-1554-2

    Article  Google Scholar 

  81. Chen, C., Liang, H., Zhao, S., et al.: A novel multi-image super-resolution reconstruction method using anisotropic fractional order adaptive norm. Vis. Comput. 31, 1217–1231 (2015). https://doi.org/10.1007/s00371-014-1007-5

    Article  Google Scholar 

  82. Sadaka, N.G., Karam, L.J.: Efficient super-resolution driven by saliency selectivity. In: 18th IEEE International Conference on Image Processing, IEEE xplore (2011). https://doi.org/10.1109/icip.2011.6115645

  83. Zhao, J.X., Cao, Y., Fan, D.P., Cheng, M.M., Li, X.Y., Zhang, L.: Contrast prior and fluid pyramid integration for RGBD salient object detection. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Xplore (2019). https://doi.org/10.1109/cvpr.2019.00405

  84. Fu, K., Zhao, Q., Gu, I.Y.H., Yang, J.: Deepside: a general deep framework for salient object detection. Neurocomputing (2019). https://doi.org/10.1016/j.neucom.2019.04.062

    Article  Google Scholar 

  85. Fu, K., Zhao, Q., Gu, I.: Refinet: a deep segmentation assisted refinement network for salient object detection. IEEE Trans. Multimed. (2018). https://doi.org/10.1109/tmm.2018.2859746

    Article  Google Scholar 

  86. Fan, D.P., Cheng, M.M., Liu, J.J., Gao, S.H., Hou, Q., Borji, A.: Salient objects in clutter: bringing salient object detection to the foreground. In: Computer Vision and Pattern Recognition, ECCV 2018. arXiv:1803.06091 (2018)

  87. Xiong, Y., Shao, F., Meng, X., Zhou, B., Ho, Y.: Sparse representation for no-reference quality assessment of satellite stereo images. IEEE Access 7, 106295–106306 (2019). https://doi.org/10.1109/ACCESS.2019.2932015

    Article  Google Scholar 

  88. Pendurkar, S., Banerjee, B., Saha, S., Bovolo, F.: single image super-resolution for optical satellite scenes using deep deconvolutional network. In: Ricci, E., Rota, B.S., Snoek, C., Lanz, O., Messelodi, S., Sebe, N. (eds.) Image Analysis and Processing. ICIAP 2019. Lecture Notes in Computer Science, vol. 11751. Springer, Cham (2019)

    Google Scholar 

  89. Zhang, H., Zhang, L., Shen, H.: A super-resolution reconstruction algorithm for hyper spectral images. Signal Process. 92(9), 2082–2096 (2012)

    Google Scholar 

  90. Bi, Z., Li, J., Liu, Z.-S.: Super resolution SAR imaging via parametric spectral estimation methods. IEEE Trans. Aerosp. Electron. Syst. 35(1), 267–281 (1999). https://doi.org/10.1109/7.745697

    Article  Google Scholar 

  91. Rohith, G., Vasuki, A.: A novel approach to super resolution image reconstruction algorithm from low resolution panchromatic images. In: 3rd International Conference on Signal Processing, Communication and Networking (ICSCN), pp. 1–8 (2015). https://doi.org/10.1109/icscn.2015.7219842

  92. Qifang, X., Guoqing, Y., Pin, L.: Super-resolution reconstruction of satellite video images based on interpolation method. Proc. Comput. Sci. 107, 454–459 (2017)

    Google Scholar 

  93. Huang, D., Liu, H.: A short survey of image super resolution algorithms. J. Comput. Sci. Technol. 2, 19–29 (2015)

    Google Scholar 

  94. Gerchberg, R.W.: Super-resolution through error energy reduction. J. Mod. Opt. 21(9), 709–720 (1974)

    Google Scholar 

  95. Santis, P.D., Gori, F.: On an iterative method for super-resolution. J. Mod. Opt. 22(8), 691–695 (1975)

    Google Scholar 

  96. Pineda, F., Ayma, V., Aduviri, R., Beltran, C.: Super resolution approach using generative adversarial network models for improving satellite image resolution. In: Lossio-Ventura, J., Condori-Fernandez, N., Valverde-Rebaza, J. (eds.) Information Management and Big Data. SIMBig 2019. Communications in Computer and Information Science, vol. 1070. Springer, Cham (2020)

    Google Scholar 

  97. Luo, Y., Zhou, L., Wang, S., Wang, Z.: Video satellite imagery super resolution via convolutional neural networks. IEEE Geosci. Remote Sens. Lett. 14(12), 2398–2402 (2017). https://doi.org/10.1109/LGRS.2017.2766204

    Article  Google Scholar 

  98. Tatem, A.J., Lewis, H.G., Atkinson, P.M., Nixon, M.S.: Super-resolution target identification from remotely sensed images using a Hopfield neural network. IEEE Trans. Geosci. Remote Sens. 39(4), 781–796 (2001). https://doi.org/10.1109/36.917895

    Article  Google Scholar 

  99. Liebel, L., Körner, M.: Single-image super resolution for multispectral remote sensing data using convolutional neural networks, Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 41, 883–890 (2016)

    Google Scholar 

  100. Tuna, C., Unal, G., Sertel, E.: Single-frame super resolution of remote-sensing images by convolutional neural networks. Int. J. Remote Sens. 39, 2463–2479 (2018)

    Google Scholar 

  101. Huang, N., Yang, Y., Liu, J., Gu, X., Cai, H.: Single-image super-resolution for remote sensing data using deep residual-learning neural network. In: Proceedings of the Springer International Conference on Neural Information Processing; Guangzhou, China, 14–18 November 2017, pp. 622–630 (2017)

  102. Lei, S., Shi, Z., Zou, Z.: Super-resolution for remote sensing images via local-global combined network. IEEE Geosci. Remote Sens. Lett. 14(8), 1243–1247 (2017). https://doi.org/10.1109/LGRS.2017.2704122

    Article  Google Scholar 

  103. Xu, W., Guangluan, X., Wang, Y., Sun, X., Lin, D., Yirong, W.: High quality remote sensing image super-resolution using deep memory connected network. In: Proceedings of the IGARSS 2018—2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain, 22–27 July 2018, pp. 8889–8892 (2018)

  104. Wang, T., Sun, W., Qi, H., Ren, P.: Aerial image super resolution via wavelet multiscale convolutional neural networks. IEEE Geosci. Remote Sens. Lett. 15(5), 769–773 (2019). https://doi.org/10.1109/LGRS.2018.2810893

    Article  Google Scholar 

  105. Ma, W., Pan, Z., Guo, J., Lei, B.: Achieving super-resolution remote sensing images via the wavelet transform combined with the recursive res-net. IEEE Trans. Geosci. Remote Sens. 57(6), 3512–3527 (2019). https://doi.org/10.1109/TGRS.2018.2885506

    Article  Google Scholar 

  106. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 21–26 July 2017, pp. 3147–3155 (2017)

  107. Lu, T., Wang, J., Zhang, Y., Wang, Z., Jiang, J.: Satellite image super-resolution via multi-scale residual deep neural network. Remote Sens. 11, 1588 (2019)

    Google Scholar 

  108. Pan, Z., Ma, W., Guo, J., Lei, B.: Super-resolution of single remote sensing image based on residual dense backprojection networks. IEEE Trans. Geosci. Remote Sens. 57(10), 7918–7933 (2019). https://doi.org/10.1109/TGRS.2019.2917427

    Article  Google Scholar 

  109. Haut, J.M., Fernandez-Beltran, R., Paoletti, M.E., Plaza, J., Plaza, A., Pla, F.: A new deep generative network for unsupervised remote sensing single-image super-resolution. IEEE Trans. Geosci. Remote Sens. 56(11), 6792–6810 (2018). https://doi.org/10.1109/TGRS.2018.2843525

    Article  Google Scholar 

  110. Jiang, K., Wang, Z., Yi, P., Wang, G., Lu, T., Jiang, J.: Edge-enhanced GAN for remote sensing image superresolution. IEEE Trans. Geosci. Remote Sens. 57(8), 5799–5812 (2019). https://doi.org/10.1109/tgrs.2019.2902431

    Article  Google Scholar 

  111. Ma, W., Pan, Z., Guo, J., Lei, B.: Super-resolution of remote sensing images based on transferred generative adversarial network. In: Proceedings of the IGARSS 2018 2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain, 22–27 July 2018, pp. 1148–1151 (2018)

  112. Shuai, Y., Wang, Y., Peng, Y., Xia, Y.: Accurate image super-resolution using cascaded multi-column convolutional neural networks. In: Proceedings of the 2018 IEEE International Conference on Multimedia and Expo (ICME), San Diego, CA, USA, 23–27 July, pp. 1–6 (2018)

  113. Huang, W., Xiao, L., Wei, Z., Liu, H., Tang, S.: A new pan-sharpening method with deep neural networks. IEEE Geosci. Remote Sens. Lett. 12(5), 1037–1041 (2015). https://doi.org/10.1109/LGRS.2014.2376034

    Article  Google Scholar 

  114. Cai, W., Xu, Y., Wu, Z., Liu, H., Qian, L., Wei, Z.: Pan-sharpening based on multilevel coupled deep network. In: Proceedings of the IGARSS 2018—2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain, 22–27 July 2018, pp. 7046–7049 (2018)

  115. Wei, Y., Yuan, Q., Shen, H., Zhang, L.: Boosting the accuracy of multispectral image pansharpening by learning a deep residual network. IEEE Geosci. Remote Sens. Lett. 14(10), 1795–1799 (2017). https://doi.org/10.1109/LGRS.2017.2736020

    Article  Google Scholar 

  116. Azarang, A., Ghassemian, H.: A new pansharpening method using multi resolution analysis framework and deep neural networks. In: Proceedings of the 2017 3rd International Conference on IEEE Pattern Recognition and Image Analysis (IPRIA), Shahrekord, Iran, 19–20 April 2017, pp. 1–6 (2017)

  117. Azarang, A., Manoochehri, H.E., Kehtarnavaz, N.: Convolutional auto encoder-based multispectral image fusion. IEEE Access 7, 35673–35683 (2019)

    Google Scholar 

  118. Pouliot, D., Latifovic, R., Pasher, J., Duffe, J.: Landsat super-resolution enhancement using convolution neural networks and sentinel-2 for training. Remote Sens. 10, 394 (2018)

    Google Scholar 

  119. Meer, P.: From a robust hierarchy to a hierarchy of robustness. In: Davis, L.S. (ed.) Foundations of Image Understanding. The Springer International Series in Engineering and Computer Science, vol. 628. Springer, Boston (2001)

    Google Scholar 

  120. Rohith, G., Kumar, L.S.: Performance analysis of satellite image super resolution using deep learning techniques. In: IEEE Bombay Section Signature Conference (IBSSC), Mumbai, India, pp. 1–6 (2019). https://doi.org/10.1109/ibssc47189.2019.8973105

  121. Rhyma Purnamasayangsukasih, P., Norizah, K., Ismail, A.A., Shamsudin, I.: A review of uses of satellite imagery in monitoring mangrove forests. In: IOP Conference Series: Earth and Environmental Science (2016). https://doi.org/10.1088/1755-1315/37/1/012034

  122. Yue, L., Shen, H., Li, J., Yuan, Q., Hongyan, Z., Zhang, L.: Image super-resolution: the techniques, applications and future. Signal Process. 14, 389–408 (2016)

    Google Scholar 

  123. Ma, Y., Zhang, H., Xue, Y., Zhang, S.: Super-resolution image reconstruction based on K-means-Markov network. In: Fifth International Conference on Natural Computation, Tianjin, pp. 316–318 (2009)

  124. Peleg, T., Elad, M.: A statistical prediction model based on sparse representations for single image super-resolution. IEEE Trans. Image Process. 23(6), 2569–2582 (2014)

    MathSciNet  MATH  Google Scholar 

  125. Schultz, R.R., Stevenson, R.L.: A Bayesian approach to image expansion for improved definition. IEEE Trans. Image Process. 3(3), 233–242 (1994)

    Google Scholar 

  126. Xie, H., Zhang, F., Zhang, J., Xu, Z., Li, L.: Study of super resolution processing methods for thick pinhole image. J. Signal Inform. Process. Sci. Res. 4, 222–227 (2013)

    Google Scholar 

  127. Molina, R., Vegab, M., Mateos, J., Katsaggelos, A.K.: Parameter estimation in Bayesian reconstruction of multispectral images using super resolution techniques. In: Proceedings of IEEE International Conference on Image Processing, USA, pp. 1749–1752 (2006)

  128. Zhao, J., Chen, C., Zhou, Z., Cao, F.: Single image super-resolution based on adaptive convolutional sparse coding and convolutional neural networks. J. Vis. Commun. Image Rep. 58, 651–661 (2019)

    Google Scholar 

  129. Liu, N., Xing, X., Li, Y., Zhu, A.: Sparse representation based image super-resolution on the KNN based dictionaries. Opt. Laser Technol. 110, 135–144 (2019)

    Google Scholar 

  130. Li, H., Lam, K.M., Wang, M.: Image super-resolution via feature-augmented random forest. Signal Process. Image Commun. 72, 25–34 (2019)

    Google Scholar 

  131. Shermeyer, J., Van Etten, A.: The effects of super-resolution on object detection performance in satellite imagery. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA, pp. 1432–1441 (2019)

  132. Mirotznik, M., Mathews, S.: A Practical Enhanced-Resolution Integrated Optical-Digital Imaging Camera (PERIODIC) (2009). https://doi.org/10.1117/12.819484

  133. Dare, P.M.: Shadow analysis in high-resolution satellite imagery of urban areas. Photogramm. Eng. Remote Sens. 71(2), 169–177 (2005)

    Google Scholar 

  134. Noor, D.F., Li, L., Li, Z., Bhattacharyya, S.: Multi-frame super resolution with deep residual learning on flow registered non-integer pixel images. In: IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, pp. 2164–2168 (2019). https://doi.org/10.1109/icip.2019.8803156

  135. Haefner, B., Quéau, Y., Möllenhoff, T., Cremers, D.: Fight Ill-Posedness with Ill-Posedness: single-shot variational depth super-resolution from shading. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018). https://doi.org/10.1109/cvpr.2018.00025

  136. Zhang, J., Gai, D., Zhang, X., et al.: Multi-example feature-constrained back-projection method for image super-resolution. Comput. Vis. Media 3, 73–82 (2017)

    MATH  Google Scholar 

  137. Gil, M.L., Arza, M., Ortiz, J., DEM Ávila, A.: Shading method for the correction of pseudoscopic effect on multi-platform satellite imagery. GISci. Remote Sens. 51, 630–643 (2014). https://doi.org/10.1080/15481603.2014.988433

    Article  Google Scholar 

  138. Ran, Q., Xu, X., Zhao, S., et al.: Remote sensing images super-resolution with deep convolution networks. Multimed. Tools Appl. 79, 8985–9001 (2020). https://doi.org/10.1007/s11042-018-7091-1

    Article  Google Scholar 

  139. Augustsson, E., Timofte, R.: Challenge on single image super resolution: dataset and study, in proceedings of IEEE conference on computer vision and pattern recognition workshops, pp. 126–135 (2017)

  140. Walsh, D.O., Nielsen-Delaney, P.A.: a direct method for super-resolution. J. Opt. Soc. Am. A 11, 572–579 (1994)

    Google Scholar 

  141. Zeiler, M.D., Taylor, G.W., Fergus, R.: Adaptive de convolutional networks for mid and high level feature learning. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2018–2025. IEEE (2011)

  142. Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. Pattern. Anal. Mach. Intel. 24(9), 1167–1183 (2002)

    Google Scholar 

  143. Enríquez-Cervantes, C.J., Rodríguez-Dagnino, R.M.: A super-resolution image reconstruction using natural neighbor interpolation. Computación y Sistemas 19(2), 211–231 (2015)

    Google Scholar 

  144. Zareapoor, M., Jain, D.K., Yang, J.: Local spatial information for image super-resolution. Cognit. Syst. Res. 52, 49–57 (2018)

    Google Scholar 

  145. Salakhutdinov, R., Larochelle, H.: Efficient learning of deep Boltzmann machines. J. Mach. Learn. Res. Proc. Track. 9, 693–700 (2010)

    Google Scholar 

  146. Ciresan, D.C., Meier, U., Masci, J., Gambardella, L.M., Schmidhuber, J.: Flexible, high performance convolutional neural networks for image classification. In: International Joint Conference on Artificial Intelligence IJCAI-2011, pp. 1237–1242 (2011). https://doi.org/10.5591/978-1-57735-516-8/ijcai11-210

  147. Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: Proceedings of IEEE International Conference on Computer Vision, Japan (2009)

  148. Timofte, R., De Smet, V., Gool, L.V.: Anchored neighborhood regression for fast example-based super-resolution. In: IEEE International Conference on Computer Vision, Sydney, NSW, pp. 1920–1927 (2013)

  149. Zhang, J., et al.: Multi-Example Feature-Constrained Back-Projection Method for Image, Computational Visual Media, vol. 3, pp. 73–82. Springer, New York (2017)

    Google Scholar 

  150. Miravet, C., Rodrguez, F.B.: Accurate and robust image super- resolution by neural processing of local image representations. In: Proceedings of International Conference on Artificial Neural Networks, Poland, pp. 499–505 (2005)

Download references

Funding

No funding was received

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, National Institute of Technology, Puducherry, India

    G. Rohith & Lakshmi Sutha Kumar

Authors
  1. G. Rohith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lakshmi Sutha Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to G. Rohith.

Ethics declarations

Conflict of interest

G. Rohith declares that he has no conflict of interest. Lakshmi Sutha Kumar declares that she has no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rohith, G., Kumar, L.S. Paradigm shifts in super-resolution techniques for remote sensing applications. Vis Comput 37, 1965–2008 (2021). https://doi.org/10.1007/s00371-020-01957-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01957-8

Keywords

Search

Navigation