Skip to main content
SpringerLink
Log in

Satellite Imagery Super-Resolution Using Squeeze-and-Excitation-Based GAN

  • Original Paper
  • Published:
International Journal of Aeronautical and Space Sciences Aims and scope Submit manuscript

Abstract

Single Image Super Resolution (SISR) elevates spectral and spatial image resolution beyond the sensor capabilities. Convolutional Neural Networks (CNNs) have dominated current mainstream approaches. However, the utilization of pixel-based loss function hinders achieving realistic perceptual results at large upscale factors. Recently, Generative Adversarial Network (GAN) attained more realistic, crisp results in natural image, but the complex nature of satellite images limits the performance. In this work, we address these challenges equipped with the promising results of Squeeze-and-Excitation (SE) in classification tasks. A Spatial and Channel Squeeze-and-Excitation GAN (SCSE-GAN) is introduced. The proposed generator stacked SCSE block after each residual block to recalibrate and ensure features flow and amplify high-frequency details. In addition, skip/residual connection was utilized in the GAN generator network to further boost the performance. Wasserstein distance with gradient penalty (WGAN-GP) was adopted to stabilize training and avoid gradient vanishing phenomena. Finally, we conducted various experiments on two open-source benchmarks namely: RSSCN7 and Kaggle datasets, to systematically evaluate the proposed framework performance. According to the obtained results, the proposed approach excels other approaches quantitatively and visually. Quantitatively, the results show a boost by a considerable margin of 2%, and 3% in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM), respectively. Visually, the proposed method shows a sharper, less smooth image compared with benchmark SISR approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://www.kaggle.com/c/draper-satellite-image-chronology/data.

References

  1. Anwar S, Khan S, Barnes N (2019) A deep journey into super-resolution: A survey. arXiv preprint. arXiv:1904.07523

  2. Wang L, Chen W, Yang W, Bi F, Yu FR (2020) A state-of-the-art review on image synthesis with generative adversarial networks. IEEE Access 8:63514–63537

    Article  Google Scholar 

  3. Yang W, Zhang X, Tian Y, Wang W, Xue J-H, Liao Q (2019) Deep learning for single image super-resolution: a brief review. IEEE Trans Multimed 21(12):3106–3121

    Article  Google Scholar 

  4. Moustafa M, Ebeid HM, Helmy A, Nazmy TM, Tolba MF (2016) Rapid real-time generation of super-resolution hyperspectral images through compressive sensing and GPU. Int J Remote Sens 37(18):4201–4224

    Article  Google Scholar 

  5. Moustafa MS, Ebied HM, Helmy AK, Nazamy TM, Tolba MF (2017) Acceleration of super-resolution for multispectral images using self-example learning and sparse representation. Comput Electr Eng 62:249–265

    Article  Google Scholar 

  6. Moustafa M, Ebeid HM, Helmy A, Nazamy TM, Tolba MF (2015) Super-resolution: sparse dictionary design method using quantitative comparison. In: 2015 IEEE seventh international conference on intelligent computing and information systems (ICICIS), IEEE, pp. 383–389

  7. Alom MZ et al (2019) A state-of-the-art survey on deep learning theory and architectures. Electronics 8(3):292

    Article  Google Scholar 

  8. Jiao L, Zhao J (2019) A survey on the new generation of deep learning in image processing. IEEE Access 7:172231–172263

    Article  Google Scholar 

  9. Zou Q, Ni L, Zhang T, Wang Q (2015) Deep learning based feature selection for remote sensing scene classification. IEEE Geosci Remote Sens Lett 12(11):2321–2325

    Article  Google Scholar 

  10. Moustafa MS, Ahmed S, Hamed AA (2020) Learning to hash with convolutional network for multi-label remote sensing image retrieval

  11. Elkholy MM, Mostafa M, Ebied HM, Tolba MF (2020) Hyperspectral unmixing using deep convolutional autoencoder. Int J Remote Sens 41(12):4799–4819

    Article  Google Scholar 

  12. Zou F et al (2020) Arbitrary-oriented object detection via dense feature fusion and attention model for remote sensing super-resolution image. Neural Comput Appl 32:1–14

    Google Scholar 

  13. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  14. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint. arXiv:1409.1556

  15. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778

  16. Szegedy C et al. (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1–9

  17. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818–2826

  18. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European conference on computer vision, 2014: Springer, pp. 184–199

  19. Dong C, Loy CC, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  20. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1646–1654

  21. Lai W-S, Huang J-B, Ahuja N, Yang M-H (2018) Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Trans Pattern Anal Mach Intell 41(11):2599–2613

    Article  Google Scholar 

  22. Gu J, Sun X, Zhang Y, Fu K, Wang L (2019) Deep residual squeeze and excitation network for remote sensing image super-resolution. Remote Sens 11(15):1817

    Article  Google Scholar 

  23. Goodfellow IJ et al (2014) Generative adversarial networks. Adv Neural Inf Process Syst 3(6):1

    Google Scholar 

  24. Qian Z, Huang K, Wang Q, Xiao J, Zhang R (2019) Generative adversarial classifier for handwriting characters super-resolution. arXiv preprint. arXiv:1901.06199

  25. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint. arXiv:1511.06434

  26. Seddik MEA, Tamaazousti M, Lin J (2020) Generative collaborative networks for single image super-resolution. Neurocomputing 398:293–303

    Article  Google Scholar 

  27. Sharma A, Jindal N, Rana P (2020) Potential of generative adversarial net algorithms in image and video processing applications—a survey. Multimed Tools Appl 79(37):1–31

    Google Scholar 

  28. López-Tapia S, Lucas A, Molina R, Katsaggelos AK (2020) A single video super-resolution GAN for multiple downsampling operators based on pseudo-inverse image formation models. Digit Signal Process 104:102801. https://doi.org/10.1016/j.dsp.2020.102801

    Article  Google Scholar 

  29. Salgueiro Romero L, Marcello J, Vilaplana V (2020) Super-resolution of sentinel-2 imagery using generative adversarial networks. Remote Sens 12(15):2424

    Article  Google Scholar 

  30. Ledig C et al. (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681–4690

  31. Jiang K, Wang Z, Yi P, Wang G, Lu T, Jiang J (2019) Edge-enhanced GAN for remote sensing image superresolution. IEEE Trans Geosci Remote Sens 57(8):5799–5812

    Article  Google Scholar 

  32. Wang Z, Jiang K, Yi P, Han Z, He Z (2020) Ultra-dense GAN for satellite imagery super-resolution. Neurocomputing 398:328–337

    Article  Google Scholar 

  33. You C et al (2019) CT super-resolution GAN constrained by the identical, residual, and cycle learning ensemble (GAN-CIRCLE). IEEE Trans Med Imaging 39(1):188–203

    Article  Google Scholar 

  34. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: International conference on machine learning, PMLR, pp. 214–223

  35. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville A (2017) Improved training of wasserstein gans. arXiv preprint. arXiv:1704.00028

  36. Ran Q, Xu X, Zhao S, Li W, Du Q (2020) Remote sensing images super-resolution with deep convolution networks. Multimed Tools Appl 79(13):8985–9001

    Article  Google Scholar 

  37. Arun P, Buddhiraju KM, Porwal A, Chanussot J (2020) CNN based spectral super-resolution of remote sensing images. Signal Process 169:107394

    Article  Google Scholar 

  38. Zhang D, Shao J, Li X, Shen HT (2020) Remote sensing image super-resolution via mixed high-order attention network. IEEE Trans Geosci Remote Sens. https://doi.org/10.1109/TGRS.2020.3009918

    Article  Google Scholar 

  39. Mei S, Jiang R, Li X, Du Q (2020) Spatial and spectral joint super-resolution using convolutional neural network. IEEE Trans Geosci Remote Sens 58(7):4590–4603

    Article  Google Scholar 

  40. Shen H, Lin L, Li J, Yuan Q, Zhao L (2020) A residual convolutional neural network for polarimetric SAR image super-resolution. ISPRS J Photogramm Remote Sens 161:90–108

    Article  Google Scholar 

  41. Cheng X, Li X, Yang J, Tai Y (2018) SESR: single image super resolution with recursive squeeze and excitation networks. in 24th international conference on pattern recognition (ICPR), pp. 147–152

  42. Zhang S, Yuan Q, Li J, Sun J, Zhang X (2020) Scene-adaptive remote sensing image super-resolution using a multiscale attention network. IEEE Trans Geosci Remote Sens 58(7):4764–4779

    Article  Google Scholar 

  43. Dong X, Sun X, Jia X, Xi Z, Gao L, Zhang B (2020) Remote sensing image super-resolution using novel dense-sampling networks. IEEE Trans Geosci Remote Sens. https://doi.org/10.1109/TGRS.2020.2994253

    Article  Google Scholar 

  44. Salvetti F, Mazzia V, Khaliq A, Chiaberge M (2020) Multi-image super resolution of remotely sensed images using residual attention deep neural networks. Remote Sens 12(14):2207

    Article  Google Scholar 

  45. Wang X, Wu Y, Ming Y, Lv H (2020) Remote sensing imagery super resolution based on adaptive multi-scale feature fusion network. Sensors 20(4):1142

    Article  Google Scholar 

  46. Zhang Y, Zong R, Han J, Zhang D, Rashid T, Wang D (2020) TransRes: a deep transfer learning approach to migratable image super-resolution in remote urban sensing. In: 2020 17th Annual IEEE international conference on sensing, communication, and networking (SECON), IEEE, pp. 1–9

  47. Roy AG, Navab N, Wachinger C (2018) Recalibrating fully convolutional networks with spatial and channel “squeeze and excitation” blocks. IEEE Trans Med Imaging 38(2):540–549

    Article  Google Scholar 

  48. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7132–7141

  49. Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: ICML

  50. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint. arXiv:1412.6980

  51. Hore A, Ziou D (2010) Image quality metrics: PSNR vs. SSIM. In: 2010 20th international conference on pattern recognition, IEEE, pp. 2366–2369

Download references

Author information

Authors and Affiliations

  1. Data Reception, Analysis and Receiving Station Affairs, National Authority of Remote Sensing and Space Science, Cairo, Egypt

    Marwa S. Moustafa & Sayed A. Sayed

Authors
  1. Marwa S. Moustafa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sayed A. Sayed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sayed A. Sayed.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moustafa, M.S., Sayed, S.A. Satellite Imagery Super-Resolution Using Squeeze-and-Excitation-Based GAN. Int. J. Aeronaut. Space Sci. 22, 1481–1492 (2021). https://doi.org/10.1007/s42405-021-00396-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42405-021-00396-6

Keywords

Search

Navigation