Skip to main content
SpringerLink
Log in

Image Desaturation for SDO/AIA Using Deep Learning

  • Published:
Solar Physics Aims and scope Submit manuscript

Abstract

The Atmospheric Imaging Assembly (AIA) on board the Solar Dynamics Observatory (SDO) (launched in February 2010) provides uninterrupted full-disk solar images over 10 wavebands. In the case of violent solar flares, saturation would happen to SDO/AIA images in their core regions, which leads to signal loss, hindering us to understand physical mechanism behind solar flares. This paper introduces a deep learning based image restoration model which can recover signal of saturation region by referring to other normal/valid region within an image. The proposed model, namely PCGAN, combines partial convolution (PC) and conditional generative adversarial network (GAN). The PC module was originally designed for image inpainting, for repairing images with scratches and holes. In addition, a new comprehensive loss function consists of an adversarial loss, a pixel reconstruction loss, a gradient loss, a perceptual loss, a style loss and a total variation loss. Moreover, for validating the proposed model, a new dataset consisting of paired saturated and normal SDO/AIA images is established. Experimental results demonstrate that the proposed PCGAN can get appealing desaturated solar images with respect to both objective and subjective validations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7

Similar content being viewed by others

Notes

  1. See ftp://ftp.swpc.noaa.gov/pub/warehouse/.

  2. See https://docs.sunpy.org/en/stable/guide/acquiring_data/fido.html#finding-and-downloading-data-using-fido.

References

  • Gatys, L.A., Ecker, A.S., Bethge, M.: 2016, Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2414.

    Google Scholar 

  • Guastavino, S., Piana, M., Massone, A.M., Schwartz, R., Benvenuto, F.: 2019, Desaturating SDO/AIA observations of solar flaring storms. Astrophys. J. 882(2), 109. DOI.

    Article  ADS  Google Scholar 

  • Ioffe, S., Szegedy, C.: 2015, Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, 448.

    Google Scholar 

  • Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: 2017, Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1125.

    Google Scholar 

  • Johnson, J., Alahi, A., Fei-Fei, L.: 2016, Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision, Springer, Berlin, 694.

    Google Scholar 

  • Kingma, D.P., Ba, J.: 2014, Adam: a method for stochastic optimization. arXiv preprint, arXiv.

  • Kucuk, A., Banda, J.M., Angryk, R.A.: 2017, A large-scale solar dynamics observatory image dataset for computer vision applications. Sci. Data 4, 170096. DOI. ADS.

    Article  Google Scholar 

  • Lemen, J.R., Title, A.M., Akin, D.J., Boerner, P.F., Chou, C., Drake, J.F., Duncan, D.W., Edwards, C.G., Friedlaender, F.M., Heyman, G.F., Hurlburt, N.E., Katz, N.L., Kushner, G.D., Levay, M., Lindgren, R.W., Mathur, D.P., McFeaters, E.L., Mitchell, S., Rehse, R.A., Schrijver, C.J., Springer, L.A., Stern, R.A., Tarbell, T.D., Wuelser, J.-P., Wolfson, C.J., Yanari, C., Bookbinder, J.A., Cheimets, P.N., Caldwell, D., Deluca, E.E., Gates, R., Golub, L., Park, S., Podgorski, W.A., Bush, R.I., Scherrer, P.H., Gummin, M.A., Smith, P., Auker, G., Jerram, P., Pool, P., Soufli, R., Windt, D.L., Beardsley, S., Clapp, M., Lang, J., Waltham, N.: 2012, The Atmospheric Imaging Assembly (AIA) on the Solar Dynamics Observatory (SDO). Solar Phys. 275(1-2), 17. DOI. ADS.

    Article  ADS  Google Scholar 

  • Liu, G., Reda, F.A., Shih, K.J., Wang, T.-C., Tao, A., Catanzaro, B.: 2018, Image inpainting for irregular holes using partial convolutions. In: Proceedings of the European Conference on Computer Vision (ECCV), 85.

    Google Scholar 

  • Ma, C., Rao, Y., Cheng, Y., Chen, C., Lu, J., Zhou, J.: 2020, Structure-preserving super resolution with gradient guidance. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

    Google Scholar 

  • Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., Paul Smolley, S.: 2017, Least squares generative adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, 2794.

    Google Scholar 

  • Miyato, T., Kataoka, T., Koyama, M., Yoshida, Y.: 2018, Spectral normalization for generative adversarial networks. arXiv preprint, arXiv.

  • Nazeri, K., Ng, E., Joseph, T., Qureshi, F., Ebrahimi, M.: 2019, Edgeconnect: structure guided image inpainting using edge prediction. In: The IEEE International Conference on Computer Vision (ICCV) Workshops.

    Google Scholar 

  • Odena, A., Buckman, J., Olsson, C., Brown, T.B., Olah, C., Raffel, C., Goodfellow, I.: 2018, Is generator conditioning causally related to gan performance? arXiv preprint, arXiv.

  • Pesnell, W.D., Thompson, B.J., Chamberlin, P.C.: 2012, The Solar Dynamics Observatory (SDO). Solar Phys. 275(1), 3. DOI.

    Article  ADS  Google Scholar 

  • Ronneberger, O., Fischer, P., Brox, T.: 2015, U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Berlin, 234.

    Google Scholar 

  • Schwartz, R.A., Torre, G., Piana, M.: 2014, Systematic de-saturation of images from the Atmospheric Imaging Assembly in the Solar Dynamics Observatory. Astrophys. J. 793(2), L23. DOI.

    Article  ADS  Google Scholar 

  • Schwartz, R.A., Torre, G., Massone, A.M., Piana, M.: 2015, Desat: a solar software tool for image de-saturation in the Atmospheric Image Assembly onboard the Solar Dynamics Observatory. Astron. Comput. 13, 117. DOI. http://www.sciencedirect.com/science/article/pii/S2213133715000992.

    Article  ADS  Google Scholar 

  • Simonyan, K., Zisserman, A.: 2015, Very deep convolutional networks for large-scale image recognition. CoRR, arXiv.

  • Torre, G., Schwartz, R.A., Benvenuto, F., Massone, A.M., Piana, M.: 2015, Inverse diffraction for the Atmospheric Imaging Assembly in the Solar Dynamics Observatory. Inverse Probl. 31(9), 095006. DOI.

    Article  ADS  MathSciNet  MATH  Google Scholar 

  • Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: 2004, Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600.

    Article  ADS  Google Scholar 

  • Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., Catanzaro, B.: 2018, High-resolution image synthesis and semantic manipulation with conditional gans. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 8798.

    Google Scholar 

  • Zhang, H., Goodfellow, I., Metaxas, D., Odena, A.: 2018, Self-attention generative adversarial networks. arXiv preprint, arXiv.

  • Zhao, D., Xu, L., Chen, L., Yan, Y., Duan, L.-Y.: 2019, Mask-Pix2Pix network for overexposure region recovery of solar image. Adv. Astron. 2019, 5343254. DOI. ADS.

    Article  ADS  Google Scholar 

  • Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: 2017, Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, 2223.

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by the National Natural Science Foundation of China under Grants 11790301 and 11790305, the Open Research Project of the State Key Laboratory of Media Convergence and Communication, Communication University of China, China (No. SKLMCC2020KF004). It was also sponsored by CAAI-Huawei MindSpore Open Fund.

Author information

Authors and Affiliations

  1. Key Laboratory of Solar Activity, National Astronomical Observatories, Chinese Academy of Sciences, Beijing, 100101, China

    Xuexin Yu, Long Xu & Yihua Yan

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Xuexin Yu

  3. Peng Cheng Laboratory, Shenzhen, 518000, China

    Long Xu

Authors
  1. Xuexin Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Long Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yihua Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Long Xu.

Ethics declarations

Disclosure of Potential Conflicts of Interest

The authors declare that they have no conflicts of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, X., Xu, L. & Yan, Y. Image Desaturation for SDO/AIA Using Deep Learning. Sol Phys 296, 56 (2021). https://doi.org/10.1007/s11207-021-01808-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11207-021-01808-2

Keywords

Search

Navigation