Skip to main content
SpringerLink
Log in

Fusion of Multispectral and Panchromatic Images by Integrating Standard PCA with Rotated Wavelet Transform

  • Research Article
  • Published:
Journal of the Indian Society of Remote Sensing Aims and scope Submit manuscript

Abstract

Many pansharpening algorithms are based on the principle of extracting spatial details from panchromatic (PAN) images and injecting them into multispectral (MS) images. In this paper, we present two fusion approach based on same principle by integrating standard principle component analysis (PCA) with decimated and undecimated rotated wavelet transform. When decimated/subsampled rotated wavelet transform (SSRWT) is used for fusion of MS and PAN images, three visual artifacts get introduced in the fused image namely color distortion, shifting effect and shift distortion. To eliminate color distortion, SSRWT is integrated with standard PCA, i.e., PCA–SSRWT. Color distortion is significantly mitigated, but shifting effect and shift distortion persist in the fused image of PCA–SSRWT. After employing undecimated/nonsubsampled rotated wavelet transform (NSRWT), shifting effect and shift distortion get eliminated with minimum color distortion. However, fused image as a result of NSRWT is spectrally high but spatially low. In order to improve spatial quality and remove visual artifacts observed in SSRWT and PCA–SSRWT, NSRWT is integrated with standard PCA, i.e., PCA–NSRWT. Visual and quantitative analysis is carried out to validate the quality of fused image for all the algorithms. Visual interpretation suggests that fused image obtained using PCA–NSRWT is superior to fused images of SSRWT, PCA and NSRWT. The overall quantitative analysis manifests that the PCA–NSRWT is consistent with visual interpretation and performs better than state-of-the-art methods. PCA–NSRWT not only removes visual artifacts but also improves spectral and spatial quality of the fused image compared to individual PCA, SSRWT, NSRWT and PCA–SSRWT. Based on visual and quantitative analysis, it is observed that PCA works better with undecimated compared to decimated rotated wavelet transform for fusion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  • Aiazzi, B., Alparone, L., Baronti, S., & Garzelli, A. (2002). Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Transactions on Geoscience and Remote Sensing, 40(10), 2300–2312

    Article  Google Scholar 

  • Aiazzi, B., Alparone, L., Baronti, S., Garzelli, A., & Selva, M. (2012). Twenty-five years of pansharpening: A critical review and new developments. In Signal and image processing for remote sensing (pp. 552–599) CRC Press.

  • Amolins, K., Zhang, Y., & Dare, P. (2007). Wavelet based image fusion techniques—An introduction, review and comparison. ISPRS Journal of Photogrammetry and Remote Sensing, 62(4), 249–263

    Article  Google Scholar 

  • Bamberger, R. H., & Smith, M. J. (1992). A filter bank for the directional decomposition of images: Theory and design. IEEE Transactions on Signal Processing, 40(4), 882–893

    Article  Google Scholar 

  • Burt, P., & Adelson, E. (1983). The Laplacian pyramid as a compact image code. IEEE Transactions on Communications, 31(4), 532–540

    Article  Google Scholar 

  • Carper, W. J., Lillesand, T. M., & Kiefer, R. W. (1990). The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data. Photogrammetric Engineering and Remote Sensing, 56(4), 459–467

    Google Scholar 

  • Chavan, S. S., Pawar, A., & Talbar, S. N. (2018). Multimodality medical image fusion using non-subsampled rotated wavelet transform for cancer treatment. International Journal of Computational Systems Engineering, 4(2–3), 96–105

    Article  Google Scholar 

  • Chibani, Y., & Houacine, A. (2002). The joint use of IHS transform and redundant wavelet decomposition for fusing multispectral and panchromatic images. International Journal of Remote Sensing, 23(18), 3821–3833

    Article  Google Scholar 

  • Da Cunha, A. L., Zhou, J., & Do, M. N. (2006). The nonsubsampled contourlet transform: Theory, design, and applications. IEEE Transactions on Image Processing, 15(10), 3089–3101

    Article  Google Scholar 

  • Do, M. N., & Vetterli, M. (2005). The contourlet transform: An efficient directional multiresolution image representation. IEEE Transactions on Image Processing, 14(12), 2091–2106

    Article  Google Scholar 

  • Easley, G., Labate, D., & Lim, W. Q. (2008). Sparse directional image representations using the discrete shearlet transform. Applied and Computational Harmonic Analysis, 25(1), 25–46

    Article  Google Scholar 

  • Ehlers, M., Klonus, S., Johan Åstrand, P., & Rosso, P. (2010). Multi-sensor image fusion for pansharpening in remote sensing. International Journal of Image and Data Fusion, 1(1), 25–45

    Article  Google Scholar 

  • Faragallah, O. S. (2018). Enhancing multispectral imagery spatial resolution using optimized adaptive PCA and high-pass modulation. International Journal of Remote Sensing, 39(20), 6572–6586.

    Article  Google Scholar 

  • Garzelli, A., Nencini, F., & Capobianco, L. (2008). Optimal MMSE pan sharpening of very high resolution multispectral images. IEEE Transactions on Geoscience and Remote Sensing, 46(1), 228–236

    Article  Google Scholar 

  • González-Audícana, M., Saleta, J. L., Catalán, R. G., & García, R. (2004). Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition. IEEE Transactions on Geoscience and Remote Sensing, 42(6), 1291–1299

    Article  Google Scholar 

  • Jagalingam, P., & Hegde, A. V. (2015). A review of quality metrics for fused image. Aquatic Procedia, 4, 133–142

    Article  Google Scholar 

  • Johnson, B. A., Tateishi, R., & Hoan, N. T. (2013). A hybrid pansharpening approach and multiscale object-based image analysis for mapping diseased pine and oak trees. International Journal of Remote Sensing, 34(20), 6969–6982

    Article  Google Scholar 

  • Kim, S. D., & Udpa, S. (2000). Texture classification using rotated wavelet filters. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 30(6), 847–852

    Article  Google Scholar 

  • Kingsbury, N. (1999). Image processing with complex wavelets. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 357(1760), 2543–2560

    Article  Google Scholar 

  • Labate, D., Lim, W. Q., Kutyniok, G., & Weiss, G. (2005). Sparse multidimensional representation using shearlets. In Wavelets XI, International Society for Optics and Photonics (Vol. 5914, p. 59140U).

  • Laben, C. A., & Brower, B. V. (2000). Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening. US Patent 6,011,875.

  • Ling, Y., Ehlers, M., Usery, E., & Madden, M. (2008). Effects of spatial resolution ratio in image fusion. International Journal of Remote Sensing, 29(7), 2157–2167

    Article  Google Scholar 

  • Li, H., Manjunath, B., & Mitra, S. K. (1995). Multisensor image fusion using the wavelet transform. Graphical Models and Image Processing, 57(3), 235–245

    Article  Google Scholar 

  • Liu, X., Wang, Y., & Liu, Q. (2018). PSGAN: A generative adversarial network for remote sensing image PAN-sharpening. In 2018 25th IEEE international conference on image processing (ICIP) (pp. 873–877). IEEE.

  • Ma, J., Yu, W., Chen, C., Liang, P., Guo, X., & Jiang, J. (2020). Pan-GAN: An unsupervised learning method for pan-sharpening in remote sensing image fusion using a generative adversarial network. Information Fusion, 62, 110-120.

    Article  Google Scholar 

  • Ma, J., Yu, W., Liang, P., Li, C., & Jiang, J. (2019). FusionGAN: A generative adversarial network for infrared and visible image fusion. Information Fusion, 48, 11–26

    Article  Google Scholar 

  • Mallat, S. G. (1989). A theory for multiresolution signal decomposition: The wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(7), 674–693

    Article  Google Scholar 

  • Mangolini, M. (1994). Contribution of pixel-based multi-sensor satellite image fusion in remote sensing and photo-interpretation. PhD Thesis, University of Nice Sophia-Antipolis

  • Masi, G., Cozzolino, D., Verdoliva, L., & Scarpa, G. (2016). Pansharpening by convolutional neural networks. Remote Sensing, 8(7), 594

    Article  Google Scholar 

  • Nason, G. P. & Silverman, B. W. (1995). The stationary wavelet transform and some statistical applications. In Wavelets and statistics (pp. 281–299). Springer.

  • Nunez, J., Otazu, X., Fors, O., Prades, A., Pala, V., & Arbiol, R. (1999). Multiresolution-based image fusion with additive wavelet decomposition. IEEE Transactions on Geoscience and Remote Sensing, 37(3), 1204–1211

    Article  Google Scholar 

  • Palsson, F., Sveinsson, J. R., Ulfarsson, M. O., & Benediktsson, J. A. (2016). Quantitative quality evaluation of pansharpened imagery: Consistency versus synthesis. IEEE Transactions on Geoscience and Remote Sensing, 54(3), 1247–1259

    Article  Google Scholar 

  • Pohl, C., & Van Genderen, J. L. (1998). Review article multisensor image fusion in remote sensing: Concepts, methods and applications. International Journal of Remote Sensing, 19(5), 823–854

    Article  Google Scholar 

  • Pohl, C., & Van Genderen, J. L. (2016). Remote sensing image fusion: A practical guide. CRC Press

  • Pradhan, P. S., King, R. L., Younan, N. H., & Holcomb, D. W. (2006). Estimation of the number of decomposition levels for a wavelet-based multiresolution multisensor image fusion. IEEE Transactions on Geoscience and Remote Sensing, 44(12), 3674–3686

    Article  Google Scholar 

  • Shah, V. P., Younan, N. H., & King, R. L. (2008). An efficient pan-sharpening method via a combined adaptive PCA approach and contourlets. IEEE Transactions on Geoscience and Remote Sensing, 46(5), 1323–1335

    Article  Google Scholar 

  • Vicinanza, M. R., Restaino, R., Vivone, G., DallaMura, M., & Chanussot, J. (2015). A pansharpening method based on the sparse representation of injected details. IEEE Geoscience and Remote Sensing Letters, 12(1), 180–184

    Article  Google Scholar 

  • Vivone, G., Alparone, L., Chanussot, J., Dalla Mura, M., Garzelli, A., Licciardi, G. A., Restaino, R., & Wald, L. (2015). A critical comparison among pansharpening algorithms. IEEE Transactions on Geoscience and Remote Sensing, 53(5), 2565–2586

    Article  Google Scholar 

  • Wald, L. (1999). Some terms of reference in data fusion. IEEE Transactions on Geoscience and Remote Sensing, 37(3), 1190–1193

    Article  Google Scholar 

  • Xu, H., Ma, J., Shao, Z., Zhang, H., Jiang, J., & Guo, X. (2021). SDPNet: A deep network for pan-sharpening with enhanced information representation. IEEE Transactions on Geoscience and Remote Sensing, 59(5), 4120–4134.

    Article  Google Scholar 

  • Xydeas, C., & Petrovic, V. (2000). Objective image fusion performance measure. Electronics Letters, 36(4), 308–309

    Article  Google Scholar 

  • Yang, J., Fu, X., Hu, Y., Huang, Y., Ding, X., & Paisley, J. (2017). PanNet: A deep network architecture for pan-sharpening. In Proceedings of the IEEE International Conference on Computer Vision (pp. 5449–5457).

  • Zhang, H. K., & Huang, B. (2015). A new look at image fusion methods from a Bayesian perspective. Remote Sensing, 7(6), 6828–6861

    Article  Google Scholar 

  • Zhang, K., Zhang, F., & Yang, S. (2019). Fusion of multispectral and panchromatic images via spatial weighted neighbor embedding. Remote Sensing, 11(5), 557

    Article  Google Scholar 

  • Zhang, Y., & Hong, G. (2005). An IHS and wavelet integrated approach to improve pan-sharpening visual quality of natural colour IKONOS and QuickBird images. Information Fusion, 6(3), 225–234

    Article  Google Scholar 

  • Zhong, S., Zhang, Y., Chen, Y., & Wu, D. (2017). Combining component substitution and multiresolution analysis: A novel generalized BDSD pansharpening algorithm. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(6), 2867–2875

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, SGGS Institute of Engineering and Technology, Nanded, Maharashtra, 431606, India

    Rishikesh G. Tambe

  2. Department of Electronics and Telecommunication Engineering, SGGS Institute of Engineering and Technology, Nanded, Maharashtra, 431606, India

    Sanjay N. Talbar

  3. Department of Electronics and Telecommunication Engineering, Don Bosco Institute of Technology, Kurla (W), Mumbai, Maharashtra, 400070, India

    Satishkumar S. Chavan

Authors
  1. Rishikesh G. Tambe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sanjay N. Talbar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Satishkumar S. Chavan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rishikesh G. Tambe.

Ethics declarations

Conflict of interest

The author declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tambe, R.G., Talbar, S.N. & Chavan, S.S. Fusion of Multispectral and Panchromatic Images by Integrating Standard PCA with Rotated Wavelet Transform. J Indian Soc Remote Sens 49, 2033–2055 (2021). https://doi.org/10.1007/s12524-021-01373-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12524-021-01373-y

Keywords

Search

Navigation