Abstract
Image fusion plays a vital role in providing better visualization of remotely sensed image data. Most earth observation satellites have sensors that provide both high spatial resolution panchromatic (PAN) images and low resolution multispectral (MS) images. In this paper, we propose a new fusion algorithm that optimally combines spectral information from MS image and spatial information from the PAN image of the same scene to create a single comprehensive fused image. As the performance of the fusion scheme relies on the choice of fusion rule, the proposed algorithm is based on a weighted averaging fusion rule that uses optimal weights obtained from brain storm optimization (BSO) algorithm for the fusion of high frequency and low frequency coefficients obtained by applying Curvelet transform to the source images. The objective function in BSO is formulated with twin objectives of maximizing the entropy and minimizing the root mean square error. The fusion results are compared with the existing fusion techniques, such as Brovey, principal component analysis, discrete wavelet transform, non sub-sampled contourlet transform, and intensity hue saturation. From the experimental results and analysis, the proposed fusion algorithm gives a better fusion performance in terms of subjective and objective measures than the traditional algorithms. As a benefit, the proposed fusion scheme preserves spectral information of the MS image with increased spatial resolution and edge information.
Similar content being viewed by others
References
Alparone, L., et al. (2007). Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data-fusion contest. IEEE Transactions on Geoscience and Remote Sensing, 45(10), 3012–3021.
Ballester, C., Caselles, V., Igual, L., Verdera, J., & Rougé, B. (2006). A variat-ional model for P + XS image fusion. International Journal of Computer Vision, 69(1), 43–58.
Bavirisetti, D. P., & Dhuli, R. (2016). Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen–Loeve transform. IEEE Sensors Journal, 16(1), 203–209.
Chavez, P. S., Jr., Sides, S. C., & Anderson, J. A. (1991). Comparison of three different methods to merge multiresolution and multispectral data—Landsat TM and SPOT panchromatic. Photogrammetric Engineering and Remote Sensing, 57(3), 295–303.
Chen, C., Li, Y., Liu, W., & Huang, J. (2015). SIRF: Simultaneous satellite image registration and fusion in a unified framework. IEEE Transactions on Image Processing, 24(11), 4213–4224.
Chen, S., Zhang, R., Su, H., Tian, J., & Xia, J. (2010). SAR and multispectral image fusion using generalized IHS transform based on à trous wavelet and EMD decompositions. IEEE Sensors Journal, 10(3), 737–745.
Choi, M. (2006). A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter. IEEE Transactions on Geoscience and Remote Sensing, 44(6), 1672–1682.
Choi, M., Kim, R. Y., Nam, M.-R., & Kim, H. O. (2005). Fusion of multispectral and panchromatic satellite images using the curvelet transform. IEEE Geoscience and Remote Sensing Letters, 2(2), 136–140.
El-Mezouar, M. C., Taleb, N., Kpalma, K., & Ronsin, J. (2011). An IHS-based fusion for color distortion reduction and vegetation enhancement in IKONOS Imagery. IEEE Transactions on Geoscience and Remote Sensing, 49(5), 1590–1602.
Fonseca, L., Namikawa, L., Castejon, E., Carvalho, L., Pinho, C., & Pagamisse, A. (1992). Image fusion for remote sensing applications (pp. 153–178).
Garzelli, Andrea, Nencini, F., & Capobianco, L. (2008). Optimal MMSE pan sharpening of very high resolution multispectral images. IEEE Transactions on Geoscience and Remote Sensing, 46(1), 228–236.
Gross, H., & Schott, J. (1998). Application of spectral mixture analysis and image fusion techniques for image sharpening. Remote Sensing of Environment, 63(2), 85–94.
Huang, W., Xiao, L., Wei, Z., Liu, H., & Tang, S. (2015). A new Pan-sharpening method with deep neural networks. IEEE Geoscience and Remote Sensing Letters, 12(5), 1037–1041.
Joshi, M. V., Bruzzone, L., & Chaudhuri, S. (2006). A model-based approach to multiresolution fusion in remotely sensed images. IEEE Transactions on Geoscience and Remote Sensing, 44(9), 2459–2562.
Joshi, M., & Jalobeanu, A. (2010). MAP estimation for multiresolution fusionin remotely sensed images using an IGMRF prior model. IEEE Transactions on Geoscience and Remote Sensing, 48(3), 1245–1255.
Kim, Y., Lee, C., Han, D., Kim, Y., & Kim, Y. (2011). Improved additive-wavelet image fusion. IEEE Geoscience and Remote Sensing Letters, 8(2), 263–267.
Li, X., Pang, Y., & Yuan, Y. (2010). L1-norm-based 2D PCA. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 40(4), 1170–1175.
Li, S., & Yang, B. (2010). Hybrid multiresolution method for multisensor multimodal image fusion. IEEE Sensors Journal, 10(9), 1519–1526.
Liu, Z., Blasch, E., Xue, Z., Zhao, J., Laganiere, R., & Wu, W. (2012). Objective assessment of multi-resolution image fusion algorithms for context enhancement in night vision: A comparative study. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(1), 94–109.
Liu, Q., Wang, Y., Zhang, Z., & Liu, L. (2014). Pan-sharpening based on weighted red black wavelets. IET Image Processing, 8(8), 477–488.
Mahyari, A. G., & Yazdi, M. (2010). Fusion of panchromatic and multispectral images using temporal Fourier transform. IET Image Process, 4(4), 255–260.
Mahyari, A. G., & Yazdi, M. (2011). Panchromatic and multispectral image fusion based on maximization of both spectral and spatial similarities. IEEE Transactions on Geoscience and Remote Sensing, 49(6), 1976–1985.
Massip, P., Blanc, P., & Wald, L. (2012). A method to better account for modulation transfer functions in ARSIS-based pansharpening methods. IEEE Transactions on Geoscience and Remote Sensing, 50(3), 800–808.
Mitianoudis, N., & Stathaki, T. (2008). Optimal contrast correction for ICA-based fusion of multimodal images. IEEE Sensors Journal, 8(12), 2016–2026.
Palsson, F., Sveinsson, J. R., Ulfarsson, M. O., & Benediktsson, J. A. (2015). Model-based fusion of multi- and hyperspectral images using PCA and wavelets. IEEE Transactions on Geoscience and Remote Sensing, 53(5), 2652–2663.
Petrovic, V. (2007). Subjective tests for image fusion evaluation and objective metric validation. Information Fusion, 8(2), 2018–2216.
Peyre, G. (2005). Geometric multi-echelles pour les images at les textures. Report Ecolepolytechnique, centre de MathematiquesAppliquees, Palaiseau, France.
Pohl, C., & Van Genderen, J. L. (1998). Multisensor image fusion in remote sensing: Concepts, method and applications. International Journal of Remote Sensing, 19(5), 823–854.
Shahdoosti, H. R., & Ghassemian, H. (2015). Fusion of MS and PAN images preserving spectral quality. IEEE Geoscience and Remote Sensing Letters., 12(3), 611–615.
Shi, Y. (2011). Brain storm optimization algorithm. In ICSI 2011: Advances in swarm intelligence (pp. 303–309).
Song, Y., Wu, W., Liu, Z., Yang, X., Liu, K., & Lu, W. (2016). An adaptive pansharpening method by using weighted least squares filter. IEEE Geoscience and Remote Sensing Letters, 13(1), 18–22.
Thomas, C., Ranchin, T., Wald, L., & Chanussot, J. (2008). ‘Synthesis of multispectral images to high spatial resolution: A critical review of fusion methods based on remote sensing physics. IEEE Transactions on Geoscience and Remote Sensing, 46(5), 1301–1312.
Upla, K. P., Gajjar, P. P., & Joshi, M. V. (2015a). An edge preserving multi-resolution fusion: Use of contourlet transform and MRF prior. IEEE Transaction on Geoscience and Remote Sensing, 53(6), 3210–3220.
Upla, K. P., Joshi, M. V., & Gajjar, P. P. (2015b). An edge preserving multiresolution fusion: Use of contourlet transform and MRF prior. IEEE Transactions on Geoscience and Remote Sensing, 53(6), 3210–3220.
Vaillancourt, J., Blasch, E., Gu, G., Lu, X., & Reinhardt, K. (2016). A demonstration of on-chip self-registered image fusion using a voltage-tunable mid-wave and long-wave infrared dual-band focal plane array. IEEE Sensors Journal, 16(11), 4374–4379.
Wald, L. (2000). Quality of high resolution synthesized images: Is there a simple criterion. In Proceedings of 3rd conference on fusion earth data, merging point measurements, raster maps remotely sensed images (pp. 99–103).
Wang, W., Jiao, L., & Yang, S. (2015). Novel adaptive component-substitution-based pan-sharpening using particle swarm optimization. IEEE Geoscience and Remote Sensing Letters, 12(4), 781–785.
Wei, Z. G., Yuan, J. H., & Cai, Y. L. (1999). A picture quality evaluation method based on human perception. Acta Electronica Sinica, 27(4), 79–82.
Yang, Y., Tong, S., Huang, S., & Lin, P. (2015). Multifocus image fusion based on NSCT and focused area detection. IEEE Sensors Journal, 15(5), 2824–2838.
Yuhas, R. H., Goetz, A.F., & Boardman, J. W. (1992). Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm. In Proceedings of summaries 3rd annual JPL airborne geoscience workshop (Vol. 1, pp. 147–149).
Zhang, Q., Cao, Z., Hu, Z., Jia, Y., & Wu, X. (2015). Joint image registration and fusion for panchromatic and multispectral images. IEEE Geoscience and Remote Sensing Letters, 12(3), 467–471.
Zhang, L., Shen, H., Gong, W., & Zhang, H. (2012). Adjustable model-based fusion method for multispectral and panchromatic images. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 42(6), 1693–1704.
Zhong, S., Zhang, Y., Chen Y., & Wu, D. (2017). Combining component substitution and multi resolution analysis: A novel generalized BDSD Pan sharpening algorithm. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10, 2867–2875.
Zhou, Z., Peng, S., Wang, B., Hao, Z., & Chen, S. (2012). An optimized approach for pansharpening very high resolution multispectral images. IEEE Geoscience and Remote Sensing Letters, 9(4), 735–739.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kanmani, M., Narasimhan, V. An optimal weighted averaging fusion strategy for remotely sensed images. Multidim Syst Sign Process 30, 1911–1935 (2019). https://doi.org/10.1007/s11045-019-00636-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11045-019-00636-9