Skip to main content
SpringerLink
Log in

Infrared and visible image fusion using modified spatial frequency-based clustered dictionary

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Infrared and visible image fusion is an active area of research as it provides fused image with better scene information and sharp features. An efficient fusion of images from multisensory sources is always a challenge for researchers. In this paper, an efficient image fusion method based on sparse representation with clustered dictionary is proposed for infrared and visible images. Firstly, the edge information of visible image is enhanced by using a guided filter. To extract more edge information from the source images, modified spatial frequency is used to generate a clustered dictionary from the source images. Then, non-subsampled contourlet transform (NSCT) is used to obtain low-frequency and high-frequency sub-bands of the source images. The low-frequency sub-bands are fused using sparse coding, and the high-frequency sub-bands are fused using max-absolute rule. The final fused image is obtained by using inverse NSCT. The subjective and objective evaluations show that the proposed method is able to outperform other conventional image fusion methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Availability of data and materials

Not applicable.

Code availability

Not applicable.

References

  1. Ma J, Ma Y, Li C (2019) Infrared and visible image fusion methods and applications: a survey. Inf Fusion 45:153–178

    Article  Google Scholar 

  2. Wang J, Peng J, Feng X, He G, Fan J (2014) Fusion method for infrared and visible images by using non-negative sparse representation. Infrared Phys Technol 67:477–489

    Article  Google Scholar 

  3. Kumar P, Mittal A, Kumar P (2006) Fusion of thermal infrared and visible spectrum video for robust surveillance. In: Proceedings of the Indian conference on computer vision, graphics and image processing, pp 528–539

  4. Singh R, Vatsa M, Noore A (2008) Integrated multilevel image fusion and match score fusion of visible and infrared face images for robust face recognition. Pattern Recogn 41(3):880–893

    Article  Google Scholar 

  5. Han J, Bhanu B (2007) Fusion of color and infrared video for moving human detection. Pattern Recogn 40(6):1771–1784

    Article  Google Scholar 

  6. Reinhard E, Adhikhmin M, Gooch B, Shirley P (2001) Color transfer between images. IEEE Comput Graph Appl 21(5):34–41

    Article  Google Scholar 

  7. Burt P, Adelson E (1983) The laplacian pyramid as a compact image code. IEEE Trans Commun 31(4):532–540

    Article  Google Scholar 

  8. Li S, Yang B, Hu J (2011) Performance comparison of different multi-resolution transforms for image fusion. Inf Fusion 12(2):74–84

    Article  Google Scholar 

  9. Li Q, Du J, Xu L (2015) Visible and infrared video fusion using uniform discrete curvelet transform and spatial-temporal information. Chin J Electron 24(4):761–766

    Article  Google Scholar 

  10. Lewis J, Callaghan RO, Nikolov S, Bull D, Canagarajah N (2007) Pixel- and region based image fusion with complex wavelets. Inf Fusion 8(2):119–130

    Article  Google Scholar 

  11. Zhang Q, Guo B (2009) Multifocus image fusion using the nonsubsampled contourlet transform. Signal Process 89(7):1334–1346

    Article  Google Scholar 

  12. Chen Y, Xiong J, Liu HL, Fan Q (2014) Fusion method of infrared and visible images based on neighborhood characteristic and regionalization in NSCT domain. Opt Int J Light Electron Opt 125(17):4980–4984

    Article  Google Scholar 

  13. Cai J, Cheng Q, Peng M, Song Y (2017) Fusion of infrared and visible images based on nonsubsampled contourlet transform and sparse K-SVD dictionary learning. Infrared Phys Technol 82:85–95

    Article  Google Scholar 

  14. Kong WW, Lei YJ, Lei Y, Lu S (2011) Image fusion technique based on non-subsampled contourlet transform and adaptive unit-fast-linking pulse-coupled neural network. IET Image Proc 5(2):113–121

    Article  Google Scholar 

  15. Li S, Kang X, Fang L, Hu J, Yin H (2017) Pixel-level image fusion: a survey of the state of the art. Inf Fusion 33:100–112

    Article  Google Scholar 

  16. Dogra A, Goyal B, Agrawal S (2017) From multi-scale decomposition to non-multi-scale decomposition methods: a comprehensive survey of image fusion techniques and its applications. IEEE Access 5:16040–16067

    Article  Google Scholar 

  17. Oliveira FPM, Tavares JMRS (2014) Medical image registration: a review. Comput Methods Biomech Biomed Eng 17(2):73–93

    Article  Google Scholar 

  18. Aharon M, Elad M, Bruckstein A (2006) K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54(11):4311–4322

    Article  Google Scholar 

  19. Yang B, Li S (2009) Multifocus image fusion and restoration with sparse representation. IEEE Trans Instrum Meas 59(4):884–892

    Article  Google Scholar 

  20. Yu N, Qiu T, Bi F, Wang A (2011) Image features extraction and fusion based on joint sparse representation. IEEE J Sel Top Signal Process 5(5):1074–1082

    Article  Google Scholar 

  21. Zhang Q, Liu Y, Blum RS, Han J, Tao D (2018) Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images: a review. Inf Fusion 40:57–75

    Article  Google Scholar 

  22. Yang B, Li S (2012) Pixel-level image fusion with simultaneous orthogonal matching pursuit. Inf Fusion 13(1):10–19

    Article  Google Scholar 

  23. Nejati M, Samavi S, Shirani S (2015) Multi-focus image fusion using dictionary-based sparse representation. Inf Fusion 25:72–84

    Article  Google Scholar 

  24. Liu Y, Wang Z (2015) Simultaneous image fusion and denoising with adaptive sparse representation. IET Image Proc 9(5):347–357

    Article  Google Scholar 

  25. Kim M, Han DK, Ko H (2016) Joint patch clustering-based dictionary learning for multimodal image fusion. Inf Fus 27:198–214

    Article  Google Scholar 

  26. Liu Z, Yin H, Fang B, Chai Y (2015) A novel fusion scheme for visible and infrared images based on compressive sensing. Opt Commun 335:168–177

    Article  Google Scholar 

  27. Xiang T, Yan L, Gao R (2015) A fusion algorithm for infrared and visible images based on adaptive dual-channel unit-linking PCNN in NSCT domain. Infrared Phys Technol 69:53–61

    Article  Google Scholar 

  28. Liu CH, Qi Y, Ding WR (2017) Infrared and visible image fusion method based on saliency detection in sparse domain. Infrared Phys Technol 83:94–102

    Article  Google Scholar 

  29. Do MN, Vetterli M (2005) The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans Image Process 14(12):2091–2106

    Article  Google Scholar 

  30. Da Cunha AL, Zhou J, Do MN (2006) The nonsubsampled contourlet transform: theory, design, and applications. IEEE Trans Image Process 15(10):3089–3101

    Article  Google Scholar 

  31. Mallat S, Zhang Z (1993) Matching pursuits with time-frequency dictionaries. IEEE Trans Signal Process 41(12):3397–3415

    Article  Google Scholar 

  32. Pati YC, Rezaiifar R, Krishnaprasad PS (1993) Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: Proceedings of 27th Asilomar conference on signals, systems, computing, vol 1, pp 40–44

  33. Yao Y, Guo P, Xin X, Jiang Z (2014) Image fusion by hierarchical joint sparse representation. Cogn Comput 6(3):281–292

    Article  Google Scholar 

  34. Mairal J, Bach F, Ponce J, Sapiro G. (2009) Online dictionary learning for sparse coding. In: Proceedings of the 26th annual international conference on machine learning, pp 689–696

  35. Aishwarya N, Thangammal CB (2018) Visible and infrared image fusion using DTCWT and adaptive combined clustered dictionary. Infrared Phys Technol 93:300–309

    Article  Google Scholar 

  36. Jinju J, Santhi N, Ramar K, Bama BS (2019) Spatial frequency discrete wavelet transform image fusion technique for remote sensing applications. Eng Sci Technol Int J 22(3):715–726

    Google Scholar 

  37. Li H, Manjunath BS, Mitra SK (1995) Multisensor image fusion using the wavelet transform. Graph Models Image Process 57(3):235–245

    Article  Google Scholar 

  38. Toet A (2014) TNO image fusion dataset. https://figshare.com/articles/TNO_Image_Fusion_Dataset/1008029. Accessed Jan 2018

  39. Liu Y, Liu S, Wang Z (2015) A general framework for image fusion based on multi-scale transform and sparse representation. Inf Fusion 24:147–164

    Article  Google Scholar 

  40. Qu G, Zhang D, Yan P (2002) Information measure for performance of image fusion. Electron Lett 38(7):313–315

    Article  Google Scholar 

  41. Piella G, Heijmans H (2003) A new quality metric for image fusion. In: Proceedings of 10th international conference on image processing, pp 173–176

  42. Yang C, Zhang JQ, Wang XR, Liu X (2008) A novel similarity based quality metric for image fusion. Inf Fusion 9(2):156–160

    Article  Google Scholar 

  43. Xydeas CA, Petrovic V (2000) Objective image fusion performance measure. Electron Lett 36(4):308–309

    Article  Google Scholar 

Download references

Funding

Not applicable.

Author information

Authors and Affiliations

  1. ECE, UIET, Panjab University, Chandigarh, 160014, India

    Sumit Budhiraja, Rajat Sharma & Sunil Agrawal

  2. Chandigarh University, Gharuan, Mohali, Punjab, 140413, India

    Balwinder S. Sohi

Authors
  1. Sumit Budhiraja
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rajat Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sunil Agrawal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Balwinder S. Sohi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sumit Budhiraja.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Budhiraja, S., Sharma, R., Agrawal, S. et al. Infrared and visible image fusion using modified spatial frequency-based clustered dictionary. Pattern Anal Applic 24, 575–589 (2021). https://doi.org/10.1007/s10044-020-00919-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-020-00919-z

Keywords

Search

Navigation