Skip to main content
SpringerLink
Log in

Thermal Face Recognition under Spatial Variation Conditions

  • APPLIED PROBLEMS
  • Published:
Pattern Recognition and Image Analysis Aims and scope Submit manuscript

Abstract

This paper proposes a novel method for degraded thermal face recognition using Hu Li moments. The method deals with spatial variations of thermal images resulting from low resolution and changes in head pose. We describe thermal images with a set of scalar quantities that can capture its significant features at component level. Each thermal image is divided into components where the statistical features of these components are combined using a fusion method. The method finds a combination of multiple statistical patterns to produce an integrated result that is enhanced in terms of information content for correct classification. We show that local representations using Hu Li moments provide higher discriminability and offer robustness against variability due to spatial changes. To evaluate the performance of the proposed method, we conduct thorough experiments and detailed analysis on a database consisting of 7500 images of different poses and variable spatial resolutions. This database is generated from an initial one that originally consists of 1500 images, by down-sampling the original images to four different low resolution levels beside the original level. The results show that the proposed method achieves significantly high recognition rates over various conditions resembling practical scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.
Fig. 13.
Fig. 14.
Fig. 15.
Fig. 16.

Similar content being viewed by others

REFERENCES

  1. B. A. Rajoub and R. Zwiggelaar, “Thermal facial analysis for deception detection,” IEEE Trans. Inf. Forensics Secur. 9 (6), 1015–1023 (2014).

    Article  Google Scholar 

  2. X. P. V. Maldague, Theory and Practice of Infrared Technology for Nondestructive Testing (Wiley, New York, 2001).

    Google Scholar 

  3. R. Gade and T. B. Moeslund, “Thermal cameras and applications: A survey,” Mach. Vision Appl. 25 (1), 245–262 (2014). https://doi.org/10.1007/s00138-013-0570-5

    Article  Google Scholar 

  4. X. Chen, P. J. Flynn, and K. W. Bowyer, “IR and visible light face recognition,” Comput. Vision Image Understand. 99 (3), 332–358 (2005).

    Article  Google Scholar 

  5. N. Zaeri, “Component-based thermal face recognition,” Br. J. Appl. Sci. Technol. 4 (6), 945–966 (2014).

    Article  Google Scholar 

  6. N. Zaeri, F. Baker, and R. Dib, “Thermal face recognition using moments invariants,” Int. J. Signal Process. Syst. 3 (2), 94–99 (2015). https://doi.org/10.12720/ijsps.3.2.94-99

    Article  Google Scholar 

  7. N. Zaeri, “Pose invariant thermal face recognition using AMI moments,” in Proc. 2016 UKSim-AMSS 18th Int. Conf. on Computer Modelling & Simulation (UKSim 2016) (Cambridge, UK, 2016), IEEE, pp. 60–64.

  8. N. Zaeri, “Hu Li moments for low resolution thermal face recognition,” in Proc. 2018 UKSim-AMSS 20th Int. Conf. on Modelling & Simulation (UKSim 2018) (Cambridge, UK, 2018), IEEE, pp. 79–83.

  9. D. A. Socolinsky, A. Selinger, and J. D. Neuheisel, “Face recognition with visible and thermal infrared imagery,” Comput. Vision Image Understand. 91 (1–2), 72–114 (2003).

    Article  Google Scholar 

  10. A. Srivastana and X. Liu, “Statistical hypothesis pruning for recognizing faces from infrared images,” Image Vision Comput. 21 (7), 651–661 (2003).

    Article  Google Scholar 

  11. Z. Xie, G. Liu, S. Wu, and Z. Fang, “Infrared face recognition based on blood perfusion and Fisher linear discrimination analysis,” in Proc. 2009 IEEE Int. Workshop on Imaging Systems and Techniques (IST 2009) (Shenzhen, China, 2009), IEEE, pp. 85–88.

  12. S. Wu, W. Lin, and S. Xie, “Skin heat transfer model of facial thermograms and its application in face recognition,” Pattern Recogn. 41 (8), 2718–2729 (2008).

    Article  Google Scholar 

  13. R. S. Choraś, “Thermal Face Recognition,” in Image Processing and Communications Challenges 7, Ed. by R. S. Choraś, Advances in Intelligent Systems and Computing (Springer, Cham, 2016), Vol. 389, pp. 37–46.

  14. G. Hermosilla, P. Loncomilla, and J. Ruiz-del-Solar, “Thermal face recognition using local interest points and descriptors for HRI applications,” in RoboCup 2010: Robot Soccer World Cup XIV, Ed. by J. Ruiz-del-Solar, E. Chown, and P. G. Plöger, Lecture Notes in Computer Science (Springer, Berlin, Heidelberg, 2010), Vol. 6556, pp. 25–35. https://doi.org/10.1007/978-3-642-20217-9_3.

  15. Y. Lu, J. Yang, S. Wu, Z. Fang, and Z. Xie, “Normalization of infrared facial images under variant ambient temperatures,” in Advanced Biometric Technologies, Ed. by G. Chetty and J. Yang (InTech, Rijeka, 2011), pp. 113–132. ISBN: 978-953-307-487-0.

    Google Scholar 

  16. R. S. Ghiass, O. Arandjelović, H. Bendada, and X. Maldague, “Illumination-invariant face recognition from a single image across extreme pose using a dual dimension AAM ensemble in the thermal infrared spectrum,” in Proc. 2013 Int. Joint Conf. on Neural Networks (IJCNN) (Dallas, TX, USA, 2013), IEEE, pp. 1-10, 2013.

  17. D. Bhattacharjee, A. Seal, S. Ganguly, M. Nasipuri, and D. K. Basu, “A comparative study of human thermal face recognition based on Haar wavelet transform and local binary pattern,” Comput. Intell. Neurosci. 2012, Article ID 261089, 12 pages (2012). https://doi.org/10.1155/2012/261089

  18. A. Seal, S. Ganguly, D. Bhattacharjee, M. Nasipuri, and D. K. Basu, “Automated thermal face recognition based on minutiae extraction,” Int. J. Comput. Intell. Stud. 2 (2), 133–156 (2013).

    Article  Google Scholar 

  19. Z. Wu, M. Peng, and T. Chen, “Thermal face recognition using convolutional neural network,” in Proc. 2016 Int. Conf. on Optoelectronics and Image Processing (ICOIP 2016) (Warsaw, Poland, 2016), IEEE, pp. 6–9.

  20. J. Lezama, Q. Qiu, and G. Sapiro, “Not afraid of the dark: NIR-VIS face recognition via cross-spectral hallucination and low-rank embedding,” in Proc. 2017 30th IEEE Conf. on Computer Vision and Pattern Recognition (CVPR 2017) (Honolulu, HI, 2017), IEEE, pp. 6807–6816|

  21. Y. Bi, M. Lv, Y. Wei, N. Guan, and W. Yi, “Multi-feature fusion for thermal face recognition,” Infrared Phys. Technol. 77, 366–374 (2016).

    Article  Google Scholar 

  22. G. Hermosilla Vigneau, J. L. Verdugo, G. F. Castro, F. Pizarro, and E. Vera, “Thermal face recognition under temporal variation conditions,” IEEE Access 5, 9663–9672 (2017). https://doi.org/10.1109/access.2017.2704296

    Article  Google Scholar 

  23. P. Liu and L. Yin, “Spontaneous thermal facial expression analysis based on trajectory-pooled fisher vector descriptor,” in Proc. 2017 IEEE Int. Conf. on Multimedia and Expo (ICME) (Hong Kong, China, 2017), IEEE, pp. 835–840.

  24. M. Saquib Sarfraz and R. Stiefelhagen, “Deep perceptual mapping for thermal to visible face recognition,” in Proc. British Machine Vision Conference (BMVC) (Swansea, UK, 2015), Ed. by X. Xie, M. W. Jones, and G. K. L. Tam (BMVA Press, 2015), pp. 9.1–9.11.

  25. S. Hu, N. J. Short, B. S. Riggan, et al., “A polarimetric thermal database for face recognition research,” in Proc. 2016 29th IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2016) (Las Vegas, NV, USA, 2016), IEEE, pp. 184–187. https://doi.org/10.1109/cvprw.2016.30.

  26. S. M. Iranmanesh, A. Dabouei, H. Kazemi, and N. M. Nasrabadi, “Deep cross polarimetric thermal-to-visible face recognition,” in Proc. 2018 Int. Conf. on Biometrics (ICB 2018) (Gold Coast, Australia, 2018), IEEE, pp. 166–173. https://doi.org/10.1109/icb2018.2018.00034.

  27. H. Maeng, S. Liao, D. Kang, S.-W. Lee, and A. K. Jain, “Nighttime face recognition at long distance: cross-distance and cross-spectral matching,” in Computer Vision — ACCV2012: 11th Asian Conference on Computer Vision, Daejeon, Korea, November 2012, Revised Selected Papers, Part II, Ed. by K. M. Lee, Y. Matsushita, J. M. Rehg, and Z. Hu, Lecture Notes in Computer Science (Springer, Berlin, Heidelberg, 2013), Vol. 7725, pp. 708–721.

  28. F. Juefei-Xu, D. K. Pal, and M. Savvides, “NIR-VIS heterogeneous face recognition via cross-spectral joint dictionary learning and reconstruction,” in Proc. 2015 IEEE Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW 2015) (Boston, MA, USA, 2015), IEEE, pp. 141–150. https://doi.org/10.1109/cvprw.2015.7301308.

  29. D. Yi, Z. Lei, and S. Z. Li, “Shared representation learning for heterogenous face recognition,” in Proc. 2015 11th IEEE Int. Conf. and Workshops on Automatic Face and Gesture Recognition (FG) (Ljubljana, Slovenia, 2015), IEEE, Vol. 1, pp. 1–7.

  30. S. Farokhi, U. Ullah Sheikh, J. Flusser, and B. Yang, “Near infrared face recognition using Zernike moments and Hermite kernels,” Inf. Sci. 316, 234–245 (2015).

    Article  Google Scholar 

  31. S. Wang, B. Pan, H. Chen, and Q. Ji, “Thermal augmented expression recognition,” IEEE Trans. Cybern. 48 (7), 2203-2214 (2018). https://doi.org/, 2018.https://doi.org/10.1109/tcyb.2017.2786309

  32. G. Hermosilla Vigneau, J. Verdugo, G. F. Castro, F. Pizarro, and E. Vera, “Thermal face recognition under temporal variation conditions,” IEEE Access 5, 9663–9672 (2017). https://doi.org/10.1109/acccess.2017.2704296

    Article  Google Scholar 

  33. D. U. Silverthorn, Human Physiology: An Integrated Approach, 6th ed. (Pearson, San Francisco, 2012).

    Google Scholar 

  34. S. I. Fox, Human Physiology, 13th ed. (McGraw-Hill, New York, 2012).

    Google Scholar 

  35. L. Sherwood, Human Physiology: From Cells to Systems, 7th ed. (Cengage Learning, Belmont, 2008).

    Google Scholar 

  36. J. B. West, Respiratory Physiology: The Essentials, 8th ed. (Lippincott Williams & Wilkins, Philadelphia, 2008).

    Google Scholar 

  37. R. E. Klabunde, Cardiovascular Physiology Concepts, 2nd ed. (Lippincott Williams & Wilkins, Philadelphia, 2012).

    Google Scholar 

  38. A. R. Crossman and D. Neary, Neuroanatomy: An Illustrated Colour Text, 4th ed. (Churchill Livingstone, London, 2010).

    Google Scholar 

  39. M. F. Bear, B. W. Connors, and M. A. Paradiso, Neuroscience: Exploring the Brain, 3rd ed. (Lippincott Williams and Wilkins, Philadelphia, 2007).

    Google Scholar 

  40. D. Sandidge, Digital Infrared Photography (Wiley, Hoboken, NJ, 2009).

    Google Scholar 

  41. J. R. Howell, M. P. Mengüç, and R. Siegel, Thermal Radiation Heat Transfer, 5th ed. (CRC Press, Boca Raton, FL, 2010).

    Book  Google Scholar 

  42. J. Flusser, J. Kautsky, and F. Šroubek, “Implicit moment invariants,” Int. J. Comput. Vision 86 (1), 72–86 (2010). https://doi.org/10.1007/s11263-009-0259-4

    Article  MathSciNet  Google Scholar 

  43. L. Jin and Z. Tianxu, “Fast algorithm for generation of moment invariants,” Pattern Recogn. 37 (8), 1745–1756 (2004).

    Article  Google Scholar 

  44. N. H. Foon, Y.-H. Pang, A. T. B. Jin, and D. N. C. Ling, “An efficient method for human face recognition using wavelet transform and Zernike moments,” in Proc. Int. Conf. on Computer Graphics, Imaging and Visualization (CGIV 2004) (Penang, Malaysia, 2004), IEEE, pp. 65–69.

  45. J. Flusser, T. Suk, and B. Zitová, 2D and 3D Image Analysis by Moments (Wiley, Hoboken, NJ, 2016). ISBN 978-1119039358.

    Book  Google Scholar 

  46. A. Nabatchian, E. Abdel-Raheem, and M. Ahmadi, “Human face recognition using different moment invariants: A comparative study,” in Proc. 2008 Congress on Image and Signal Processing (CISP 2008) (Sanya, Hainan, China, 2008), IEEE, Vol. 3, pp. 661–666.

  47. Y. Bentoutou, N. Taleb, M. Chikr El Mezouar, M. Taleb, and L. Jetto, “An invariant approach for image registration in digital subtraction angiography,” Pattern Recogn. 35 (12), 2853–2865 (2002).

    Article  Google Scholar 

  48. V. Ojansivu and J. Heikkilä, “A method for blur and affine invariant object recognition using phase-only bispectrum,” in Image Analysis and Recognition, ICIAR 2008, Ed. by A. Campilho and M. Kamel, Lecture Notes in Computer Science (Springer, Berlin, Heidelberg, 2008), Vol. 5112, pp. 527–536.

    Google Scholar 

  49. A. Nabatchian, I. Makaremi, E. Abdel-Raheem, and M. Ahmadi, “Pseudo-Zernike moment invariants for recognition of faces using different classifiers in FERET database,” in Proc. 2008 Third Int. Conf. on Convergence and Hybrid Information Technology (ICCIT 2008) (Busan, South Korea, 2008), IEEE, pp. 933–936.

  50. N. S. V. Rao, W. Wu, and C. W. Glover, “Algorithms for recognizing planar polygonal configurations using perspective images,” IEEE Trans. Rob. Autom. 8 (4), 480–486 (1992).

    Article  Google Scholar 

  51. A. M. Guzman, M. Goryawala, J. Wang, A. Barreto, J. Andrian, N. Rishe, and M. Adjouadi, “Thermal imaging as a biometrics approach to facial signature authentication,” IEEE J. Biomed. Health Inf. 17 (1), 214–222 (2013).

    Article  Google Scholar 

  52. P. Buddharaju, I. T. Pavlidis, P. Tsiamyrtzis, and M. Bazakos, “Physiology–based face recognition in the thermal infrared spectrum,” IEEE Trans. Pattern Anal. Mach. Intell. 29 (4), 613–626 (2007).

    Article  Google Scholar 

  53. N. Zaeri, “Discriminant phase component for face recognition,” J. Electr. Comput. Eng. 2012, Article ID 718915, 12 pages (2012). https://doi.org/10.1155/2012/718915

  54. X. Chen, L. Gu, S. Z. Li, and H.-J. Zhang, “Learning representative local features for face detection,” in Proc. 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001) (Kauai, HI, USA, 2001), IEEE, Vol. 1, pp. I-1126–I-1131.

  55. B. Heisele and T. Koshizen, “Components for face recognition,” in Proc. 6th IEEE Int. Conf. on Automatic Face and Gesture Recognition (FGR 2004) (Seoul, South Korea, 2004), IEEE, pp. 153–158.

  56. K. Tan and S. Chen, “Adaptively weighted sub-pattern PCA for face recognition,” Neurocomput. 64 (1–4), 505– 511 (2005).

    Article  Google Scholar 

  57. W.-Y. Lin, M.-Y. Chen, K. R. Widder, Y. H. Hu, and N. Boston, “Fusion of multiple facial regions for expression-invariant face recognition,” in Proc. 2007 IEEE 9th Int. Workshop on Multimedia Signal Processing (MMSP 2007) (Crete, Greece, 2007), IEEE, pp. 426–429,

  58. B. Zhang, S. Shan, X. Chen, and W. Gao, “Histogram of Gabor phase patterns (HGPP): A novel object representation approach for face recognition,” IEEE Trans. Image Process. 16 (1), 57–68 (2007).

    Article  MathSciNet  Google Scholar 

  59. J. Zou, Q. Ji, and G. Nagy, “A comparative study of local matching approach for face recognition,” IEEE Trans. Image Process. 16 (10), 2617–2628 (2007).

    Article  MathSciNet  Google Scholar 

  60. B. Hua and T. Liu, “Facial expression recognition based on local feature bidirectional 2DPCA,” in Proc. 2009 Int. Conf. on Information Technology and Computer Science (ITCS 2009) (Kiev, Ukraine, 2009), IEEE, Vol. 1, pp. 301–304.

  61. M. De Marsico, M. Nappi, and D. Riccio, “FARO: FAce Recognition against Occlusions and expression variations,” IEEE Trans. Syst., Man, Cybern. — Part A: Syst. Humans 40 (1), 121–132 (2010).

    Article  Google Scholar 

  62. Y. Xu, D. Zhang, J. Yang, and J.-Y. Yang, “A two-phase test sample sparse representation method for use with face recognition,” IEEE Trans. Circuits Syst. Video Technol. 21 (9), 1255–1262 (2011).

    Article  MathSciNet  Google Scholar 

  63. Y. Xu, Q. Zhu, Z. Fan, D. Zhang, J. Mi, and Z. Lai, “Using the idea of the sparse representation to perform coarse-to-fine face recognition,” Inf. Sci. 238, 138–148 (2013).

    Article  MathSciNet  Google Scholar 

  64. L. Liu, P. Fieguth, G. Zhao, M. Pietikäinen, and D. Hu, “Extended local binary patterns for face recognition,” Inf. Sci. 358–359, 56–72 (2016).

    Article  Google Scholar 

  65. Md. Zia Uddin, W. Khaksar, and J. Torresen, “Facial expression recognition using salient features and Convolutional Neural Network,” IEEE Access 5, 26146–26161 (2017). https://doi.org/10.1109/access.2017.2777003

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Studies, Arab Open University, P.O. Box 3322, 13033, Safat, Kuwait

    Naser Zaeri

Authors
  1. Naser Zaeri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Naser Zaeri.

Ethics declarations

The authors declare that they have no conflicts of interest.

Additional information

Naser Zaeri is an associate professor with the Department of Information Technology and Computing (ITC) at the Arab Open University (AOU), Kuwait. He has obtained his PhD in Electrical Engineering from University of Surrey, United Kingdom in 2008. He has obtained his M.Sc. and B.Sc. degrees in Electrical Engineering from Kuwait University (Honor List). He served as the Director of Research and Development at the AOU during (2012–2014). He was the Head of ITC Department at the AOU during (2010–2012). Also, he was with College of Engineering and Petroleum – Kuwait University, as a lecturer. He served as a consultant for many authorities and ministries. Dr. Zaeri has participated in and in charge of many projects in different fields of engineering and technology. He has more than 40 different publications in international journals and conferences. Also, he has served as a reviewer for many international reputable journals. His areas of interest are digital image processing, biometrics, pattern classification, and communications systems.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zaeri, N. Thermal Face Recognition under Spatial Variation Conditions. Pattern Recognit. Image Anal. 30, 108–124 (2020). https://doi.org/10.1134/S1054661820010174

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1054661820010174

Keywords:

Search

Navigation