Skip to main content
Log in

An auxiliary gaze point estimation method based on facial normal

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Considering the main disadvantage of the existing gaze point estimation methods which restrict user’s head movement and have potential injury on eyes, we propose a gaze point estimation method based on facial normal and binocular vision. Firstly, we calibrate stereo cameras to determine the extrinsic and intrinsic parameters of the cameras; Secondly, face is quickly detected by Viola–Jones framework and the center position of the two irises can be located based on integro-differential operators; The two nostrils and mouth are detected based on the saturation difference and their 2D coordinates can be calculated; Thirdly, the 3D coordinates of these five points are obtained by stereo matching and 3D reconstruction; After that, a plane fitting algorithm based on least squares is adopted to get the approximate facial plane, then, the normal via the midpoint of the two pupils can be figured out; Finally, the point-of-gaze can be obtained by getting the intersection point of the facial normal and the computer screen. Experimental results confirm the accuracy and robustness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Arantxa Villanueva and Rafael Cabeza (2008) A novel gaze estimation system with one calibration point. IEEE Transactions on Systems, Man, and Cybernetics 38(6):1123–1138

    Google Scholar 

  2. Shao G, Che M, Zhang B, Cen K, Gao W (2010) A novel simple 2D model of eye gaze estimation. In: IEEE 2nd International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), vol 1. Nanjing, Jiangsu, pp 300–304

  3. Zhu ZW, Ji Q (2007) Novel eye gaze tracking techniques under natural head movement. IEEE Trans Biomed Eng 54(12):2246–2260

    Article  MathSciNet  Google Scholar 

  4. Ning Zhang, Lei Chang, Xiping Xu (2012) Research on the technology of three-dimensional reconstruction based on machine vision. Laser Optoelectron Prog. doi:10.3788/LOP49.051001

    Google Scholar 

  5. JONESM (2009) Face recognition: where we are and where to go from here. IEEE TEIS 129:770–777

    Google Scholar 

  6. Wen G, Bo C, Shiguang Shan et al (2008) The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Trans Syst Man Cybern 38(1):149–161

    Article  Google Scholar 

  7. Yamazoe H, Utsumi A, Yonezawa T, Abe S (2008) Remote gaze estimation with a single camera based on facial-feature tracking without special calibration actions. In: Proceedings of the 2008 symposium on eye tracking research & applications. ACM, pp 245–250

  8. Yanhui S, Wenyong W, Xiaochun C (2005) The amelioration to the face recognition algorithm based on the Viola-Jones frame. J Northeast Norm Univ 37(3):24–27

    Google Scholar 

  9. Freund Yoav, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139

    Article  MathSciNet  MATH  Google Scholar 

  10. Luan B, Sörös P, Sejdić E (2013) A study of brain networks associated with swallowing using graph-theoretical approaches. PLoS ONE 8(8):e73577

    Article  Google Scholar 

  11. Daugman J (2004) How iris recognition works. Circuit Syst Video Technol IEEE Trans 14(1):21–30

    Article  Google Scholar 

  12. Sun W, Guo BL, Li DJ, Jia W (2013) Fast single-image dehazing method for visible-light systems. Opt Eng 52(9):093103

    Article  Google Scholar 

  13. Wei S (2013) A new single image fog removal algorithm based on physical model. Int J Light Electron Opt 124(21):4770–4775

    Article  Google Scholar 

  14. Sun W, Han L, Guo B, Jia W, Sun M (2014) A fast color image enhancement algorithm based on max intensity channel. J Mod Opt 61(6):466–477

    Article  Google Scholar 

  15. Viola Paul, Michael J Jones (2004) “Robust real-time face detection”, Int J Comp Vis, 57.2: 137–154

  16. Tan TN, He ZF, Sun ZN (2010) Efficient and robust segmentation of noisy iris images for non-cooperative iris recognition. Image Vis Comput. doi:10.1016/j.imavis.2009.05.008

    Google Scholar 

  17. Papageorgiou Constantine, Poggio Tomaso (2000) A trainable system for object detection. Int J Comput Vis 38(1):15–33

    Article  MATH  Google Scholar 

  18. Castillo Carlos D, Jacobs DW (2009) Using stereo matching with general epipolar geometry for 2D face recognition across pose. IEEE Trans Pattern Anal Mach Intell 31(12):1198–2304

    Article  Google Scholar 

  19. Jiangbo Lu, Cai Hua, Lou Jian-Guang, Li Jiang (2007) An epipolar geometry-based fast disparity estimation algorithm for multiview image and video coding. IEEE Trans Circuit Syst Video Technol 17(6):737–750

    Article  Google Scholar 

  20. Zhao Y, Liu HX, Wang ZY et al (2009) An improved nearest neighbor searching method for classification problems. J Nanjing Univ (Nat Sci) 45:455–462

    MATH  Google Scholar 

  21. Heiko Hirschmuller and Daniel Scharstein (2009) Evaluation of stereo matching costs on images with radiometric differences. IEEE Trans Pattern Anal Mach Intell 31(9):1582–1599

    Article  Google Scholar 

  22. Andreas Geiger, Julius Ziegler and Christoph Stiller (2011) StereoScan: dense 3D reconstruction in real-time. In: IEEE Intelligent vehicles symposium (IV). Baden-Baden, pp 963–968

Download references

Acknowledgments

This work was supported by Fundamental Research Funds for the Central Universities (Grant JB141307); National Nature Science Foundation of China (NSFC) (Grants 61201290), and NSFC Grants 61105066, 61305041, 61305040; the China Scholarship Council (CSC) and the National Institutes of Health (Grant R01CA165255) of the United States.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Wei Sun.

Additional information

Wei Sun and Nan Sun are co-first authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, W., Sun, N., Guo, B. et al. An auxiliary gaze point estimation method based on facial normal. Pattern Anal Applic 19, 611–620 (2016). https://doi.org/10.1007/s10044-014-0407-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-014-0407-5

Keywords

Navigation