Skip to main content
SpringerLink
Log in

Robust face tracking using multiple appearance models and graph relational learning

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

This paper addresses the problem of appearance matching across different challenges while doing visual face tracking in real-world scenarios. In this paper, FaceTrack is proposed that utilizes multiple appearance models with its long-term and short-term appearance memory for efficient face tracking. It demonstrates robustness to deformation, in-plane and out-of-plane rotation, scale, distractors and background clutter. It integrates on the advantages of the tracking-by-detection by using a face detector that tackles the drastic scale appearance change of a face. A weighted score-level fusion strategy is proposed to obtain the face tracking output having the highest fusion score by generating candidates around possible face locations. FaceTrack showcases impressive performance when initiated automatically by outperforming several state-of-the-art trackers, except Struck by a very minute margin: 0.001 in precision and 0.017 in success, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. Lowe [13] uses the ratio test to eradicate matches higher than 0.8. In FaceTrack experiments, 0.75 is used.

  2. The Gaussian kernel parameters are \(\sigma =6.0\), with a \(5\times 5\) filter size. The denominator, \(\varTheta \) of the exponential kernel is taken as 8000.0.

  3. Partial update: by replacing 12.5% of the face appearance features in ICM model and 10% of the face appearance features in BDM model, respectively, full update: by replacing 100% of the ICM and BDM model.

  4. Any other face detector can be used for initialization purpose.

  5. https://github.com/samhare/struck.

  6. https://github.com/gnebehay/CppMT.

  7. https://github.com/sinbycos/TUNA.

References

  1. Bashbaghi, S., Granger, E., Sabourin, R., Bilodeau, G.A.: Robust watch-list screening using dynamic ensembles of svms based on multiple face representations. Mach. Vis. Appl. 28(1), 219–241 (2017)

    Article  Google Scholar 

  2. Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15(6), 1373–1396 (2003)

    Article  Google Scholar 

  3. Chakravorty, T., Bilodeau, G.A., Granger, E.: Contextual object tracker with structure encoding. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4937–4941 (2015)

  4. Chakravorty, T., Bilodeau, G.A., Granger, É.: Tracking using numerous anchor points. Mach. Vis. Appl. 29(2), 247–261 (2018)

    Article  Google Scholar 

  5. Dewan, M.A.A., Granger, E., Marcialis, G.L., Sabourin, R., Roli, F.: Adaptive appearance model tracking for still-to-video face recognition. Pattern Recogn. 49, 129–151 (2016)

    Article  Google Scholar 

  6. Elhamifar, E., Vidal, R.: Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2765–2781 (2013)

    Article  Google Scholar 

  7. Grabner, H., Grabner, M., Bischof, H.: Real-time tracking via on-line boosting. In: Proceedings BMVC, pp 6.1–6.10 (2006)

  8. Hare, S., Saffari, A., Torr, P.H.S.: Struck: structured output tracking with kernels. pp 263–270 (2011). https://doi.org/10.1109/ICCV.2011.6126251

  9. Henriques, J., Caseiro, R., Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: Proceedings of the 12th European Conference on Computer Vision—Volume Part IV, ECCV’12, pp 702–715. Springer, Berlin (2012)

  10. Holder, L.B., Cook, D.J.: Graph-based relational learning: current and future directions. SIGKDD Explor. Newsl. 5(1), 90–93 (2003)

    Article  Google Scholar 

  11. Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012)

    Article  Google Scholar 

  12. Liao, S., Jain, A.K., Li, S.Z.: A fast and accurate unconstrained face detector. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 211–223 (2016)

    Article  Google Scholar 

  13. Lowe, D.G.: Object recognition from local scale-invariant features. In: Proceedings of the International Conference on Computer Vision—vol. 2, ICCV ’99. IEEE Computer Society, Washington, DC (1999)

  14. Lu, J., Wang, G., Deng, W., Jia, K.: Reconstruction-based metric learning for unconstrained face verification. IEEE Trans. Inf. Forensics Secur. 10(1), 79–89 (2015)

    Article  Google Scholar 

  15. Mei, X., Ling, H.: Robust visual tracking using l1 minimization. In: IEEE 12th International Conference on Computer Vision, ICCV 2009, Kyoto, Japan, 27 Sept–4 Oct 2009, pp. 1436–1443 (2009)

  16. Nebehay, G., Pflugfelder, R.: Consensus-based matching and tracking of keypoints for object tracking. In: IEEE Winter Conference on Applications of Computer Vision, pp. 862–869 (2014)

  17. Ross, D.A., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. 77(1–3), 125–141 (2008)

    Article  Google Scholar 

  18. Salti, S., Cavallaro, A., Di Stefano, L.: Adaptive appearance modeling for video tracking: survey and evaluation. Trans. Image Process. 21(10), 4334–4348 (2012)

    Article  MathSciNet  Google Scholar 

  19. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)

    Article  Google Scholar 

  20. St-Charles, P.L., Bilodeau, G.A.: Improving background subtraction using local binary similarity patterns. In: IEEE Winter Conference on Applications of Computer Vision, pp. 509–515 (2014)

  21. Wang, D., Lu, H., Yang, M.H.: Least soft-threshold squares tracking. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2371–2378 (2013)

  22. Wolf, L., Hassner, T., Maoz, I.: Face recognition in unconstrained videos with matched background similarity. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 11, p. 529534. IEEE Computer Society, USA (2011)

  23. Wu, Y., Lim, J., Yang, M.H.: Online object tracking: a benchmark. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2411–2418 (2013)

Download references

Acknowledgements

This work was supported in part by FRQNT team Grant #167442 and by REPARTI (Regroupement pour l’étude des environnements partagés intelligents répartis) FRQNT strategic cluster.

Author information

Authors and Affiliations

  1. LITIV Laboratory, Department of Computer and Software Engineering, Polytechnique Montreal, Montreal, QC, H3T1J4, Canada

    Tanushri Chakravorty & Guillaume-Alexandre Bilodeau

  2. LIVIA, Department of Systems Engineering, École de technologie supérieure, Montreal, QC, H3C1K3, Canada

    Éric Granger

Authors
  1. Tanushri Chakravorty
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guillaume-Alexandre Bilodeau
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Éric Granger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tanushri Chakravorty.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chakravorty, T., Bilodeau, GA. & Granger, É. Robust face tracking using multiple appearance models and graph relational learning. Machine Vision and Applications 31, 23 (2020). https://doi.org/10.1007/s00138-020-01071-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-020-01071-8

Keywords

Search

Navigation