research-article

3D Facial Similarity Measurement and Its Application in Facial Organization

Authors:
Chenlei Lv

School of Artificial Intelligence, Beijing Normal University, Beijing, China

School of Artificial Intelligence, Beijing Normal University, Beijing, China
View Profile

,
Zhongke Wu

School of Artificial Intelligence, Beijing Normal University, Beijing, China

School of Artificial Intelligence, Beijing Normal University, Beijing, China
View Profile

,
Xingce Wang

School of Artificial Intelligence, Beijing Normal University, Beijing, China

School of Artificial Intelligence, Beijing Normal University, Beijing, China
View Profile

,
Mingquan Zhou

School of Artificial Intelligence, Beijing Normal University, Beijing, China

School of Artificial Intelligence, Beijing Normal University, Beijing, China
View Profile

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 16 Issue 3Article No.: 82pp 1–20https://doi.org/10.1145/3397765

Published:05 July 2020Publication History

ACM Transactions on Multimedia Computing, Communications, and Applications

Abstract

We propose a novel framework for 3D facial similarity measurement and its application in facial organization. The construction of the framework is based on Kendall shape space theory. Kendall shape space is a quotient space that is constructed by shape features. In Kendall shape space, the shape features can be measured and is robust to similarity transformations. In our framework, a 3D face is represented by the facial feature landmarks model (FFLM), which can be regarded as the facial shape features. We utilize the geodesic in Kendall shape space to represent the FFLM similarity measurement, which can be regarded as the 3D facial similarity measurement. The FFLM similarity measurement is robust to facial expressions, head poses, and partial facial data. In our experiments, we compute the distance between different FFLMs in two public facial databases: FRGC2.0 and BosphorusDB. On average, we achieve a rank-one facial recognition rate of 98%. Based on the similarity results, we propose a method to construct the facial organization. The facial organization is a hierarchical structure that is achieved from the facial clustering by FFLM similarity measurement. Based on the facial organization, the performance of face searching in a large facial database can be improved obviously (about 400% improvement in experiments).

References

R. Ahdid, S. Safi, M. Fakir, and B. Manaut.2017. Geodesic distance on Riemannian manifold using Jacobi iterations in 3D face recognition system. Int. J. Inf. Commun. Technol. 6, 1 (2017), 10--19.Google Scholar
R. Ahdid, S. Safi, and B. Manaut. 2015. Three dimensional face surfaces analysis using geodesic distance. J. Comput. Sci. Applic. 3 (2015), 67--72.Google Scholar
B. Amberg, S. Romdhani, and T. Vetter. 2007. Optimal step nonrigid ICP algorithms for surface registration. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'07). 1--8.Google Scholar
A. N. Ansari and M. Abdel-Mottaleb. 2005. Automatic facial feature extraction and 3D face modeling using two orthogonal views with application to 3D face recognition. Pattern Recog. 38, 12 (2005), 2549--2563.Google ScholarDigital Library
S. Berretti, A. Bimbo, and P. Pala. 2010. 3D face recognition using iso-geodesic stripes. IEEE Trans. Pattern Anal. Mach. Intell. 32, 12 (2010), 2162--2177.Google ScholarDigital Library
S. Berretti, N. Werghi, A. D. Bimbo et al. 2013. Matching 3D face scans using interest points and local histogram descriptors. Comput. Graph. 37, 5 (2013), 509--525.Google ScholarDigital Library
S. Berretti, N. Werghi, A. D. Bimbo, et al. 2014. Selecting stable key points and local descriptors for person identification using 3D face scans. Vis. Comput. 30, 11 (2014), 1275--1292.Google ScholarDigital Library
V. Blanz and T. Vetter. 2003. Shape analysis of elastic curves in Euclidean spaces. IEEE Trans. Pattern Anal. Mach. Intell. 25, 9 (2003), 1063--1074.Google ScholarDigital Library
J. Booth, A. Roussos, A. Ponniah, et al. 2018. Large scale 3D morphable models. Int. J. Comput. Vis. 126, 2--4 (2018), 233--254.Google ScholarDigital Library
A. M. Bronstein, M. M. Bronstein, R. Kimmel, et al. 2005. Three dimensional face recognition. Int. J. Comput. Vis. 64, 1 (2005), 5--30.Google ScholarDigital Library
C. Cao, Y. Weng, K. Zhou, et al. 2014. FaceWarehouse: A 3D facial expression database for visual computing. IEEE Trans. Vis. Comput. Graph. 20, 3 (2014), 413--425.Google ScholarDigital Library
H. Chen and B. Bhanu. 2007. Human ear recognition in 3D. IEEE Trans. Pattern Anal. Mach. Intell. 29, 4 (2007), 718--737.Google ScholarDigital Library
T. F. Cootes, G. J. Edwards, and C. J. Taylor. 2001. Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23, 6 (2001), 681--685.Google ScholarDigital Library
T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham. 1995. Active shape models—their training and application. Comput. Vis. Image Underst. 61, 1 (1995), 38--59.Google ScholarDigital Library
C. Ding and D. Tao. 2015. Robust face recognition via multimodal deep face representation. IEEE Trans. Multimedia 17, 11 (2015), 2049--2058.Google ScholarDigital Library
C. Dorai and A. K. Jain. 1997. COSMOS-A representation scheme for 3D free-form objects. IEEE Trans. Pattern Anal. Mach. Intell. 19, 10 (1997), 1115--1130.Google ScholarDigital Library
H. Drira, B. B. Amor, A. Srivastava, et al. 2013. 3D face recognition under expressions, occlusions, and pose variations. IEEE Trans. Pattern Anal. Mach. Intell. 35, 9 (2013), 2270--2283.Google ScholarDigital Library
H. Drira, B. B. Amor, A. Srivastava, and M. Daoudi. 2009. A Riemannian analysis of 3D nose shapes for partial human biometrics. In Proceedings of the IEEE International Conference on Computer Vision. 2050--2057.Google Scholar
I. L. Dryden, A. Kume, Le Huiling, et al. 2008. A multi-dimensional scaling approach to shape analysis. Biometrika 95, 4 (2008), 779--798.Google ScholarCross Ref
B. Efraty, E. Bilgazyev, S. Shah, et al. 2012. Profile-based 3D-aided face recognition. Pattern Recog. 45, 1 (2012), 43--53.Google ScholarDigital Library
M. Emambakhsh and A. Evans. 2016. Nasal patches and curves for expression-robust 3D face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39, 5 (2016), 995--1007.Google ScholarDigital Library
C. Ferrari, G. Lisanti, S. Berretti, et al. 2017. A dictionary learning based 3D morphable shape model. IEEE Trans. Multimedia 19, 12 (2017), 2666--2058.Google ScholarCross Ref
B. J. Frey and D. Dueck. 2007. Clustering by passing messages between data points. Science 315, 5814 (2007), 972--976.Google Scholar
A. V. Gaikwad, S. J. Shigwan, and S. P. Awate. 2015. A statistical model for smooth shapes in Kendall shape space. In Medical Image Computing and Computer-Assisted Intervention (MICCAI'15), N. Navab, J. Hornegger, W. Wells, and A. Frangi (Eds.), Vol. 9351. 628--635.Google Scholar
S. Ganguly, D. Bhattacharjee, and M. Nasipuri. 2017. Fuzzy matching of edge and curvature based features from range images for 3D face recognition. Intell. Autom. Soft Comput. 23, 1 (2017), 51--62.Google ScholarCross Ref
S. Z. Gilani and A. Mian. 2016. Towards large-scale 3D face recognition. In Proceedings of the IEEE International Conference on Digital Image Computing: Techniques and Applications. 1--8.Google Scholar
S. Gilani, A. Mian, and P. Eastwood. 2017. Deep, dense and accurate 3D face correspondence for generating population specific deformable models. Pattern Recog. 69 (2017), 238--250.Google ScholarDigital Library
S. Z. Gilani, F. Shafait, and A. Mian. 2015. Shape-based automatic detection of a large number of 3D facial landmarks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4639--4648.Google Scholar
B. Gokberk, A. Ali Salah, and L. Akarun. 2005. Rank-based decision fusion for 3D shape-based face recognition. Audio Video-based Biomet. Person Authent. 3246 (2005), 1019--1028.Google ScholarDigital Library
H. Hawraa, Z. Bilal, and K. Ahmed. 2019. 3D face factorisation for face recognition using pattern recognition algorithms. Cyber. Inf. Technol. 19, 2 (2019), 28--37.Google Scholar
D. Huang, M. Ardabilian, Y. Wang, et al. 2012. 3-D face recognition using eLBP-based facial description and local feature hybrid matching. IEEE Trans. Inf. Forens. Sec. 7, 5 (2012), 1551--1565.Google ScholarDigital Library
S. Jahanbin, H. Choi, Y. Liu, et al. 2008. Three dimensional face recognition using iso-geodesic and iso-depth curves. In Proceedings of the IEEE International Conference on Biometrics: Theory, Applications and Systems. 1--6.Google ScholarCross Ref
S. Jahanbin, R. Jahanbin, A. C. Bovik, et al. 2013. Passive three dimensional face recognition using iso-geodesic contours and procrustes analysis. Int. J. Comput. Vis. 105, 1 (2013), 87--108.Google ScholarCross Ref
A. E. Johnson. 1997. Spin Images: A Representation for 3-D Surface Matching. Ph.D. Dissertation. Carnegie Mellon University, Pittsburgh, PA.Google Scholar
M. Kafai, K. Eshghi, and B. Bhanu. 2014. Discrete cosine transform locality-sensitive hashes for face retrieval. IEEE Trans. Multimedia 16, 4 (2014), 1090--1103.Google ScholarDigital Library
D. G. Kendall. 1984. Shape manifolds, procrustean metrics, and complex projective spaces. Bull. London Math. Soc. 16, 2 (1984), 81--121.Google ScholarCross Ref
P. Koppen, Z. Feng, J. Kittler, et al. 2018. Gaussian mixture 3D morphable face model. Pattern Recog. 74 (2018), 617--628.Google ScholarDigital Library
S. Kurtek and H. Drira. 2015. A comprehensive statistical framework for elastic shape analysis of 3D faces. Comput. Graph. 51 (2015), 52--59.Google ScholarDigital Library
Y. Lei, M. Bennamoun, M. Hayat, et al. 2014. An efficient 3D face recognition approach using local geometrical signatures. Pattern Recog. 47, 2 (2014), 509--524.Google ScholarDigital Library
H. Li, D. Huang, J. M. Morvan, et al. 2015. Towards 3D face recognition in the real: A registration-free approach using fine-grained matching of 3D keypoint descriptors. Int. J. Comput. Vis. 113, 2 (2015), 128--142.Google ScholarDigital Library
H. Li, J. Sun, Z. Xu, et al. 2017. Multimodal 2D+3D facial expression recognition with deep fusion convolutional neural network. IEEE Trans. Multimedia 19, 12 (2017), 2816--2831.Google ScholarCross Ref
C. Lv, Z. Wu, X. Wang, and M. Zhou. 2019. 3D facial expression modeling based on facial landmarks in single image. Neurocomputing 355 (2019), 155--162.Google ScholarDigital Library
C. Lv, Z. Wu, X. Wang, M. Zhou, and K. Toh. 2019. Nasal similarity measure of 3D faces based on curve shape space. Pattern Recog. 88 (2019), 458--469.Google ScholarCross Ref
C. Lv and J. Zhao. 2018. 3D face recognition based on local conformal parameterization and iso-geodesic stripes analysis. Math. Prob. Eng. 2018 (2018), 1--10.Google Scholar
Z. Wang, M. Xu, Y. Ren, et al. 2018. Saliency detection in face videos: A data-driven approach. IEEE Trans. Multimedia 20, 6 (2018), 1335--1349.Google ScholarCross Ref
S. A. Mahmood, R. F. Ghani, and A. A. Kerim. 2014. 3D face recognition using pose invariant nose region detector. In Proceedings of the Computer Science and Electronic Engineering Conference. 103--108.Google Scholar
I. Mpiperis, S. Malassiotis, and M. G. Strintzis. 2008. Bilinear models for 3-D face and facial expression recognition. IEEE Trans. Inf. Forens. Sec. 3, 3 (2008), 498--511.Google ScholarDigital Library
A. Ouamane, A. Chouchane, et al. 2017. Efficient tensor-based 2D+3D face verification. IEEE Trans. Inf. Forens. Sec. 12, 11 (2017), 2751--2762.Google ScholarCross Ref
A. Patel and W. A. P. Smith. 2015. Manifold-based constraints for operations in face space. Pattern Recog. 52 (2015), 206--217.Google ScholarDigital Library
A. P. Pentland. 1991. Face recognition using eigenfaces. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 586--591.Google Scholar
P. Perakis, G. Passalis, T. Theoharis, and I. Kakadiaris. 2013. 3D facial landmark detection under large yaw and expression variations. IEEE Trans. Pattern Anal. Mach. Intell. 35, 7 (2013), 1552--1564.Google ScholarDigital Library
P. J. Phillips, P. J. Flynn, T. Scruggs, et al. 2006. Preliminary face recognition grand challenge results. Autom. Face Gest. Recog. 15--24.Google Scholar
A. Savran, N. Alyuz, H. Dibeklioglu, et al. 2008. Bosphorus database for 3D face analysis. In Proceedings of the 1st COST 2101 Workshop on Biometrics and Identity Management. 47--56.Google ScholarDigital Library
S. Schwab, T. Chateau, C. Blanc, and L. Trassoudaine. 2013. A multi-cue spatio-temporal framework for automatic frontal face clustering in video sequences. Eurasip J. Image Vid. Proc. 2013, 1 (Jan. 2013), 1--12.Google Scholar
D. Smeets, J. Keustermans, D. Vandermeulen, et al. 2013. MeshSIFT: Local surface features for 3D face recognition under expression variations and partial data. Comput. Vis. Image Underst. 117, 2 (2013), 158--169.Google ScholarDigital Library
S. Soltanpour and Q. Wu. 2019. Weighted extreme sparse classifier and local derivative pattern for 3D face recognition. IEEE Trans. Image Proc. 28, 6 (2019), 3020--3033.Google ScholarCross Ref
F. M. Sukno, J. L. Waddington, and P. F. Whelan. 2015. 3-D facial landmark localization with asymmetry patterns and shape regression from incomplete local features. IEEE Trans. Cyber. 45, 9 (2015), 1717--1730.Google ScholarCross Ref
V. Surazhsky, T. Surazhsky, D. Kirsanov, et al. 2005. Fast exact and approximate geodesics on meshes. ACM Trans. Graph. 24, 3 (2005), 553--560.Google ScholarDigital Library
Y. Tang, H. Li, X. Sun, et al. 2017. Principal curvature measures estimation and application to 3D face recognition. J. Math. Imag. Vis. 2017, 3 (2017), 1--23.Google Scholar
Y. Wang, G. Pan, Z. Wu, and Y. Wang. 2006. Exploring facial expression effects in 3D face recognition using partial ICP. In Proceedings of the Asian Conference on Computer Vision. 3851 (2006), 581--590.Google Scholar
P. Yan and K. W. Bowyer. 2007. Biometric recognition using 3D ear shape. IEEE Trans. Pattern Anal. Mach. Intell. 29, 8 (2007), 1297--1308.Google ScholarDigital Library
L. Yuan, W. Liu, and Y. Li. 2016. Non-negative dictionary based sparse representation classification for ear recognition with occlusion. Neurocomputing 171, C (2016), 540--550.Google Scholar
W. Zeng, D. Samaras, and X. Gu. 2010. Ricci flow for 3D shape analysis. IEEE Trans. Pattern Anal. Mach. Intell. 32, 4 (2010), 662--677.Google ScholarDigital Library
L. Zhang, L. Li, H. Li, et al. 2016. 3D ear identification using block-wise statistics-based features and LC-KSVD. IEEE Trans. Multimedia 18, 8 (2016), 1531--1541.Google ScholarDigital Library
X. Zhu, Z. Lei, J. Yan, et al. 2015. High-fidelity pose and expression normalization for face recognition in the wild. Comput. Vis. Pattern Recog. 787--796.Google Scholar

Index Terms

3D Facial Similarity Measurement and Its Application in Facial Organization
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Shape representations
  2. Computer graphics
    1. Shape modeling
      1. Shape analysis

Recommendations

3D facial expression modeling based on facial landmarks in single image
Abstract
Facial expression modeling is important for many applications such as human emotional analysis and facial animation. Generally, facial expression modeling from single 2D facial image is difficult. Different head poses and scales of ...
Read More
Facial marks: soft biometric for face recognition
ICIP'09: Proceedings of the 16th IEEE international conference on Image processing

We propose to utilize micro features, namely facial marks (e.g., freckles, moles, and scars) to improve face recognition and retrieval performance. Facial marks can be used in three ways: i) to supplement the features in an existing face matcher, ii) to ...
Read More
Constructing 3D facial hierarchical structure based on surface measurements

In this paper, we propose a novel framework for 3D facial similarity measures and facial data organization. The 3D facial similarity measures of our method are based on iso-geodesic stripes and conformal parameterization. Using the conformal ...
Read More

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Published in
ACM Transactions on Multimedia Computing, Communications, and Applications Volume 16, Issue 3
August 2020
364 pages
ISSN:1551-6857
EISSN:1551-6865
DOI:10.1145/3409646
Editor:
Alberto Del Bimbo
University of Firenze, Italy
Issue’s Table of Contents
Copyright © 2020 ACM
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
- Published: 5 July 2020
- Online AM: 7 May 2020
- Revised: 1 April 2020
- Accepted: 1 April 2020
- Received: 1 November 2019
Published in tomm Volume 16, Issue 3

Permissions
Request permissions about this article.
Request Permissions

Check for updates
Author Tags
Kendall shape space
face recognition
facial data organization
facial feature landmarks model
Qualifiers
- research-article
- Research
- Refereed
Conference
Funding Sources
Other Metrics
View Article Metrics

Article Metrics
- 11
  Total Citations
  View Citations
- 239
  Total Downloads
- Downloads (Last 12 months)28
- Downloads (Last 6 weeks)7
Other Metrics
View Author Metrics
Cited By
View all

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format .

View HTML Format

3D Facial Similarity Measurement and Its Application in Facial Organization

ACM Transactions on Multimedia Computing, Communications, and Applications

Abstract

References

Cited By

Index Terms

Recommendations

3D facial expression modeling based on facial landmarks in single image

Facial marks: soft biometric for face recognition

Constructing 3D facial hierarchical structure based on surface measurements