Pattern Recognition Letters ( IF 3.255 ) Pub Date : 2021-01-13 , DOI: 10.1016/j.patrec.2020.12.021 Usman Saeed
Soft biometrics, although not discriminant enough for person recognition provides additional information that aids traditional person recognition. Initially, attempts were made to integrate appearance-based facial soft biometrics, such as facial marks, skin color, and hair color/style, but more recently behavior-based facial soft biometrics, such as head dynamics, visual speech, and facial expressions have also been studied. Facial expressions are further classified as macro and micro-expressions and most of the existing studies using facial expressions as a soft biometric have focused on macro-expressions. Therefore, in this study, we investigate the utility of micro-expressions as a soft biometric for person recognition. The proposed system is based on the fusion of traditional facial features that model the facial appearance with soft biometric features that model the micro-expressions in an image sequence. We tested a texture-based traditional feature extraction technique, two motion-based soft biometric techniques, and several fusion methods at feature, rank, and decision level. The experiments were conducted on three commonly used micro-expression databases and exhibit an improvement of around 5% identification rate when soft biometric traits are fused with traditional face recognition at decision level.