当前位置: X-MOL 学术Pattern Recogn. Lett. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Sequential fusion of facial appearance and dynamics for depression recognition
Pattern Recognition Letters ( IF 3.9 ) Pub Date : 2021-07-21 , DOI: 10.1016/j.patrec.2021.07.005
Qian Chen 1 , Iti Chaturvedi 2 , Shaoxiong Ji 3 , Erik Cambria 1
Affiliation  

In mental health assessment, it is validated that nonverbal cues like facial expressions can be indicative of depressive disorders. Recently, the multimodal fusion of facial appearance and dynamics based on convolutional neural networks has demonstrated encouraging performance in depression analysis. However, correlation and complementarity between different visual modalities have not been well studied in prior methods. In this paper, we propose a sequential fusion method for facial depression recognition. For mining the correlated and complementary depression patterns in multimodal learning, a chained-fusion mechanism is introduced to jointly learn facial appearance and dynamics in a unified framework. We show that such sequential fusion can provide a probabilistic perspective of the model correlation and complementarity between two different data modalities for improved depression recognition. Results on a benchmark dataset show the superiority of our method against several state-of-the-art alternatives.



中文翻译:

面部外观和动态的顺序融合用于抑郁症识别

在心理健康评估中,已证实面部表情等非语言线索可以指示抑郁症。最近,基于卷积神经网络的面部外观和动力学的多模态融合在抑郁症分析中表现出令人鼓舞的表现。然而,在先前的方法中还没有很好地研究不同视觉模式之间的相关性和互补性。在本文中,我们提出了一种用于面部抑郁识别的顺序融合方法。为了挖掘多模态学习中相关和互补的抑郁模式,引入了一种链式融合机制,在统一框架中联合学习面部外观和动态。我们表明,这种顺序融合可以提供两种不同数据模式之间模型相关性和互补性的概率视角,以改善抑郁症的识别。基准数据集的结果显示了我们的方法相对于几种最先进的替代方案的优越性。

更新日期:2021-07-29
down
wechat
bug