当前位置: X-MOL 学术arXiv.cs.HC › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
ABC-Net: Semi-Supervised Multimodal GAN-based Engagement Detection using an Affective, Behavioral and Cognitive Model
arXiv - CS - Human-Computer Interaction Pub Date : 2020-11-17 , DOI: arxiv-2011.08690
Pooja Guhan and Manas Agarwal and Naman Awasthi and Gloria Reeves and Dinesh Manocha and Aniket Bera

We present ABC-Net, a novel semi-supervised multimodal GAN framework to detect engagement levels in video conversations based on psychology literature. We use three constructs: behavioral, cognitive, and affective engagement, to extract various features that can effectively capture engagement levels. We feed these features to our semi-supervised GAN network that does regression using these latent representations to obtain the corresponding valence and arousal values, which are then categorized into different levels of engagements. We demonstrate the efficiency of our network through experiments on the RECOLA database. To evaluate our method, we analyze and compare our performance on RECOLA and report a relative performance improvement of more than 5% over the baseline methods. To the best of our knowledge, our approach is the first method to classify engagement based on a multimodal semi-supervised network.

中文翻译:

ABC-Net:使用情感、行为和认知模型的基于半监督多模式 GAN 的参与检测

我们提出了 ABC-Net,这是一种新颖的半监督多模态 GAN 框架,用于检测基于心理学文献的视频对话中的参与度。我们使用三个结构:行为、认知和情感参与,来提取可以有效捕捉参与水平的各种特征。我们将这些特征提供给我们的半监督 GAN 网络,该网络使用这些潜在表示进行回归以获得相应的效价和唤醒值,然后将其分类为不同级别的参与度。我们通过对 RECOLA 数据库的实验证明了我们网络的效率。为了评估我们的方法,我们分析和比较了我们在 RECOLA 上的性能,并报告了相对于基准方法的 5% 以上的相对性能改进。据我们所知,
更新日期:2020-11-18
down
wechat
bug