Light field (LF) cameras provide rich spatio-angular visual representations by sensing the visual scene from multiple perspectives and have recently emerged as a promising technology to boost the performance of human-machine systems such as biometrics and affective computing. Despite the significant success of LF representation for constrained facial image analysis, this technology has never been used for face and expression recognition in the wild . In this context, this paper proposes a new deep face and expression recognition solution, called CapsField, based on a convolutional neural network and an additional capsule network that utilizes dynamic routing to learn hierarchical relations between capsules. CapsField extracts the spatial features from facial images and learns the angular part-whole relations for a selected set of 2D sub-aperture images rendered from each LF image. To analyze the performance of the proposed solution in the wild, the first in the wild LF face dataset, along with a new complementary constrained face dataset captured from the same subjects recorded earlier have been captured and are made available. A subset of the in the wild dataset contains facial images with different expressions, annotated for usage in the context of face expression recognition tests. An extensive performance assessment study using the new datasets has been conducted for the proposed and relevant prior solutions, showing that the CapsField proposed solution achieves superior performance for both face and expression recognition tasks when compared to the state-of-the-art.

中文翻译：

---

CapsField：使用胶囊路线在野外基于光场的面部和表情识别

光场（LF）摄像机通过从多个角度感知视觉场景来提供丰富的时空视角表示，并且最近成为一种有前途的技术，可以提高人机系统（如生物识别和情感计算）的性能。尽管LF表示法在约束人脸图像分析方面取得了巨大成功，但该技术从未用于人脸和表情识别在野外 。在这种情况下，本文提出了一种新的深层面孔和表情识别解决方案，称为CapsField，该解决方案基于卷积神经网络和另一个利用动态路由学习胶囊之间的层次关系的胶囊网络。CapsField从面部图像中提取空间特征，并学习从每个LF图像渲染的一组选定的2D子孔径图像的角整体角度关系。为了分析提出的解决方案在野外的性能，这是野外LF人脸数据集中的第一个，以及一个新的补充受约束的从较早记录的相同被摄对象捕获的面部数据集已被捕获并可用。野生数据集中的子集包含具有不同表情的脸部图像，并注明在脸部表情识别测试中使用。已针对提议的和相关的现有解决方案使用新的数据集进行了广泛的性能评估研究，表明与现有技术相比，CapsField提出的解决方案在面部和表情识别任务上均具有出色的性能。