In this paper, we propose an approach to learn generic multi-modal mesh surface representations using a novel scheme for fusing texture and geometric data. Our approach defines an inverse mapping between different geometric descriptors computed on the mesh surface or its down-sampled version, and the corresponding 2D texture image of the mesh, allowing the construction of fused geometrically augmented images (FGAI). This new fused modality enables us to learn feature representations from 3D data in a highly efficient manner by simply employing standard CNNs in a transfer-learning mode. The proposed approach is both computationally and memory efficient, preserves intrinsic geometric information and learns highly discriminative feature representations by effectively fusing shape and texture information at data level. The efficacy of our approach is demonstrated for the tasks of facial action unit detection and expression classification. The extensive experiments conducted on the Bosphorus and BU-4DFE datasets show that our method produces a significant boost in the performance when compared to state-of-the-art solutions.

中文翻译：

---

使用融合几何增强图像学习 3D 形状表示：在面部表情和动作单元检测中的应用

在本文中，我们提出了一种使用融合纹理和几何数据的新方案来学习通用多模态网格表面表示的方法。我们的方法定义了在网格表面或其下采样版本上计算的不同几何描述符与网格的相应 2D 纹理图像之间的逆映射，从而允许构建融合几何增强图像 (FGAI)。这种新的融合模式使我们能够通过在转移学习模式下简单地采用标准 CNN 以高效的方式从 3D 数据中学习特征表示。所提出的方法在计算和内存方面都是高效的，保留了内在的几何信息，并通过在数据级别有效地融合形状和纹理信息来学习高度区分的特征表示。我们的方法在面部动作单元检测和表情分类任务中的有效性得到了证明。在博斯普鲁斯海峡和 BU-4DFE 数据集上进行的大量实验表明，与最先进的解决方案相比，我们的方法显着提高了性能。