Head pose estimation is a fundamental task for face and social related research. Although 3D morphable model (3DMM) based methods relying on depth information usually achieve accurate results, they usually require frontal or mid-profile poses which preclude a large set of applications where such conditions can not be garanteed, like monitoring natural interactions from fixed sensors placed in the environment. A major reason is that 3DMM models usually only cover the face region. In this paper, we present a framework which combines the strengths of a 3DMM model fitted online with a prior-free reconstruction of a 3D full head model providing support for pose estimation from any viewpoint. In addition, we also proposes a symmetry regularizer for accurate 3DMM fitting under partial observations, and exploit visual tracking to address natural head dynamics with fast accelerations. Extensive experiments show that our method achieves state-of-the-art performance on the public BIWI dataset, as well as accurate and robust results on UbiPose, an annotated dataset of natural interactions that we make public and where adverse poses, occlusions or fast motions regularly occur.

中文翻译：

---

HeadFusion：结合 3D 可变形模型和 3D 重建的 360 度头部姿势跟踪


头部姿势估计是面部和社会相关研究的一项基本任务。虽然依赖于深度信息的基于 3D 变形模型 (3DMM) 的方法通常会获得准确的结果，但它们通常需要正面或中间轮廓的姿势，这妨碍了无法保证此类条件的大量应用，例如通过放置的固定传感器监控自然交互在环境中。一个主要原因是 3DMM 模型通常只覆盖面部区域。在本文中，我们提出了一个框架，它将在线拟合的 3DMM 模型的优点与 3D 全头部模型的无先验重建相结合，为从任何角度进行姿势估计提供支持。此外，我们还提出了一种对称正则化器，用于在部分观测下进行精确的 3DMM 拟合，并利用视觉跟踪来解决快速加速的自然头部动力学问题。大量实验表明，我们的方法在公共 BIWI 数据集上实现了最先进的性能，并且在 UbiPose 上实现了准确而稳健的结果，UbiPose 是我们公开的自然交互的带注释数据集，其中存在不良姿势、遮挡或快速运动经常发生。