当前位置: X-MOL 学术Int. J. Comput. Vis. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Real-Time Accurate 3D Head Tracking and Pose Estimation with Consumer RGB-D Cameras
International Journal of Computer Vision ( IF 19.5 ) Pub Date : 2017-02-02 , DOI: 10.1007/s11263-017-0988-8
David Joseph Tan , Federico Tombari , Nassir Navab

We demonstrate how 3D head tracking and pose estimation can be effectively and efficiently achieved from noisy RGB-D sequences. Our proposal leverages on a random forest framework, designed to regress the 3D head pose at every frame in a temporal tracking manner. One peculiarity of the algorithm is that it exploits together (1) a generic training dataset of 3D head models, which is learned once offline; and, (2) an online refinement with subject-specific 3D data, which aims for the tracker to withstand slight facial deformations and to adapt its forest to the specific characteristics of an individual subject. The combination of these works allows our algorithm to be robust even under extreme poses, where the user’s face is no longer visible on the image. Finally, we also propose another solution that utilizes a multi-camera system such that the data simultaneously acquired from multiple RGB-D sensors helps the tracker to handle challenging conditions that affect a subset of the cameras. Notably, the proposed multi-camera frameworks yields a real-time performance of approximately 8 ms per frame given six cameras and one CPU core, and scales up linearly to 30 fps with 25 cameras.

中文翻译:

使用消费类 RGB-D 相机进行实时准确的 3D 头部跟踪和姿势估计

我们展示了如何从嘈杂的 RGB-D 序列中有效地实现 3D 头部跟踪和姿势估计。我们的提议利用了随机森林框架,旨在以时间跟踪的方式在每一帧回归 3D 头部姿势。该算法的一个特点是它共同利用了 (1) 3D 头部模型的通用训练数据集,离线学习一次;(2) 对特定主题的 3D 数据进行在线细化,旨在使跟踪器能够承受轻微的面部变形并使其森林适应单个主题的特定特征。这些工作的结合使我们的算法即使在极端姿势下也能保持鲁棒性,此时用户的脸在图像上不再可见。最后,我们还提出了另一种利用多摄像头系统的解决方案,这样从多个 RGB-D 传感器同时获取的数据有助于跟踪器处理影响摄像头子集的挑战性条件。值得注意的是,所提出的多摄像头框架在给定六个摄像头和一个 CPU 内核的情况下可产生每帧约 8 毫秒的实时性能,并在 25 个摄像头的情况下线性扩展到 30 fps。
更新日期:2017-02-02
down
wechat
bug