We present a sensor-fusion method that exploits a depth camera and a gyroscope to track the articulation of a hand in the presence of excessive motion blur. In case of slow and smooth hand motions, the existing methods estimate the hand pose fairly accurately and robustly, despite challenges due to the high dimensionality of the problem, self-occlusions, uniform appearance of hand parts, etc. However, the accuracy of hand pose estimation drops considerably for fast-moving hands because the depth image is severely distorted due to motion blur. Moreover, when hands move fast, the actual hand pose is far from the one estimated in the previous frame, therefore the assumption of temporal continuity on which tracking methods rely, is not valid. In this paper, we track fast-moving hands with the combination of a gyroscope and a depth camera. As a first step, we calibrate a depth camera and a gyroscope attached to a hand so as to identify their time and pose offsets. Following that, we fuse the rotation information of the calibrated gyroscope with model-based hierarchical particle filter tracking. A series of quantitative and qualitative experiments demonstrate that the proposed method performs more accurately and robustly in the presence of motion blur, when compared to state of the art algorithms, especially in the case of very fast hand rotations.

中文翻译：

---

存在过度运动模糊的3D手部跟踪。

我们提出了一种传感器融合方法，该方法利用深度相机和陀螺仪来跟踪在过度运动模糊的情况下手的关节运动。在缓慢而平稳的手运动的情况下，尽管存在由于问题的高维度，自闭塞，手部外观均匀等带来的挑战，但现有方法仍可以相当准确且稳健地估计手的姿势。但是，手的准确性对于快速移动的手，姿势估计会大大下降，因为深度图像由于运动模糊而严重失真。此外，当手快速移动时，实际的手姿势与前一帧中估计的姿势相差甚远，因此，跟踪方法所依赖的时间连续性的假设是无效的。在本文中，我们结合陀螺仪和深度相机来跟踪快速移动的手。第一步，我们校准安装在手上的深度相机和陀螺仪，以识别其时间和姿势偏移。之后，我们将校准的陀螺仪的旋转信息与基于模型的分层粒子滤波器跟踪融合在一起。一系列定量和定性实验表明，与现有算法相比，尤其是在手快速旋转的情况下，所提出的方法在存在运动模糊的情况下可以更准确，更可靠地执行。