Both region-based methods and direct methods have become popular in recent years for tracking the 6-dof pose of an object from monocular video sequences. Region-based methods estimate the pose of the object by maximizing the discrimination between statistical foreground and background appearance models, while direct methods aim to minimize the photometric error through direct image alignment. In practice, region-based methods only care about the pixels within a narrow band of the object contour due to the level-set-based probabilistic formulation, leaving the foreground pixels beyond the evaluation band unused. On the other hand, direct methods only utilize the raw pixel information of the object, but ignore the statistical properties of foreground and background regions. In this paper, we find it beneficial to combine these two kinds of methods together. We construct a new probabilistic formulation for 3D object tracking by combining statistical constraints from region-based methods and photometric constraints from direct methods. In this way, we take advantage of both statistical property and raw pixel values of the image in a complementary manner. Moreover, in order to achieve better performance when tracking heterogeneous objects in complex scenes, we propose to increase the distinctiveness of foreground and background statistical models by partitioning the global foreground and background regions into a small number of sub-regions around the object contour. We demonstrate the effectiveness of the proposed novel strategies on a newly constructed real-world dataset containing different types of objects with ground-truth poses. Further experiments on several challenging public datasets also show that our method obtains competitive or even superior tracking results compared to previous works. In comparison with the recent state-of-art region-based method, the proposed hybrid method is proved to be more stable under silhouette pose ambiguities with a slightly lower tracking accuracy.

中文翻译：

---

结合统计和光度约束的稳健单目 3D 对象跟踪方法

近年来，基于区域的方法和直接方法都变得流行，用于从单目视频序列跟踪对象的 6 自由度姿态。基于区域的方法通过最大化统计前景和背景外观模型之间的区分来估计对象的姿态，而直接方法旨在通过直接图像对齐来最小化光度误差。实际上，由于基于水平集的概率公式，基于区域的方法只关心对象轮廓窄带内的像素，而未使用评估带之外的前景像素。另一方面，直接方法仅利用对象的原始像素信息，而忽略了前景和背景区域的统计特性。在本文中，我们发现将这两种方法结合在一起是有益的。我们通过结合基于区域的方法的统计约束和直接方法的光度约束，为 3D 对象跟踪构建了一个新的概率公式。通过这种方式，我们以互补的方式利用了图像的统计属性和原始像素值。此外，为了在复杂场景中跟踪异构对象时获得更好的性能，我们建议通过将全局前景和背景区域划分为对象轮廓周围的少量子区域来增加前景和背景统计模型的独特性。我们证明了所提出的新策略在新构建的真实世界数据集上的有效性，该数据集包含具有真实姿势的不同类型的对象。对几个具有挑战性的公共数据集的进一步实验也表明，与以前的工作相比，我们的方法获得了有竞争力甚至更好的跟踪结果。与最近最先进的基于区域的方法相比，所提出的混合方法被证明在轮廓姿态模糊下更稳定，跟踪精度略低。