Motion detection and object tracking play important roles in unsupervised human–machine interaction systems. Nevertheless, the human–machine interaction would become invalid when the system fails to detect the scene objects correctly due to occlusion and limited field of view. Thus, robust long-term tracking of scene objects is vital. In this paper, we present a 3D motion detection and long-term tracking system with simultaneous 3D reconstruction of dynamic objects. In order to achieve the high precision motion detection, an optimization framework with a novel motion pose estimation energy function is provided in the proposed method by which the 3D motion pose of each object can be estimated independently. We also develop an accurate object-tracking method which combines 2D visual information and depth. We incorporate a novel boundary-optimization segmentation based on 2D visual information and depth to improve the robustness of tracking significantly. Besides, we also introduce a new fusion and updating strategy in the 3D reconstruction process. This strategy brings higher robustness to 3D motion detection. Experiments results show that, for synthetic sequences, the root-mean-square error (RMSE) of our system is much smaller than Co-Fusion (CF); our system performs extremely well in 3D motion detection accuracy. In the case of occlusion or out-of-view on real scene data, CF will suffer the loss of tracking or object-label changing, by contrast, our system can always keep the robust tracking and maintain the correct labels for each dynamic object. Therefore, our system is robust to occlusion and out-of-view application scenarios.

中文翻译：

---

多目标的同步 3D 运动检测、长期跟踪和模型重建

运动检测和对象跟踪在无监督的人机交互系统中发挥着重要作用。然而，当系统由于遮挡和视野受限而无法正确检测场景对象时，人机交互就会失效。因此，对场景对象进行稳健的长期跟踪至关重要。在本文中，我们提出了一种 3D 运动检测和长期跟踪系统，同时对动态对象进行 3D 重建。为了实现高精度的运动检测，该方法提供了一种具有新颖运动姿态估计能量函数的优化框架，可以独立估计每个对象的3D运动姿态。我们还开发了一种结合二维视觉信息和深度的精确对象跟踪方法。我们结合了一种基于 2D 视觉信息和深度的新型边界优化分割，以显着提高跟踪的鲁棒性。此外，我们还在 3D 重建过程中引入了一种新的融合和更新策略。该策略为 3D 运动检测带来了更高的鲁棒性。实验结果表明，对于合成序列，我们系统的均方根误差（RMSE）远小于Co-Fusion（CF）；我们的系统在 3D 运动检测精度方面表现非常出色。在真实场景数据被遮挡或视野外的情况下，CF 将遭受跟踪丢失或对象标签更改，相比之下，我们的系统始终可以保持稳健的跟踪并为每个动态对象维护正确的标签。因此，我们的系统对遮挡和视野外的应用场景具有鲁棒性。