The assumption of scene rigidity is common in visual SLAM algorithms. However, it limits their applicability in populated real-world environments. Furthermore, most scenarios including autonomous driving, multi-robot collaboration and augmented/virtual reality, require explicit motion information of the surroundings to help with decision making and scene understanding. We present in this paper DynaSLAM II, a visual SLAM system for stereo and RGB-D camera configurations that tightly integrates the multi-object tracking capability. DynaSLAM II makes use of instance semantic segmentation and ORB features to track dynamic objects. The structures of the static scene and the dynamic objects are optimized jointly with the trajectories of both the camera and the moving agents within a novel bundle adjustment proposal. The 3D bounding boxes of the objects are also estimated and loosely optimized within a fixed temporal window. We demonstrate that tracking dynamic objects does not only provide rich clues for scene understanding but can be also beneficial for camera tracking.

中文翻译：

---

DynaSLAM II：紧密耦合的多对象跟踪和SLAM

场景刚性的假设在视觉SLAM算法中很常见。但是，它限制了它们在人口稠密的现实环境中的适用性。此外，大多数场景（包括自动驾驶，多机器人协作和增强/虚拟现实）都需要周围环境的明确运动信息，以帮助决策和场景理解。我们在本文中介绍DynaSLAM II，这是一种用于立体声和RGB-D摄像机配置的可视SLAM系统，该系统紧密集成了多对象跟踪功能。DynaSLAM II利用实例语义分割和ORB功能来跟踪动态对象。静态场景和动态对象的结构在一个新颖的捆绑调整方案中与摄像机和移动主体的轨迹一起进行了优化。对象的3D边界框也可以在固定的时间窗口内进行估算和粗略优化。我们证明了跟踪动态对象不仅为场景理解提供了丰富的线索，而且对于相机跟踪也可能是有益的。