A reliable, real time localization functionality is crutial for actively controlled capsule endoscopy robots, which are an emerging, minimally invasive diagnostic and therapeutic technology for the gastrointestinal (GI) tract. In this study, we extend the success of deep learning approaches from various research fields to the problem of sensor fusion for endoscopic capsule robots. We propose a multi-sensor fusion based localization approach which combines endoscopic camera information and magnetic sensor based localization information. The results performed on real pig stomach dataset show that our method achieves sub-millimeter precision for both translational and rotational movements.

中文翻译：

---

内窥镜胶囊机器人基于RGB相机信息和磁定位信息的基于深度学习的融合。

可靠的实时定位功能对于主动控制的胶囊内窥镜机器人至关重要，而胶囊内窥镜机器人是胃肠道（GI）的新兴，微创诊断和治疗技术。在这项研究中，我们将深度学习方法的成功从各个研究领域扩展到内窥镜胶囊机器人的传感器融合问题。我们提出了一种基于多传感器融合的定位方法，该方法结合了内窥镜相机信息和基于磁传感器的定位信息。在真实猪胃数据集上执行的结果表明，我们的方法在平移和旋转运动方面均达到了亚毫米级的精度。