Monocular SLAM algorithms perform robustly when observing rigid scenes, however, they fail when the observed scene deforms, for example, in medical endoscopy applications. We present DefSLAM, the first monocular SLAM capable of operating in deforming scenes in real-time. Our approach intertwines Shape-from-Template (SfT) and Non-Rigid Structure-from-Motion (NRSfM) techniques to deal with the exploratory sequences typical of SLAM. A deformation tracking thread recovers the pose of the camera and the deformation of the observed map, at frame rate, by means of SfT processing a template that models the scene shape-at-rest. A deformation mapping thread runs in parallel with the tracking to update the template, at keyframe rate, by means of an isometric NRSfM processing a batch of full perspective keyframes. In our experiments, DefSLAM processes close-up sequences of deforming scenes, both in a laboratory controlled experiment and in medical endoscopy sequences, producing accurate 3D models of the scene with respect to the moving camera.

中文翻译：

---

DefSLAM：从单目序列跟踪和映射变形​​场景

单目 SLAM 算法在观察刚性场景时表现良好，但是，当观察到的场景变形时，例如在医疗内窥镜应用中，它们会失败。我们提出了 DefSLAM，这是第一个能够在实时变形场景中运行的单目 SLAM。我们的方法将形状从模板 (SfT) 和非刚性结构从运动 (NRSfM) 技术交织在一起，以处理 SLAM 典型的探索性序列。变形跟踪线程通过 SfT 处理对静止场景形状建模的模板，以帧速率恢复相机的姿态和观察到的地图的变形。变形映射线程与跟踪并行运行，通过等距 NRSfM 处理一批全透视关键帧，以关键帧速率更新模板。在我们的实验中，