Mapping and localization, preferably from a small number of observations, are fundamental tasks in robotics. We address these tasks by combining spatial structure (differentiable mapping) and end-to-end learning in a novel neural network architecture: the Differentiable Mapping Network (DMN). The DMN constructs a spatially structured view-embedding map and uses it for subsequent visual localization with a particle filter. Since the DMN architecture is end-to-end differentiable, we can jointly learn the map representation and localization using gradient descent. We apply the DMN to sparse visual localization, where a robot needs to localize in a new environment with respect to a small number of images from known viewpoints. We evaluate the DMN using simulated environments and a challenging real-world Street View dataset. We find that the DMN learns effective map representations for visual localization. The benefit of spatial structure increases with larger environments, more viewpoints for mapping, and when training data is scarce. Project website: http://sites.google.com/view/differentiable-mapping

中文翻译：

---

可微映射网络：学习用于稀疏视觉定位的结构化地图表示

映射和定位，最好是从少量的观察，是机器人技术的基本任务。我们通过将空间结构（可微映射）和端到端学习结合到一种新颖的神经网络架构中来解决这些任务：可微映射网络 (DMN)。DMN 构建了一个空间结构的视图嵌入图，并将其用于后续的带有粒子过滤器的视觉定位。由于 DMN 架构是端到端可微的，我们可以使用梯度下降联合学习地图表示和定位。我们将 DMN 应用于稀疏视觉定位，其中机器人需要针对来自已知视点的少量图像在新环境中进行定位。我们使用模拟环境和具有挑战性的真实街景数据集来评估 DMN。我们发现 DMN 为视觉定位学习了有效的地图表示。空间结构的好处随着更大的环境、更多的制图视点以及训练数据稀缺而增加。项目网站：http://sites.google.com/view/differentiable-mapping