The drone navigation requires the comprehensive understanding of both visual and geometric information in the 3D world. In this paper, we present a Visual-Geometric Fusion Network (VGF-Net), a deep network for the fusion analysis of visual/geometric data and the construction of 2.5D height maps for simultaneous drone navigation in novel environments. Given an initial rough height map and a sequence of RGB images, our VGF-Net extracts the visual information of the scene, along with a sparse set of 3D keypoints that capture the geometric relationship between objects in the scene. Driven by the data, VGF-Net adaptively fuses visual and geometric information, forming a unified Visual-Geometric Representation. This representation is fed to a new Directional Attention Model (DAM), which helps enhance the visual-geometric object relationship and propagates the informative data to dynamically refine the height map and the corresponding keypoints. An entire end-to-end information fusion and mapping system is formed, demonstrating remarkable robustness and high accuracy on the autonomous drone navigation across complex indoor and large-scale outdoor scenes.

无人机导航需要全面了解3D世界中的视觉和几何信息。在本文中，我们提出了视觉几何融合网络（VGF-Net），这是一种用于视觉/几何数据融合分析的深度网络，并构建了2.5D高度图，以便在新型环境中同时进行无人机导航。给定一个初始的粗略高度图和一系列RGB图像，我们的VGF-Net提取场景的视觉信息以及稀疏的3D关键点集，以捕获场景中对象之间的几何关系。在数据的驱动下，VGF-Net自适应融合视觉和几何信息，形成统一的视觉几何表示。该表示被馈送到新的定向注意模型（DAM），有助于增强视觉几何对象之间的关系，并传播信息性数据以动态优化高度图和相应的关键点。形成了一个完整的端到端信息融合和制图系统，展示了在复杂的室内和大型室外场景中自主无人机导航所具有的出色的鲁棒性和高精度。