Visual odometry shows excellent performance in a wide range of environments. However, in visually-denied scenarios (e.g. heavy smoke or darkness), pose estimates degrade or even fail. Thermal cameras are commonly used for perception and inspection when the environment has low visibility. However, their use in odometry estimation is hampered by the lack of robust visual features. In part, this is as a result of the sensor measuring the ambient temperature profile rather than scene appearance and geometry. To overcome this issue, we propose a Deep Neural Network model for thermal-inertial odometry (DeepTIO) by incorporating a visual hallucination network to provide the thermal network with complementary information. The hallucination network is taught to predict fake visual features from thermal images by using Huber loss. We also employ selective fusion to attentively fuse the features from three different modalities, i.e thermal, hallucination, and inertial features. Extensive experiments are performed in hand-held and mobile robot data in benign and smoke-filled environments, showing the efficacy of the proposed model.

中文翻译：

---

DeepTIO：具有视觉幻觉的深度热惯性里程计

视觉里程计在各种环境中都表现出卓越的性能。然而，在视觉上拒绝的情况下（例如浓烟或黑暗），姿势估计会降低甚至失败。当环境能见度低时，热像仪通常用于感知和检查。然而，由于缺乏强大的视觉特征，它们在里程计估计中的使用受到阻碍。部分原因是传感器测量的是环境温度曲线，而不是场景外观和几何形状。为了克服这个问题，我们提出了一种用于热惯性里程计 (DeepTIO) 的深度神经网络模型，通过结合视觉幻觉网络为热网络提供补充信息。幻觉网络被教导通过使用 Huber 损失从热图像中预测假视觉特征。我们还采用选择性融合来仔细融合来自三种不同模式的特征，即热、幻觉和惯性特征。在良性和充满烟雾的环境中对手持和移动机器人数据进行了大量实验，显示了所提出模型的有效性。