Dynamics modeling in outdoor and unstructured environments is difficult because different elements in the environment interact with the robot in ways that can be hard to predict. Leveraging multiple sensors to perceive maximal information about the robot's environment is thus crucial when building a model to perform predictions about the robot's dynamics with the goal of doing motion planning. We design a model capable of long-horizon motion predictions, leveraging vision, lidar and proprioception, which is robust to arbitrarily missing modalities at test time. We demonstrate in simulation that our model is able to leverage vision to predict traction changes. We then test our model using a real-world challenging dataset of a robot navigating through a forest, performing predictions in trajectories unseen during training. We try different modality combinations at test time and show that, while our model performs best when all modalities are present, it is still able to perform better than the baseline even when receiving only raw vision input and no proprioception, as well as when only receiving proprioception. Overall, our study demonstrates the importance of leveraging multiple sensors when doing dynamics modeling in outdoor conditions.

中文翻译：

---

越野自动驾驶汽车的多峰动力学建模

在室外和非结构化环境中进行动力学建模非常困难，因为环境中的不同元素会以难以预测的方式与机器人交互。因此，在构建模型以执行有关运动计划的目标以执行有关机器人动态的预测时，利用多个传感器感知有关机器人环境的最大信息至关重要。我们设计了一个模型，该模型能够进行长距离运动预测，并利用视觉，激光雷达和本体感受，这对于在测试时任意丢失模式是很可靠的。我们在仿真中证明我们的模型能够利用视觉预测牵引力的变化。然后，我们使用机器人在森林中导航，对训练期间看不见的轨迹进行预测的真实世界挑战性数据集来测试模型。我们在测试时尝试了不同的模态组合，结果表明，尽管我们的模型在所有模态都存在的情况下表现最佳，但即使仅接收原始视觉输入而没有本体感觉，以及仅接收原始视觉输入时，模型仍然能够比基线表现更好本体感受。总体而言，我们的研究表明在室外条件下进行动力学建模时，利用多个传感器的重要性。