This paper presents an EKF-based self-attitude estimation with a DNN (deep neural network) learning landscape information. The method integrates gyroscopic angular velocity and DNN inference in the EKF. The DNN predicts a gravity vector in a camera frame. The input of the network is a camera image, the outputs are a mean vector and a covariance matrix of the gravity. It is trained and validated with a dataset of images and corresponded gravity vectors. The dataset is collected in a flight simulator because we can easily obtain various gravity vectors, although the method is not only for UAVs. Using a simulator breaks the limitation of amount of collecting data with ground truth. The validation shows the network can predict the gravity vector from only a single shot image. It also shows that the covariance matrix expresses the uncertainty of the inference. The covariance matrix is used for integrating the inference in the EKF. Flight data of a drone is also recorded in the simulator, and the EKF-based method is tested with it. It shows the method suppresses accumulative error by integrating the network outputs.

中文翻译：

---

基于EKF的具有DNN学习景观信息的自我态度估计

本文提出了一种基于EKF的自我态度估计方法，该方法利用DNN（深度神经网络）学习景观信息。该方法将陀螺仪角速度和DNN推论整合到EKF中。DNN预测摄像机帧中的重力矢量。网络的输入是摄像机图像，输出是重力的均值向量和协方差矩阵。使用图像数据集和相应的重力矢量对它进行训练和验证。该数据集是在飞行模拟器中收集的，因为我们可以轻松获得各种重力矢量，尽管该方法不仅适用于无人机。使用模拟器打破了具有地面真实性的数据收集限制。验证表明，网络只能从单个拍摄的图像中预测重力矢量。它还表明，协方差矩阵表示推断的不确定性。协方差矩阵用于将推论整合到EKF中。无人机的飞行数据也记录在模拟器中，并以此为基础对基于EKF的方法进行测试。它显示了通过积分网络输出来抑制累积误差的方法。