The extended Kalman filter (EKF) is a widely used method in navigation applications. The EKF suffers from noise covariance uncertainty, potentially causing it to perform poorly in practice. This paper attempts to suppress the unfavorable effect of the noise covariance uncertainty to the EKF in the framework of reinforcement learning. The proposed state estimation algorithm combines the EKF and a Q-learning method, where a covariance adaptation strategy is designed based on the Q-values, leading to a gradual improvement in the estimation performance. The resultant algorithm is called the Q-learning extended Kalman filter (QLEKF), which is less sensitive to the noise covariance uncertainty. To evaluate the estimation error behavior in nonlinear uncertain systems, the stability of the filtering algorithm is investigated. A numerical simulation for spacecraft autonomous navigation is implemented to demonstrate the efficiency of the QLEKF. It is shown that the presented algorithm outperforms a traditional EKF and an adaptive extended Kalman filter (AEKF) in estimation accuracy.

中文翻译：

---

扩展卡尔曼滤波器中噪声协方差自适应的 Q 学习

扩展卡尔曼滤波器 (EKF) 是导航应用中广泛使用的方法。EKF 受到噪声协方差不确定性的影响，可能导致它在实践中表现不佳。本文试图在强化学习的框架内抑制噪声协方差不确定性对EKF的不利影响。所提出的状态估计算法结合了 EKF 和 Q 学习方法，其中基于 Q 值设计了协方差自适应策略，导致估计性能的逐步提高。由此产生的算法称为 Q 学习扩展卡尔曼滤波器 (QLEKF)，它对噪声协方差不确定性不太敏感。为了评估非线性不确定系统中的估计误差行为，研究了滤波算法的稳定性。对航天器自主导航进行了数值模拟，以证明 QLEKF 的效率。结果表明，所提出的算法在估计精度方面优于传统的 EKF 和自适应扩展卡尔曼滤波器 (AEKF)。