This paper presents a Q-learning-based target selection algorithm for spacecraft autonomous navigation using bearing observations of known visible targets. For the considered navigation system, the position and velocity of the spacecraft are estimated using an extended Kalman filter (EKF) with the measurements of inter-satellite line-of-sight (LOS) vectors obtained via an onboard star camera. This paper focuses on the selection of the appropriate target at each observation period for the star camera adaptively, such that the performance of the EKF is enhanced. To derive an effective algorithm, a Q-function is designed to select a proper observation region, while a U-function is introduced to rank the targets in the selected region. Both the Q-function and the U-function are constructed based on the sequence of innovations obtained from the EKF. The efficiency of the Q-learning-based target selection algorithm is illustrated via numerical simulations, which show that the presented algorithm outperforms the traditional target selection strategy based on a Cramer-Rao bound (CRB) in the case that the prior knowledge about the target location is inaccurate.

本文提出了一种基于Q学习的航天器自主导航目标选择算法，该算法使用已知可见目标的方位观测结果。对于考虑的导航系统，使用扩展的卡尔曼滤波器（EKF）估算航天器的位置和速度，并通过机载星型摄像头获得卫星间视线（LOS）向量的测量值。本文着重于自适应地选择在每个观察期为星型相机选择合适的目标，从而增强EKF的性能。为了获得有效的算法，设计了Q函数来选择合适的观察区域，同时引入U函数来对所选区域中的目标进行排序。Q函数和U函数都是基于从EKF获得的创新序列构建的。