This article proposes an observer-based reinforcement learning (RL) control approach to address the optimal attitude-tracking problem and application for hypersonic vehicles in the reentry phase. Due to the unknown uncertainty and nonlinearity caused by parameter perturbation and external disturbance, accurate model information of hypersonic vehicles in the reentry phase is generally unavailable. For this reason, a novel synchronous estimation is proposed to construct a composite observer for hypersonic vehicles, which consists of a neural-network (NN)-based Luenberger-type observer and a synchronous disturbance observer. This solves the identification problem of nonlinear dynamics in the reference control and realizes the estimation of the system state when unknown nonlinear dynamics and unknown disturbance exist at the same time. By synthesizing the information from the composite observer, an RL tracking controller is developed to solve the optimal attitude-tracking control problem. To improve the convergence performance of critic network weights, concurrent learning is employed to replace the traditional persistent excitation condition with a historical experience replay manner. In addition, this article proves that the weight estimation error is bounded when the learning rate satisfies the given sufficient condition. Finally, the numerical simulation demonstrates the effectiveness and superiority of the proposed approaches to attitude-tracking control systems for hypersonic vehicles.

中文翻译：

---

高超声速飞行器基于复合观察者的强化学习最优姿态跟踪控制


本文提出了一种基于观察者的强化学习（RL）控制方法来解决高超声速飞行器再入阶段的最佳姿态跟踪问题和应用。由于参数扰动和外部扰动带来的未知不确定性和非线性，高超声速飞行器再入阶段的准确模型信息通常无法获得。为此，提出了一种新颖的同步估计来构建高超声速飞行器复合观测器，该观测器由基于神经网络（NN）的Luenberger型观测器和同步扰动观测器组成。解决了参考控制中非线性动力学的辨识问题，实现了未知非线性动力学和未知扰动同时存在时系统状态的估计。通过综合来自复合观测器的信息，开发了 RL 跟踪控制器来解决最优姿态跟踪控制问题。为了提高批评网络权重的收敛性能，采用并发学习以历史经验回放方式代替传统的持续激励条件。另外，本文证明了当学习率满足给定的充分条件时，权重估计误差是有界的。最后，数值模拟证明了所提出的高超声速飞行器姿态跟踪控制系统方法的有效性和优越性。