ABSTRACT

In order to obviate the requirement of drift dynamics in adaptive dynamic programming, integral reinforcement learning (IRL) has been proposed as an alternate formulation of Bellman equation. However control coupling dynamics is still needed to obtain closed-form expression of optimal control effort. In addition to this, initial stabilizing controller and two sets of neural networks (NN) (known as Actor-Critic) are required to implement IRL scheme. In order to remedy these, this paper presents a critic-only IRL controller coupled with an experience replay (ER)-based identifier to solve optimal tracking control problem for an unknown continuous-time systems under actuator constraints. The presented control architecture is shown to yield tighter residual sets for state tracking error and error in NN weights. The simulation results establish the efficacy of the presented control scheme on continuous-time systems.

摘要

为了避免自适应动态规划中对漂移动力学的要求，积分强化学习（IRL）已被提出作为贝尔曼方程的替代公式。然而，仍然需要控制耦合动力学来获得最佳控制努力的封闭形式。除此之外，需要初始稳定控制器和两组神经网络 (NN)（称为 Actor-Critic）来实现 IRL 方案。为了解决这些问题，本文提出了一个仅批评者的 IRL 控制器与基于经验重放 (ER) 的标识符相结合，以解决执行器约束下未知连续时间系统的最优跟踪控制问题。所提出的控制架构显示为状态跟踪误差和 NN 权重误差产生更紧密的残差集。