This paper presents a model-free optimal approach based on reinforcement learning for solving the output regulation problem for discrete-time systems under disturbances. This problem is first broken down into two optimization problems: 1) a constrained static optimization problem is established to find the solution to the output regulator equations (i.e., the feedforward control input) and 2) a dynamic optimization problem is established to find the optimal feedback control input. Solving these optimization problems requires the knowledge of the system dynamics. To obviate this requirement, a model-free off-policy algorithm is presented to find the solution to the dynamic optimization problem using only measured data. Then, based on the solution to the dynamic optimization problem, a model-free approach is provided for the static optimization problem. It is shown that the proposed algorithm is insensitive to the probing noise added to the control input for satisfying the persistence of excitation condition. Simulation results are provided to verify the effectiveness of the proposed approach.

中文翻译：

---

使用强化学习的具有未知动力学的线性离散时间系统的最优输出调节。

本文提出了一种基于无损学习的无模型最优方法，用于解决离散时间系统在扰动下的输出调节问题。首先将该问题分解为两个优化问题：1）建立约束静态优化问题以找到输出调节器方程（即前馈控制输入）的解； 2）建立动态优化问题以找到最优的反馈控制输入。解决这些优化问题需要了解系统动力学。为了避免这种需求，提出了一种无模型的偏离策略算法，该算法仅使用测量数据即可找到动态优化问题的解决方案。然后，根据动态优化问题的解决方案，针对静态优化问题提供了一种无模型的方法。结果表明，所提出的算法对满足控制条件持续性的控制输入中的探测噪声不敏感。仿真结果提供了验证所提出方法有效性的方法。