This paper addresses a design scheme of a proportional-integral-derivative (PID) controller with a new adaptive updating rule based on reinforcement learning (RL) approach for nonlinear systems. A new design scheme that RL can be used to complement the conventional PID control technology is presented. In the proposed scheme, a single radial basis function (RBF) network is considered to calculate the control policy function of Actor and the value function of Critic simultaneously. Regarding the PID controller structure, the inputs of RBF network are system errors, the difference of output as well as the second-order difference of output, and they are composed of system states. The temporal difference (TD) error in the proposed scheme involves the reinforcement signal, the current and the previous stored value of the value function. The gradient descent method is adopted based on the TD error performance index, then, the updating rules can be yielded. Therefore, the network weights and the kernel function can be calculated in an adaptive way. Finally, the numerical simulations are conducted in nonlinear systems to illustrate the efficiency and robustness of the proposed scheme. © 2021 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

中文翻译：

---

强化学习PID控制器的设计

本文提出了一种比例积分微分 (PID) 控制器的设计方案，该控制器具有基于非线性系统的强化学习 (RL) 方法的新自适应更新规则。提出了一种新的设计方案，可以使用 RL 来补充传统的 PID 控制技术。在所提出的方案中，考虑了单个径向基函数（RBF）网络同时计算Actor的控制策略函数和Critic的值函数。在PID控制器结构中，RBF网络的输入是系统误差、输出的差值和输出的二阶差值，它们由系统状态组成。所提出方案中的时间差（TD）误差涉及增强信号、价值函数的当前和先前存储的值。基于TD误差性能指标采用梯度下降法，从而得出更新规则。因此，可以以自适应方式计算网络权重和核函数。最后，数值模拟在非线性系统中进行，以说明所提出方案的效率和鲁棒性。© 2021 日本电气工程师学会。由 Wiley Periodicals LLC 出版。