In this paper, the mechanism of energy storage (ES)-based power oscillation damping is derived by the small signal and the classical electric torque method. And then, by cooperating PI with an integral reduction loop, a controller is designed to form a novel PI-IR controller to guarantee that the energy variation of ES damper is zero at the end of one oscillation. Furthermore, for the controller parameters tuning, the conventional model-based methods require a forecasting model on the uncertainty disturbances. To this end, this problem is formulated as a finite Markov decision process with unknown transition probability, and introduce a deep reinforcement learning (DRL) based model-free agent, the soft actor-critic, to obtain the real-time optimal control strategy. After numerous training, the well-trained agent can act as an experienced decision maker to provide the real-time near-optimal parameters setting for PI-IR control under different operating conditions. Time-domain and eigenvalue analysis results demonstrate the effectiveness of the proposed PI-IR controller and the superiority of the employed DRL based model-free method.

中文翻译：

---

基于储能的电网电力振荡阻尼的机理分析和实时控制：一种软 Actor-Critic 方法

本文通过小信号和经典电转矩方法推导出基于储能（ES）的功率振荡阻尼机制。然后，通过将 PI 与积分减少回路配合，设计控制器形成新颖的 PI-IR 控制器，以保证 ES 阻尼器在一次振荡结束时的能量变化为零。此外，对于控制器参数整定，传统的基于模型的方法需要对不确定性扰动的预测模型。为此，该问题被表述为具有未知转移概率的有限马尔可夫决策过程，并引入基于深度强化学习 (DRL) 的无模型代理、软actor-critic，以获得实时最优控制策略。经过无数次的训练，训练有素的代理可以充当经验丰富的决策者，为不同操作条件下的 PI-IR 控制提供实时接近最佳的参数设置。时域和特征值分析结果证明了所提出的 PI-IR 控制器的有效性和基于 DRL 的无模型方法的优越性。