This article discusses how techniques from reinforcement learning (RL) can be exploited to transition to a model-free and scalable wide-area oscillation damping control of power grids. We present two control architectures with distinct features. Performing full-dimensional RL control designs for any practical grid would require an unacceptably long learning time and result in a dense communication architecture. Our designs avoid the curse of dimensionality by employing ideas from model reduction. The first design exploits time-scale separation in the generator electro-mechanical dynamics arising from coherent clustering, and learns a controller using both electro-mechanical and non-electro-mechanical states while compensating for the error in incorporating the latter through the RL loop. The second design presents an output-feedback approach enabled by a neuro-adaptive observer using measurements of only the generator frequencies. The controller exhibits an adaptive behavior that updates the control gains whenever there is a notable change in the loads. Theoretical guarantees for closed-loop stability and performance are provided for both designs. Numerical simulations are shown for the IEEE 68-bus power system model.

中文翻译：

---

基于强化学习的广域阻尼控制的可扩展设计

本文讨论如何利用强化学习（RL）的技术过渡到电网的无模型且可扩展的广域振荡阻尼控制。我们提出了两种具有独特功能的控制体系结构。对任何实际的网格执行全尺寸RL控制设计将需要不可接受的长学习时间，并导致密集的通信体系结构。我们的设计通过采用模型简化中的思想来避免维数的诅咒。第一种设计利用了由相干聚类引起的发电机机电动力学中的时间标度分离，并学习了使用机电状态和非机电状态的控制器，同时补偿了通过RL回路合并后者的误差。第二种设计提出了一种输出反馈方法，该方法由神经适应性观察者使用仅对发生器频率的测量来实现。控制器表现出一种自适应行为，只要负载发生显着变化，该行为就会更新控制增益。两种设计均提供了闭环稳定性和性能的理论保证。显示了针对IEEE 68总线电源系统模型的数值仿真。