Machine learning related research in transient control has drawn considerable attention with the rapid increase in data measurement from power grids. Two key components, the control algorithm and system structure, work together to determine the control performance. The design of control laws, the selection of phase measurement units, the allocation of power resources, and the scheduling of communication topology in limited cyber-physical resources need to be considered. Many existing scheduling or planning schemes specialized for control structure are designed based on various linearized analytical models or the optimization of steady states. However, the transient dynamics of power grids are nonlinear and parts of these dynamics are usually unknown. Linearized analytical models cannot represent the transient dynamics of power grids with large disturbances. This article proposes a sparse neural network based reinforcement learning scheme to optimize the control system structure for the transient stability enhancement of power grids with energy storage systems. One adjustable group sparse weight matrix is introduced to formulate both control structure and actor–critic networks. This strategy enables the proposed scheme to simultaneously schedule the control system structure and design the control laws by online learning without solving any combinational optimization problems or requiring any linearized analytical models. The sufficient conditions of learning stability, control stability, and group sparsity are thoroughly studied by mathematical analysis. The proposed scheme is simulated on an IEEE 118-bus test system for verification. The simulation results confirm the feasibility, advantages, and adaptability of the proposed method.

中文翻译：

---

稀疏促进强化学习储能系统暂态控制的结构调度


随着电网数据测量的快速增加，瞬态控制中的机器学习相关研究引起了人们的广泛关注。控制算法和系统结构这两个关键组件共同决定控制性能。需要考虑控制律的设计、相位测量单元的选择、功率资源的分配以及有限信息物理资源下的通信拓扑的调度。许多现有的专用于控制结构的调度或规划方案都是基于各种线性化分析模型或稳态优化来设计的。然而，电网的瞬态动态是非线性的，并且这些动态的一部分通常是未知的。线性化分析模型无法代表大扰动电网的暂态动态。本文提出了一种基于稀疏神经网络的强化学习方案来优化控制系统结构，以增强储能系统的暂态稳定性。引入一种可调整的组稀疏权重矩阵来制定控制结构和行动者-批评者网络。该策略使得所提出的方案能够通过在线学习同时调度控制系统结构并设计控制律，而无需解决任何组合优化问题或需要任何线性化分析模型。通过数学分析深入研究了学习稳定性、控制稳定性和群体稀疏性的充分条件。该方案在IEEE 118总线测试系统上进行了仿真验证。仿真结果验证了该方法的可行性、优点和适应性。