As the penetration rate of new energies, energy storage devices and electric vehicles increases continuously in power grid, power grid faces strong random disturbances, as well as the issues of downfall in frequency control ability induced by the factors such as system inertia reduction and frequency control under-capacity in traditional power units. Therefore, an algorithm of automatic generation control with DQ( σ , λ) is proposed for the multi-area interconnected power grid. By integrating the eligibility trace, DQ( σ , λ) adopts the principle of double Q-learning to unify double Q(λ) and double Expected-Sarsa(λ), thereby the over-estimation of Q in the algorithm and the phenomenon of over-fitting can be avoided. The simulations are provided to demonstrate the effectiveness of the proposed algorithm, where the improved IEEE standard two-area load-frequency control model and the model of the multi-area interconnected power grid based on Central China Power Grid are adopted. The results show that the phenomena of random disturbances and frequency instability can be eliminated in multi-area interconnected power grid. The convergence is faster and the dynamic performance is better in the proposed algorithm compared with the traditional algorithms.

中文翻译：

---

多区域互联电网自动发电控制的多步统一强化学习方法

随着新能源，储能装置和电动汽车在电网中的渗透率不断提高，电网面临强烈的随机干扰，以及由于系统惯性降低和频率控制等因素引起的频率控制能力下降的问题。传统功率单元的容量不足。因此，采用DQ（ σ ，λ）被提出用于多区域互连电网。通过整合资格跟踪DQ（ σ ，λ）采用双重Q学习的原理将双重Q（λ）和双重Expected-Sarsa（λ）统一起来，从而避免了算法中Q的过高估计和过度拟合现象。通过仿真验证了所提算法的有效性，采用了改进的IEEE标准两区负荷频率控制模型和基于华中电网的多区互联电网模型。结果表明，在多区域互联电网中，可以消除随机干扰和频率不稳定性现象。与传统算法相比，该算法收敛速度更快，动态性能更好。