The increasing electric vehicles (EVs) at charging stations will impose great challenges on the conventional voltage control in distribution networks. In this article, a two-stage voltage control strategy based on deep reinforcement learning is proposed to mitigate voltage violations caused by the uncertainty of EVs and load. In the first stage, the charging demand of EVs is predicted based on trip chain theory and simulated by Monte Carlo simulation. The optimal power flow is then performed to determine the day-ahead dispatch of on-load tap changer and capacitor banks. In the second stage, the real-time voltage control problem is formulated as a Markov Game considering both reactive power control and vehicle to grid modes of EVs. The problem is solved by the deep deterministic policy gradient algorithm to develop a well-trained control strategy that can be implemented online. Moreover, a novel customized charging criterion is proposed to conduct the charging behavior of EVs and guarantee full charging at the departure time. The proposed approach is tested on the IEEE 33-bus and 123-bus distribution systems and comparative simulation results show the effectiveness in addressing voltage problems.

中文翻译：

---

采用强化学习方法的配电网电动汽车定制电压控制策略


充电站电动汽车（EV）的增加将对配电网的传统电压控制带来巨大挑战。本文提出了一种基于深度强化学习的两级电压控制策略，以减轻电动汽车和负载的不确定性引起的电压违规。第一阶段，基于出行链理论预测电动汽车的充电需求，并通过蒙特卡罗模拟进行模拟。然后执行最佳潮流以确定有载分接开关和电容器组的日前调度。在第二阶段，实时电压控制问题被表述为马尔可夫博弈，同时考虑电动汽车的无功功率控制和车辆到电网模式。该问题通过深度确定性策略梯度算法来解决，以开发可以在线实施的训练有素的控制策略。此外，提出了一种新颖的定制充电标准来指导电动汽车的充电行为并保证在出发时充满电。该方法在 IEEE 33 总线和 123 总线配电系统上进行了测试，比较仿真结果表明了解决电压问题的有效性。