Governments are currently subsidizing growth in the electric car market and the associated infrastructure in order to accelerate the transition to more sustainable mobility. To avoid the grid overload that results from simultaneously charging too many electric vehicles, there is a need for smart charging coordination systems. In this paper, we propose a charging coordination system based on Reinforcement Learning using an artificial neural network as a function approximator. Taking into account the baseload present in the power grid, a central agent creates forward-looking, coordinated charging schedules for an electric vehicle fleet of any size. In contrast to optimization-based charging strategies, system dynamics such as future arrivals, departures, and energy consumption do not have to be known beforehand. We implement and compare a range of parameter variants that differ in terms of the reward function and prioritized experience. Subsequently, we use a case study to compare our Reinforcement Learning algorithm with several other charging strategies. The Reinforcement Learning-based charging coordination system is shown to perform very well. All electric vehicles have enough energy for their next trip on departure and charging is carried out almost exclusively during the load valleys at night. Compared with an uncontrolled charging strategy, the Reinforcement Learning algorithm reduces the variance of the total load by 65%. The performance of our Reinforcement Learning concept comes close to that of an optimization-based charging strategy. However, an optimization algorithm needs to know certain information beforehand, such as the vehicle’s departure time and its energy requirement on arriving at the charging station. Our novel Reinforcement Learning-based charging coordination system therefore offers a flexible, easily adaptable, and scalable approach for an electric vehicle fleet under realistic operating conditions.

政府目前正在为电动汽车市场和相关基础设施的增长提供补贴，以加速向更可持续的出行方式过渡。为了避免由于同时给太多的电动汽车充电而导致的电网过载，需要智能充电协调系统。在本文中，我们提出了一种基于强化学习的充电协调系统，该系统使用人工神经网络作为函数逼近器。考虑到电网中存在的基本负荷，中央代理会为任何规模的电动汽车车队创建前瞻性，协调一致的充电时间表。与基于优化的充电策略相反，系统动力学（例如未来到达，离开和能源消耗）不必事先知道。我们实现并比较了一系列在奖励功能和优先体验方面有所不同的参数变量。随后，我们使用一个案例研究将我们的强化学习算法与其他几种收费策略进行比较。基于强化学习的收费协调系统显示出很好的性能。所有电动汽车在出发时都有足够的能量供下次旅行使用，充电几乎只在夜间的负荷谷期间进行。与不受控制的充电策略相比，强化学习算法将总负载的方差降低了65％。我们的强化学习概念的性能接近于基于优化的收费策略。但是，优化算法需要事先知道某些信息，例如车辆的出发时间及其到达充电站的能量需求。因此，我们新颖的基于强化学习的充电协调系统为电动汽车车队在现实的运行条件下提供了一种灵活，易于适应和可扩展的方法。