Profit maximization of electric vehicle charging station (EVCS) operation yields an increasing investment for the deployment of EVCSs, thereby increasing the penetration of electric vehicles (EVs) and supporting high-quality charging service to EV users. However, existing model-based approaches for profit maximization of EVCSs may exhibit poor performance owing to the underutilization of massive data and inaccurate modeling of EVCS operation in a dynamic environment. Furthermore, the existing approaches can be vulnerable to adversaries that abuse private EVCS operation data for malicious purposes. To resolve these limitations, we propose a privacy-preserving distributed deep reinforcement learning (DRL) framework that maximizes the profits of multiple smart EVCSs integrated with photovoltaic and energy storage systems under a dynamic pricing strategy. In the proposed framework, DRL agents using the soft actor–critic method determine the schedules of the profitable selling price and charging/discharging energy for EVCSs. To preserve the privacy of EVCS operation data, a federated reinforcement learning method is adopted in which only the local and global neural network models of the DRL agents are exchanged between the DRL agents at the EVCSs and the global agent at the central server without sharing EVCS data. Numerical examples demonstrate the effectiveness of the proposed approach in terms of convergence of the training curve for the DRL agent, adaptive profitable selling price, energy charging and discharging, sensitivity of the selling price factor, and varying weather conditions.

电动汽车充电站（EVCS）运营的利润最大化会增加对EVCS部署的投资，从而提高电动汽车（EV）的普及率，并为电动汽车用户提供高质量的充电服务。然而，由于海量数据未充分利用和动态环境中 EVCS 操作的不准确建模，现有的基于模型的 EVCS 利润最大化方法可能表现不佳。此外，现有方法可能容易受到出于恶意目的而滥用私有 EVCS 操作数据的攻击者的攻击。为了解决这些限制，我们提出了一种隐私保护分布式深度强化学习 (DRL) 框架，该框架可以在动态定价策略下最大化与光伏和储能系统集成的多个智能 EVCS 的利润。在提议的框架中，DRL 代理使用软演员-评论家方法确定 EVCS 的盈利销售价格和充电/放电能量的时间表。为了保护 EVCS 操作数据的隐私，采用联合强化学习方法，其中仅在 EVCS 的 DRL 代理和中央服务器的全局代理之间交换 DRL 代理的局部和全局神经网络模型，而不共享 EVCS数据。数值例子证明了所提出的方法在 DRL 代理训练曲线收敛方面的有效性，