Abstract A real-time, data-driven electric vehicle (EVs) routing optimization to achieve energy consumption minimization is proposed in this work. The proposed framework utilizes the concept of Double Deep Q-learning Network (DDQN) in learning the maximum travel policy of the EV as an agent. The policy model is trained to estimate the agent's optimal action per the obtained reward signals and Q-values, representing the feasible routing options. The agent's energy requirement on the road is assessed following Markov Chain Model (MCM), with Markov's unit step represented as the average energy consumption that takes into consideration the different driving patterns, agent's surrounding environment, road conditions, and applicable restrictions. The framework offers a better exploration strategy, continuous learning ability, and the adoption of individual routing preferences. A real-time simulation in the python environment that considered real-life driving data from Google's API platform is performed. Results obtained for two geographically different drives show that the proposed energy consumption minimization framework reduced the energy utilization of the EVs to reach its intended destination by 5.89% and 11.82%, compared with Google's proposed routes originally. Both drives started at 4.30 PM on April 25th, 2019, in Los Angeles, California, and Miami, Florida, to reach EV's charging stations that are located six miles away from both of the starting locations.

中文翻译：

---

使用深度强化学习和马尔可夫链模型实现电动汽车能耗最小化的实时元数据驱动路由优化

摘要 在这项工作中提出了一种实时的、数据驱动的电动汽车 (EV) 路由优化以实现能耗最小化。所提出的框架利用双深度 Q 学习网络 (DDQN) 的概念来学习电动汽车作为代理的最大旅行策略。训练策略模型以根据获得的奖励信号和 Q 值估计代理的最佳动作，代表可行的路由选项。代理在道路上的能量需求是根据马尔可夫链模型 (MCM) 进行评估的，马尔可夫的单位步长表示为考虑不同驾驶模式、代理周围环境、道路条件和适用限制的平均能耗。该框架提供了更好的探索策略、持续学习能力、以及采用个人路由偏好。在 python 环境中进行实时模拟，考虑来自 Google API 平台的真实驾驶数据。两个地理不同的驱动器获得的结果表明，与谷歌最初提出的路线相比，拟议的能源消耗最小化框架将电动汽车的能源利用率降低了 5.89% 和 11.82%。两个驱动器均于 2019 年 4 月 25 日下午 4 点 30 分开始，分别在加利福尼亚州洛杉矶和佛罗里达州迈阿密到达距离两个起始地点 6 英里的电动汽车充电站。两个地理不同的驱动器获得的结果表明，与谷歌最初提出的路线相比，拟议的能源消耗最小化框架将电动汽车的能源利用率降低了 5.89% 和 11.82%。两个驱动器均于 2019 年 4 月 25 日下午 4 点 30 分开始，分别在加利福尼亚州洛杉矶和佛罗里达州迈阿密到达距离两个起始地点 6 英里的电动汽车充电站。两个地理不同的驱动器获得的结果表明，与谷歌最初提出的路线相比，拟议的能源消耗最小化框架将电动汽车的能源利用率降低了 5.89% 和 11.82%。两个驱动器均于 2019 年 4 月 25 日下午 4 点 30 分开始，分别在加利福尼亚州洛杉矶和佛罗里达州迈阿密到达距离两个起始地点 6 英里的电动汽车充电站。