This study builds a two-level energy management strategy framework for decentralized autonomy of microgrids and optimal coordinated operation of a multi-microgrid system. To reduce the operational cost of a combined cooling, heating and power multi-microgrid system with uncertain information and to improve the accuracy of load demand prediction, a hybrid metaheuristic multi-layer reinforcement learning algorithm is proposed for the framework of a multi-microgrid system. The proposed method is composed of a weighted delayed deep deterministic policy gradient algorithm, power adjustment network, and a genetic algorithm. At the first level, the microgrid operators utilize weighted delayed deep deterministic policy gradient algorithm with power adjustment network to optimize their operational strategies; at the second level, the distribution system operator employs a genetic algorithm to adjust its operational decision-making for minimizing the operational cost of the multi-microgrid system, reducing the peak-to-average ratios and power fluctuations at the points of common coupling. The data privacy of the parties in the multi-microgrid system is protected as each entity in the system does not have direct access to other entities’ information during the decision-making process. Numerical simulation results show that the proposed weighted delayed deep deterministic policy gradient algorithm with power adjustment network can rapidly obtain high-quality deterministic approximate optimal solution for economic dispatch of the microgrid. The framework proposed in this study achieves decentralized autonomy of microgrids, reduces the operational cost of the multi-microgrid system with incomplete or uncertain information, and indirectly improves the accuracy of load demands prediction at the points of common coupling.

本研究构建了微电网分散自治和多微电网系统优化协调运行的两级能源管理策略框架。为降低信息不确定的冷热电联供多微电网系统的运行成本，提高负荷需求预测的准确性，提出一种混合元启发式多层强化学习算法，适用于多微电网系统框架。 . 该方法由加权延迟深度确定性策略梯度算法、功率调整网络和遗传算法组成。在第一级，微电网运营商利用带功率调整网络的加权延迟深度确定性策略梯度算法来优化其运营策略；在第二层，配电系统运营商采用遗传算法调整其运营决策，以最大限度地降低多微电网系统的运营成本，减少公共耦合点的峰均比和功率波动。多微电网系统中各方的数据隐私受到保护，因为系统中的每个实体在决策过程中都不能直接访问其他实体的信息。数值仿真结果表明，所提出的带功率调整网络的加权延迟深度确定性策略梯度算法能够快速获得微电网经济调度的高质量确定性近似最优解。本研究提出的框架实现了微电网的分散自治，