Since hydrogen has many advantages (e.g., free pollution, extensive sources, convenient storage and transportation), hydrogen-based multi-energy systems (HMESs) have received wide attention. However, existing works on the optimal operation of HMESs neglect building thermal dynamics, which means that the flexibility of building thermal loads can not be utilized for reducing system operation cost. In this paper, we investigate an optimal operation problem of an HMES with the consideration of building thermal dynamics. Specifically, we first formulate an expected operational cost minimization problem related to an HMES. Due to the existence of uncertain parameters, inexplicit building thermal dynamics models, temporally coupled operational constraints related to three kinds of energy storage systems and indoor temperatures, as well as the coupling between electric energy subsystems and thermal energy subsystems, it is challenging to solve the formulated problem. To overcome the challenge, we reformulate the problem as a Markov game and propose an energy management algorithm to solve it based on multi-agent discrete actor-critic with rules (MADACR). Note that the proposed algorithm does not require any prior knowledge of uncertain parameters, parameter prediction, and explicit building thermal dynamics model. Simulation results based on real-world traces show the effectiveness of the proposed algorithm.

中文翻译：

---

基于深度强化学习的氢基建筑多能系统优化运行

由于氢具有无污染、来源广泛、储运方便等诸多优点，氢基多能源系统(HMESs)受到了广泛关注。然而，现有的关于 HMES 优化运行的工作忽略了建筑热力学，这意味着建筑热负荷的灵活性不能用于降低系统运行成本。在本文中，我们研究了考虑建筑热动力学的 HMES 的优化运行问题。具体来说，我们首先制定与 HMES 相关的预期运营成本最小化问题。由于存在不确定参数、不明确的建筑热力学模型、与三种储能系统和室内温度相关的时间耦合运行约束，以及电能子系统和热能子系统之间的耦合，解决公式化问题具有挑战性。为了克服这一挑战，我们将问题重新表述为马尔可夫博弈，并提出了一种能量管理算法来解决该问题，该算法基于带规则的多智能体离散演员-评论家（MADACR）。请注意，所提出的算法不需要任何不确定参数、参数预测和显式建筑热动力学模型的先验知识。基于真实世界轨迹的仿真结果表明了所提出算法的有效性。我们将问题重新表述为马尔可夫博弈，并提出了一种能量管理算法来解决它，该算法基于带规则的多代理离散演员-评论家（MADACR）。请注意，所提出的算法不需要任何不确定参数、参数预测和显式建筑热动力学模型的先验知识。基于真实世界轨迹的仿真结果表明了所提出算法的有效性。我们将问题重新表述为马尔可夫博弈，并提出了一种能量管理算法来解决它，该算法基于带规则的多代理离散演员-评论家（MADACR）。请注意，所提出的算法不需要任何不确定参数、参数预测和显式建筑热动力学模型的先验知识。基于真实世界轨迹的仿真结果表明了所提出算法的有效性。