This paper proposes an energy management system (EMS) for the real-time operation of a pilot stochastic and dynamic microgrid on a university campus in Malta consisting of a diesel generator, photovoltaic panels, and batteries. The objective is to minimize the total daily operation costs, which include the degradation cost of batteries, the cost of energy bought from the main grid, the fuel cost of the diesel generator, and the emission cost. The optimization problem is modeled as a finite Markov decision process (MDP) by combining network and technical constraints, and Q-learning algorithm is adopted to solve the sequential decision subproblems. The proposed algorithm decomposes a multi-stage mixed-integer nonlinear programming (MINLP) problem into a series of single-stage problems so that each subproblem can be solved by using Bellman's equation. To prove the effectiveness of the proposed algorithm, three case studies are taken into consideration: ① minimizing the daily energy cost; ② minimizing the emission cost; ③ minimizing the daily energy cost and emission cost simultaneously. Moreover, each case is operated under different battery operation conditions to investigate the battery lifetime. Finally, performance comparisons are carried out with a conventional $Q$ -learning algorithm.

中文翻译：

---

基于多阶段混合整数非线性规划的微电网最优控制 $ Q $学习算法

本文提出了一种能源管理系统（EMS），用于在马耳他大学校园中实时运行随机和动态微电网试验，该系统由柴油发电机，光电板和电池组成。目的是使总的日常运营成本降至最低，包括电池的降级成本，从主电网购买的能源成本，柴油发电机的燃料成本以及排放成本。通过将网络和技术约束相结合，将优化问题建模为有限马尔可夫决策过程（MDP），并采用Q学习算法来解决顺序决策子问题。该算法将多阶段混合整数非线性规划（MINLP）问题分解为一系列单阶段问题，从而可以使用Bellman'解决每个子问题。s方程。为了证明该算法的有效性，考虑了以下三个案例：①最小化日常能源成本；②降低排放成本；③同时最小化日常能源成本和排放成本。而且，每种情况在不同的电池操作条件下进行操作以调查电池寿命。最后，使用常规方法进行性能比较$ Q $ 学习算法。