Global buildings account for about 30% of the total energy consumption and carbon emission, raising severe energy and environmental concerns. Therefore, it is significant and urgent to develop novel smart building energy management (SBEM) technologies for the advance of energy efficient and green buildings. However, it is a nontrivial task due to the following challenges. First, it is generally difficult to develop an explicit building thermal dynamics model that is both accurate and efficient enough for building control. Second, there are many uncertain system parameters (e.g., renewable generation output, outdoor temperature, and the number of occupants). Third, there are many spatially and temporally coupled operational constraints. Fourth, building energy optimization problems can not be solved in real time by traditional methods when they have extremely large solution spaces. Fifthly, traditional building energy management methods have respective applicable premises, which means that they have low versatility when confronted with varying building environments. With the rapid development of Internet of Things technology and computation capability, artificial intelligence technology find its significant competence in control and optimization. As a general artificial intelligence technology, deep reinforcement learning (DRL) is promising to address the above challenges. Notably, the recent years have seen the surge of DRL for SBEM. However, there lacks a systematic overview of different DRL methods for SBEM. To fill the gap, this article provides a comprehensive review of DRL for SBEM from the perspective of system scale. In particular, we identify the existing unresolved issues and point out possible future research directions.

中文翻译：

---

智能建筑能源管理深度强化学习综述


全球建筑约占总能源消耗和碳排放的30%，引发了严重的能源和环境问题。因此，开发新型智能建筑能源管理（SBEM）技术对于推进节能和绿色建筑具有重要而紧迫的意义。然而，由于以下挑战，这是一项艰巨的任务。首先，通常很难开发一个对于建筑控制来说既准确又有效的明确的建筑热动力学模型。其次，存在许多不确定的系统参数（例如，可再生能源发电量、室外温度和居住人数）。第三，存在许多空间和时间耦合的操作限制。第四，当建筑能源优化问题具有极大的解空间时，传统方法无法实时求解。第五，传统的建筑能源管理方法有各自的适用前提，面对变化的建筑环境通用性较低。随着物联网技术和计算能力的快速发展，人工智能技术在控制和优化方面发挥了重要作用。作为一种通用人工智能技术，深度强化学习（DRL）有望解决上述挑战。值得注意的是，近年来 SBEM 的 DRL 激增。然而，缺乏对 SBEM 的不同 DRL 方法的系统概述。为了填补这一空白，本文从系统规模的角度对 SBEM 的 DRL 进行了全面的回顾。特别是，我们确定了现有的未解决的问题，并指出了未来可能的研究方向。