Robot morphology evolution for automated HVAC system inspections using graph heuristic search and reinforcement learning
Introduction
HVAC system inspections are essential for building energy saving and greenhouse gas (GHG) emission reduction. According to related studies [1,2], about 70% of electricity consumption in the United States originates from the building sector, with HVAC systems being the largest consumer. HVAC inspection aims to identify and fix unnecessary energy loss including leaks or blockages in buildings as early as possible, given that damage, blocking, and aging of ducts are reasons for more than 30% of energy loss [3].
Currently, most HVAC system inspections are carried out manually by experienced engineers every one to two years, according to the National Air Duct Cleaners Association (NADCA) standard [4]. This manual approach is time-consuming, labor-intensive, and error-prone [5]. Recent studies and industry standards [[6], [7], [8]] focus on building highly sensed HVAC systems to reduce the need for manual inspections. These intelligent systems can achieve real-time system monitoring, alarming, and control based on pre-defined rules or learning-based control strategies [[9], [10], [11]]. However, the installation of such a complex sensor network is intrusive and expensive for existing buildings. In addition, sensors need regular maintenance to avoid false readings and failures, which can result in ignored alarms in practice due to constant false alarms [12].
Recent studies [13,14] explored the possibility of using robots to conduct HVAC inspections, including fire curtains [15], temperature distributions [16], and damages [17]. Robot inspection works for both new and existing buildings, avoiding regular maintenance of sensor networks. Compared with manual inspections, robots are efficient and precise [17]. During a regular HVAC inspection over ceilings, robots need to navigate across ducts, avoid obstacles, and inspect the HVAC system. Due to the navigational complexity of ceiling environments, inspection robots [[18], [19], [20]] are carefully designed by experts to ensure mobility. This process requires technical knowledge of robot design and control dynamics, and the process can be time-consuming when considering the entire procedure from robot design to fabrication [21]. Even so, the manually designed robot can still fail to complete tasks due to the lack of adaptation to diverse HVAC environments. As shown in Fig. 1, the steel bar barriers (Fig.1 a) on the surface and the existence of gaps (Fig. 1b) result in challenges to ground robots such as unmanned ground vehicles (UGVs). The limitation caused by ducts and chains (Fig.1 a) makes the movement of the unmanned aerial vehicle (UAV) dangerous and flexuous. For example, the UGV proposed in [13] cannot navigate in the aforementioned T-bar ceiling environment due to the reliance on wheels. The UAV designed in [18] can only operate in barrier-free ducts during non-operational hours of HVAC systems and also cannot successfully navigate in complex environments shown in Fig. 1 since the size of the UAV is larger than the limitations caused by ducts and chains. These shortcomings severely hamper the capability and adoption of robotic-based solutions for HVAC inspections. Therefore, an automatic robot design and evolution approach that can efficiently generate appropriate robot forms in a specific ceiling environment is needed.
Robot evolution refers to the self-improvement of robots through interactions with the environments while discovering useful properties from these interactions [[22], [23], [24]]. The evolution of robot morphology includes two aspects: optimizing parameters of a given shape and optimizing the shape in a limited morphological search space [25]. For HVAC inspection robots to be adaptive in different ceiling environments, the evolution of the robot shapes is necessary. Another barrier faced by robot-based HVAC system inspection stems from the fabrication of robots. For specially designed robots, production can be expensive if new molds are designed for every robot. An alternative idea is to assemble robots from primary elements (e.g., motors, servos, sensors, and body components) like LEGO bricks. Since primary elements can be fabricated in batches, the production cost can be diminished.
To address the above problems, we propose a framework where robots can evolve their morphologies for HVAC system inspection tasks (Fig. 2). This framework includes two modules. In the first module, a graph heuristic search method is used to evolve robot morphologies for specific tasks (e.g., navigation, obstacle avoidance). We design complex terrains for these tasks to evolve a robust robot form, which is composed of several pre-defined types of primary elements. The second module includes control of the evolved robot in ceiling environments. To simulate inspection activities in buildings, the robot is trained using reinforcement learning for navigation in ceiling environment models generated from Building Information Modeling (BIM) authoring tools (e.g., Revit) to represent realistic HVAC systems designs. The main contributions are:
- •
We proposed a robot morphology evolution approach that aims to evolve adaptive robots for HVAC inspection tasks in a variety of ceiling designs.
- •
We proposed a learning-based control approach, in which the BIMs of HVAC systems can be applied as general robot training environments for the training of robot control.
- •
We tested the proposed framework by evolving robot morphologies in diverse and realistic ceiling environments adopted from BIMs of real-world buildings and achieved flexible and robust robot control in complex environments using reinforcement learning.
Section snippets
Evolutionary robotics
Evolutionary robotics focus on two main aspects of research, automatic robot control inspired by evolution, and automated design of robot morphologies so they can adapt to complex environments. In robot control, early research [26,27] focused on evolving dynamical network controllers for simple robots, in which the configuration (e.g., morphologies, sensors, and motors) of robots was given by experts, and the components (e.g., legs) were controlled by artificial neurons [28]. The neurons
Graph grammar based robot morphology evolution module
In this section, we introduce the robot morphology evolution module, where we aim to allow the robots to evolve to the best morphology for inspection tasks using graph heuristic search in a robot simulation engine [35]. The terrains are designed based on several complex environments in realistic HVAC systems. Graph heuristic search consists of three phases: the design phase, the evaluation phase, and the learning phase, as shown in Fig. 3.
During the design phase, the ϵ-greedy approach is
Learning-based control in virtual ceiling environments
This module aims to test the performance and leverage the flexibility of reinforcement learning control to help the “best” robot generated in the previous module navigate in diverse ceiling environments. To achieve this goal, this module first uses BIM models of HVAC systems as inputs and transforms the BIM model into a robot simulation engine (i.e., Isaac Sim), and then applies reinforcement learning algorithms to train the robot generated in terrain 4 (This terrain includes the most
Conclusions and future work
This paper presents a framework where robots can evolve their morphologies by interacting with surrounding HVAC environments, in addressing current problems of robot-based HVAC system inspections. The presented framework includes two modules. The first module utilizes the graph heuristic search method to evolve robot morphologies according to the designed terrains, and the second module examines the best robot's ability to navigate in realistic ceiling environments. The main contributions are
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
We acknowledge the support of the Natural Sciences and Engineering Research Council of Canada (NSERC). Kangkang acknowledges the support of the CSC scholarship from the Chinese Scholarship Council.
References (62)
- et al.
A data-driven approach to extract operational signatures of HVAC systems and analyze impact on electricity consumption
Appl. Energy
(2019) - et al.
Formalized control logic fault definition with ontological reasoning for air handling units
Autom. Constr.
(2021) - et al.
Data-driven offline reinforcement learning for HVAC-systems
Energy
(2022) - et al.
Sensor data validation and fault diagnosis using Auto-Associative Neural Network for HVAC systems
J. Build. Eng.
(2020) - et al.
Estimating surface temperature from thermal imagery of buildings for accurate thermal transmittance (U-value): A machine learning perspective
J. Build. Eng.
(2020) - et al.
PyroShield - A HVAC fire curtain testing robot
Autom. Constr.
(2017) - et al.
Application of evolved locomotion controllers to a hexapod robot
Robot. Auton. Syst.
(1996) - et al.
A survey of automation-enabled human-in-the-loop systems for infrastructure visual inspection
Autom. Constr.
(2019) - et al.
Review of Unmanned Aerial System (UAS) applications in the built environment: Towards automated building inspection procedures using drones
Autom. Constr.
(2018) - et al.
Present and Future Energy Consumption of Buildings: Challenges and Opportunities towards Decarbonisation
(2021)