Elsevier

Automation in Construction

Volume 153, September 2023, 104956
Automation in Construction

Robot morphology evolution for automated HVAC system inspections using graph heuristic search and reinforcement learning

https://doi.org/10.1016/j.autcon.2023.104956Get rights and content

Highlights

  • Proposed robot grammars for automated evolution of robot morphologies.

  • Created a graph neural network based heuristic search to predict robot performance.

  • Evaluated different robot morphologies using model predictive control.

  • Developed ceiling environments in a physics engine to train robot control agents.

  • Tested robot performance in realistic environments using reinforcement learning.

Abstract

The building sector consumes more than 70% of the electricity produced in the U.S., with Heating Ventilation and Air Conditioning (HVAC) systems accounting for half of the electricity consumption. Leaks in HVAC systems have a significant impact on the energy efficiency of buildings, resulting in up to 33% of energy loss. However, the current manual approach to inspection is time-consuming and reactive, leaving room for automation. This paper presents a framework to automatically evolve robot morphology without requiring human intervention to suite any given HVAC and ceiling design. Robot morphologies are optimized using graph heuristic search based on tasks and environment designs, followed by testing of navigation abilities of the best-evolved robot in diverse ceiling environments using reinforcement learning. Tests conducted in robot simulation tools, utilizing realistic HVAC designs retrieved from Building Information Models (BIMs), demonstrate that effortless navigation in complex ceiling environments can be achieved by the evolved robots.

Introduction

HVAC system inspections are essential for building energy saving and greenhouse gas (GHG) emission reduction. According to related studies [1,2], about 70% of electricity consumption in the United States originates from the building sector, with HVAC systems being the largest consumer. HVAC inspection aims to identify and fix unnecessary energy loss including leaks or blockages in buildings as early as possible, given that damage, blocking, and aging of ducts are reasons for more than 30% of energy loss [3].

Currently, most HVAC system inspections are carried out manually by experienced engineers every one to two years, according to the National Air Duct Cleaners Association (NADCA) standard [4]. This manual approach is time-consuming, labor-intensive, and error-prone [5]. Recent studies and industry standards [[6], [7], [8]] focus on building highly sensed HVAC systems to reduce the need for manual inspections. These intelligent systems can achieve real-time system monitoring, alarming, and control based on pre-defined rules or learning-based control strategies [[9], [10], [11]]. However, the installation of such a complex sensor network is intrusive and expensive for existing buildings. In addition, sensors need regular maintenance to avoid false readings and failures, which can result in ignored alarms in practice due to constant false alarms [12].

Recent studies [13,14] explored the possibility of using robots to conduct HVAC inspections, including fire curtains [15], temperature distributions [16], and damages [17]. Robot inspection works for both new and existing buildings, avoiding regular maintenance of sensor networks. Compared with manual inspections, robots are efficient and precise [17]. During a regular HVAC inspection over ceilings, robots need to navigate across ducts, avoid obstacles, and inspect the HVAC system. Due to the navigational complexity of ceiling environments, inspection robots [[18], [19], [20]] are carefully designed by experts to ensure mobility. This process requires technical knowledge of robot design and control dynamics, and the process can be time-consuming when considering the entire procedure from robot design to fabrication [21]. Even so, the manually designed robot can still fail to complete tasks due to the lack of adaptation to diverse HVAC environments. As shown in Fig. 1, the steel bar barriers (Fig.1 a) on the surface and the existence of gaps (Fig. 1b) result in challenges to ground robots such as unmanned ground vehicles (UGVs). The limitation caused by ducts and chains (Fig.1 a) makes the movement of the unmanned aerial vehicle (UAV) dangerous and flexuous. For example, the UGV proposed in [13] cannot navigate in the aforementioned T-bar ceiling environment due to the reliance on wheels. The UAV designed in [18] can only operate in barrier-free ducts during non-operational hours of HVAC systems and also cannot successfully navigate in complex environments shown in Fig. 1 since the size of the UAV is larger than the limitations caused by ducts and chains. These shortcomings severely hamper the capability and adoption of robotic-based solutions for HVAC inspections. Therefore, an automatic robot design and evolution approach that can efficiently generate appropriate robot forms in a specific ceiling environment is needed.

Robot evolution refers to the self-improvement of robots through interactions with the environments while discovering useful properties from these interactions [[22], [23], [24]]. The evolution of robot morphology includes two aspects: optimizing parameters of a given shape and optimizing the shape in a limited morphological search space [25]. For HVAC inspection robots to be adaptive in different ceiling environments, the evolution of the robot shapes is necessary. Another barrier faced by robot-based HVAC system inspection stems from the fabrication of robots. For specially designed robots, production can be expensive if new molds are designed for every robot. An alternative idea is to assemble robots from primary elements (e.g., motors, servos, sensors, and body components) like LEGO bricks. Since primary elements can be fabricated in batches, the production cost can be diminished.

To address the above problems, we propose a framework where robots can evolve their morphologies for HVAC system inspection tasks (Fig. 2). This framework includes two modules. In the first module, a graph heuristic search method is used to evolve robot morphologies for specific tasks (e.g., navigation, obstacle avoidance). We design complex terrains for these tasks to evolve a robust robot form, which is composed of several pre-defined types of primary elements. The second module includes control of the evolved robot in ceiling environments. To simulate inspection activities in buildings, the robot is trained using reinforcement learning for navigation in ceiling environment models generated from Building Information Modeling (BIM) authoring tools (e.g., Revit) to represent realistic HVAC systems designs. The main contributions are:

  • We proposed a robot morphology evolution approach that aims to evolve adaptive robots for HVAC inspection tasks in a variety of ceiling designs.

  • We proposed a learning-based control approach, in which the BIMs of HVAC systems can be applied as general robot training environments for the training of robot control.

  • We tested the proposed framework by evolving robot morphologies in diverse and realistic ceiling environments adopted from BIMs of real-world buildings and achieved flexible and robust robot control in complex environments using reinforcement learning.

Section snippets

Evolutionary robotics

Evolutionary robotics focus on two main aspects of research, automatic robot control inspired by evolution, and automated design of robot morphologies so they can adapt to complex environments. In robot control, early research [26,27] focused on evolving dynamical network controllers for simple robots, in which the configuration (e.g., morphologies, sensors, and motors) of robots was given by experts, and the components (e.g., legs) were controlled by artificial neurons [28]. The neurons

Graph grammar based robot morphology evolution module

In this section, we introduce the robot morphology evolution module, where we aim to allow the robots to evolve to the best morphology for inspection tasks using graph heuristic search in a robot simulation engine [35]. The terrains are designed based on several complex environments in realistic HVAC systems. Graph heuristic search consists of three phases: the design phase, the evaluation phase, and the learning phase, as shown in Fig. 3.

During the design phase, the ϵ-greedy approach is

Learning-based control in virtual ceiling environments

This module aims to test the performance and leverage the flexibility of reinforcement learning control to help the “best” robot generated in the previous module navigate in diverse ceiling environments. To achieve this goal, this module first uses BIM models of HVAC systems as inputs and transforms the BIM model into a robot simulation engine (i.e., Isaac Sim), and then applies reinforcement learning algorithms to train the robot generated in terrain 4 (This terrain includes the most

Conclusions and future work

This paper presents a framework where robots can evolve their morphologies by interacting with surrounding HVAC environments, in addressing current problems of robot-based HVAC system inspections. The presented framework includes two modules. The first module utilizes the graph heuristic search method to evolve robot morphologies according to the designed terrains, and the second module examines the best robot's ability to navigate in realistic ceiling environments. The main contributions are

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

We acknowledge the support of the Natural Sciences and Engineering Research Council of Canada (NSERC). Kangkang acknowledges the support of the CSC scholarship from the Chinese Scholarship Council.

References (62)

  • Office of Energy eEfficiency Natural Resources Canada

    Energy Efficiency Trends in Canada

    (2007)
  • N.A.D.C. NADCA

    The NADCA Standard for Assessment, Cleaning, and Restoration of HVAC Systems

  • S. Ergan

    Processes, information requirements and challenges associated with corrective maintenance in relation to indoor air problem related work orders

  • A. Rowe et al.

    Large-scale campus-wide sensing and actuation

    IBM J. Res. Dev.

    (2011)
  • M.A. Hannan et al.

    A review of internet of energy based building energy management systems: issues and recommendations

    Ieee Access

    (2018)
  • H. Liu et al.

    Safe HVAC Control Via Batch Reinforcement Learning

    (2022)
  • B. Chen et al.

    Gnu-RL: A Precocial Reinforcement Learning Solution for Building HVAC Control Using a Differentiable MPC Policy

    (2019)
  • W. Cai et al.

    TEA-bot: a thermography enabled autonomous robot for detecting thermal leaks of HVAC systems in ceilings

  • P.R. Meris et al.

    Development of Mobile Robot for HVAC Component Monitoring Using Infrared Thermography

    (2020)
  • J. Moore et al.

    Facility inspection using UAVs: a case study in the University of Georgia campus

    Int. J. Remote Sens.

    (2018)
  • A. Borik et al.

    Caged Quadrotor Drone for Inspection of Central HVAC Ducts

    (2019)
  • T. Takayama et al.

    A twisted bundled tube locomotive device proposed for in-pipe mobile robot

  • D. Waleed et al.

    An in-pipe leak detection robot with a neural-network-based leak verification system

    IEEE Sensors J.

    (2019)
  • J. Pinskier et al.

    From bioinspiration to computer generation: developments in autonomous soft robot design

    Adv. Intellig. Syst.

    (2022)
  • A. Razjigaev et al.

    End-to-end design of bespoke, dexterous snake-like surgical robots: a case study with the RAVEN II

    Ieee T Robot

    (2022)
  • F. Corucci et al.

    Evolving soft locomotion in aquatic and terrestrial environments: effects of material properties and environmental transitions

    Soft Robot

    (2018)
  • V. Zykov et al.

    Evolved and designed self-reproducing modular robotics

    Ieee T Robot

    (2007)
  • A. Gupta et al.

    Embodied intelligence via learning and evolution

    Nat. Commun.

    (2021)
  • A.M. Turing

    Computing machinery and intelligence

    Mind, New Series

    (1950)
  • D. Floreano et al.

    Automatic creation of an autonomous agent: Genetic evolution of a neural network driven robot

  • R. Beer et al.

    Heterogeneous neural networks for adaptive behavior in dynamic environments

  • Cited by (0)

    View full text