Elsevier

Energy and Buildings

Volume 270, 1 September 2022, 112271
Energy and Buildings

Digital Twin-driven approach to improving energy efficiency of indoor lighting based on computer vision and dynamic BIM

https://doi.org/10.1016/j.enbuild.2022.112271Get rights and content

Abstract

Intelligent lighting systems and surveillance systems have become an important part of intelligent buildings. However, the current intelligent lighting system generally adopts independent sensor control and does not perform multi-source heterogeneous data fusion with other digital systems. This paper fully considers the linkage between the lighting system and the surveillance system and proposes a digital twin lighting (DTL) system that mainly consists of three parts. Firstly, a visualized operation and maintenance (VO&M) platform for a DTL system was established based on dynamic BIM. Secondly, the environment perception, key-frame similarity judgment, and multi-channel key-frame cut and merge mechanism were utilized to preprocess the video stream of the surveillance system in real-time. Lastly, pedestrians detected using YOLOv4 and the ambient brightness perceived by the environment perception mechanism were transmitted to the cloud database and were continuously read by the VO&M platform. The intent here was to aid timely adaptive adjustment of the digital twin and realistic lighting through the internet. The effectiveness of the proposed method was verified by experimenting with a surveillance video stream for 14 days. The key results of the experiments are as follows: (1) the accuracy rate of intelligent decision control reached 95.15%; (2) energy consumption and electricity costs were reduced by approximately 79%; and (3) the hardware cost and energy consumption of detection equipment and the time and cost of operation and maintenance (O&M) were greatly reduced.

Introduction

According to relevant statistics, the annual growth rate of the global construction footprint from 2005 to 2020 reached 6.92% [1]. However, the energy intensity of the construction sector has only continued to decline by 0.5% to 1% per year since 2010, which is far lower than the urban area expansion rate and the construction area growth rate [2]. This has led to construction energy consumption that has accounted for 40% of the world's total energy consumption [3]. Therefore, reducing the source demand of buildings is essential to alleviate the pressure and sustainability of the global energy supply [4]. One of the methods is to explore the energy-saving potential of lighting systems, which is of great significance for reducing the overall energy consumption of buildings. This is because lighting systems consume about 20% to 60% of the energy used in office buildings and rank second to the ventilation system [5], [6].

Under the conditions of ensuring the illuminance scale and lighting quality, improving the energy conversion efficiency of lighting systems has become a crucial research area of green building energy conservation [7], [8], [9]. At present, the commonly used energy-saving methods for lighting systems include the use of sensor control and LED lights instead of manual switch control and fluorescent lights [10], [11], [12], [13]. Although these methods can effectively reduce the energy consumption of lighting systems, they also cause a substantial increase in the initial installation cost [14]. The biggest drawback of these lighting systems is the inability to realize intelligent and integrated management, which presents huge difficulties for the whole life cycle O&M as well as energy consumption prediction.

To solve the limitations of existing lighting systems, more and more attention has been paid to integrating indoor intelligent lighting systems. For example, some scholars have begun to combine the Internet of Things (IoT) and sensors to complete more complex intelligent control and enhance the ability of lighting systems to interact with the environment [15], [16]. Although the current research can solve the problem of intelligent control to a certain extent, there is still a lack of methods to realize visual management and intelligent control of lighting systems in the whole process. Widespread applications of computer vision and digital twins offer alternatives to address the above difficulties. The advantages of computer vision include long-distance, non-contact, high-precision, and high-efficiency data collection, which can help in developing in-depth, whole-process intelligent decision-making control of lighting systems. Michael and Jon [17] coined the term “digital twin” in 2003. Since then, its popularity has grown and is now recognized as a key enabler of the transition to Industry 4.0. A digital twin can reflect a real-world built environment in a virtual space, and can also simulate in real time the processes connected throughout its entire life cycle, performing diagnostic and predictive analytics for O&M [18]. In addition, some scholars have developed lightweight web clients for digital twin systems based on WebGL and the Internet, which further promotes remote browsing and sharing of digital assets and provides a remote visualization, integration and collaborative management platform for digital twin systems [19].

Considering the advantages of computer vision and digital twin, this paper proposes to combine both lighting and surveillance systems to establish a VO&M platform for a DTL system. The originality and contributions of this paper are highlighted as follows: (1) provides an in-depth, full-process, and intelligent control decisions method for a lighting system; (2) formed the digital twin asset of a lighting system, which can provide remote visual management; (3) used big data analysis technology to analyze O&M data to provide a data-driven platform for realizing an integrated management of the entire life cycle of green smart buildings and lighting equipment; (4) verified that the multi-source heterogeneous data fusion of surveillance system, lighting system, and BIM is helpful to improve indoor energy efficiency for intelligent buildings.

The remainder of this paper is organized as follows. Section 2 reviews the literature on lighting systems, BIM, and computer vision. Section 3 discusses the proposed method in detail. Section 4 presents a case study to verify the effectiveness of the proposed method. Section 5 includes conclusions and future work.

Section snippets

Energy saving for lighting systems

Presently, several energy-saving research focused on light source and control systems have been conducted and achieved excellent results. For example, the use of new LED lights saved 10%-25% of lighting energy consumption [20], and the use of sensor control saved more than 50% of lighting energy consumption. Additionally, Juntunen et al. [21] use passive infrared (PIR) sensors to intelligently track the movement of pedestrians and dynamically controlled lighting devices, which saved more than

Methodology

This research combined an office building's existing surveillance system and lighting system and proposed a new digital twin-based intelligent lighting system, which can adaptively provide lighting quality that is fit for purpose and minimizes energy consumption. The proposed cloud-based system can be divided into four parts, including a realistic lighting system, detection equipment, digital assets and operation and maintenance as shown in Fig. 1.

Case study

This research used 14 days of a surveillance video stream for experiments to verify the effectiveness of the proposed method. The period is from 2020.12.02 to 2020.12.15, which is the normal school time, including Monday to Sunday; hence, it has a certain degree of representativeness. The experimental scene is a long corridor with two cameras (Fig. 14). The corridor lacks natural lighting and relies on artificial lighting to meet its lighting demand. The energy consumption of this corridor

Conclusions

Through the combination of computer vision and BIM, this paper proposes a DTL system, which provides in-depth, full-process and intelligent decision-making on lighting control. Additionally, it contributes to the collaborative management of lighting system, which can result in reducing energy consumption and electricity costs. A dynamic BIM platform based on Three.js has also been created, which can reduce O&M time and costs. In addition, the digital twin assets of the intelligent lighting

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This research is supported by the Shenzhen Science and Technology Project(No. JSGG20210802153801004), the Foundation for Distinguished Young Talents in Higher Education of Guangdong, China (FDYT) (No. 2020KQNCX060) and the Foundation for Basic and Applied Basic Research of Guangdong Province (No. 2020A1515111189).

Statement of dataset use

This paper created a novel pedestrian dataset by combining the USC public pedestrian dataset and self-made video stream key-frames. We are grateful to USC for opening access to the pedestrian dataset to other researchers. The ownership of the USC dataset belongs to the original author. This paper only uses it to complete the training of the model, and does not make any changes to the dataset. The ownership of the self-made video streaming dataset belongs to the author of this paper, and has the

References (75)

  • A. Ikuzwe et al.

    Energy-maintenance optimization for retrofitted lighting system incorporating luminous flux degradation to enhance visual comfort

    Appl. Energy

    (2020)
  • S. Tang et al.

    Development of a prototype smart home intelligent lighting control architecture using sensors onboard a mobile computing system

    Energy Build.

    (2017)
  • F. Sun et al.

    Indoor intelligent lighting control method based on distributed multi-agent framework

    Optik

    (2020)
  • M. Mahbub et al.

    IoT-Cognizant cloud-assisted energy efficient embedded system for indoor intelligent lighting, air quality monitoring, and ventilation

    Internet Things

    (2020)
  • D. Gürdür Broo et al.

    Design and implementation of a smart infrastructure digital twin

    Autom. Constr.

    (2022)
  • C. Liu et al.

    Web-based digital twin modeling and remote control of cyber-physical production systems

    Rob. Comput. Integr. Manuf.

    (2020)
  • I. Chew et al.

    Design of an energy-saving controller for an intelligent LED lighting system

    Energy Build.

    (2016)
  • E. Juntunen et al.

    Smart and dynamic route lighting control based on movement tracking

    Build. Environ.

    (2018)
  • B. Roisin et al.

    Lighting energy savings in offices using different control systems and their real consumption

    Energy Build.

    (2008)
  • Y. Gao et al.

    A wireless sensor network based on the novel concept of an I-matrix to achieve high-precision lighting control

    Build. Environ.

    (2013)
  • N. van de Meugheuvel et al.

    Distributed lighting control with daylight and occupancy adaptation

    Energy Build.

    (2014)
  • K.R. Wagiman et al.

    A new optimal light sensor placement method of an indoor lighting control system for improving energy performance and visual comfort

    J. Build. Eng.

    (2020)
  • G. Chiesa et al.

    A fuzzy-logic IoT lighting and shading control system for smart buildings

    Autom. Constr.

    (2020)
  • Z. Li et al.

    Design of Office Intelligent Lighting System Based on Arduino

    Procedia Comput. Sci.

    (2020)
  • M.R. Ibrahim et al.

    Understanding cities with machine eyes: A review of deep computer vision in urban analytics

    Cities

    (2020)
  • W. Wei et al.

    Special issue on role of computer vision in smart cities

    Image Vis. Comput.

    (2021)
  • S. Sulochanan Karthick Ramanathan et al.

    A novel face recognition technology to enhance health and safety measures in hospitals using SBC in pandemic prone areas

    Mater. Today:. Proc.

    (2021)
  • Z. Zhu et al.

    Application of attitude tracking algorithm for face recognition based on OpenCV in the intelligent door lock

    Comput. Commun.

    (2020)
  • V. Seelam et al.

    Smart attendance using deep learning and computer vision

    Mater. Today:. Proc.

    (2021)
  • S. Wei et al.

    Vision-based detection and prediction of equipment heat gains in commercial office buildings using a deep learning method

    Appl. Energy

    (2020)
  • M. Despotovic et al.

    Prediction and analysis of heating energy demand for detached houses by computer vision

    Energy Build.

    (2019)
  • A. Zawadzki

    Lighting Fitting Controller Using Image Processing System

    IFAC Proc.

    (2009)
  • C. Carrillo et al.

    Lighting control system based on digital camera for energy saving in shop windows

    Energy Build.

    (2013)
  • Y. Wu et al.

    Characterization of a quasi-real-time lighting computing system based on HDR imaging

    Energy Procedia

    (2017)
  • A. Motamed et al.

    On-site monitoring and subjective comfort assessment of a sun shadings and electric lighting controller based on novel High Dynamic Range vision sensors

    Energy Build.

    (2017)
  • Y.-Y. Liu

    Research on library lighting intelligent control based on infrared image processing techniques

    Optik

    (2015)
  • M. Shanmugam et al.

    Energy efficient intelligent light control with security system for materials handling warehouse

    Mater. Today:. Proc.

    (2021)
  • Cited by (26)

    View all citing articles on Scopus
    View full text