Classification of construction hazard-related perceptions using: Wearable electroencephalogram and virtual reality

https://doi.org/10.1016/j.autcon.2021.103975Get rights and content

Highlights

  • Fifteen ML classifiers were developed to classify hazard-related EEG signals.

  • CatBoost classifier achieved the highest performance (95.1% accuracy).

  • Nine critical EEG features contributing to classification performance were found.

  • Classifier retrained using significant features achieved acceptable performance.

  • Three EEG channels and two frequency bands are closely related to hazard perception.

Abstract

Improving construction workers' safety is one of the most critical issues in the construction industry. Methods have been developed to better identify construction hazards on a jobsite by analyzing workers' physical and physiological responses collected from the wearable devices. Among them, electroencephalogram (EEG) holds unique potential since it shows immediate abnormal responses when a hazard is perceived. However, there remain limitations in the current knowledge base to attain the ultimate goal of ubiquitous hazard identification. In this context, this study investigates the feasibility of identifying construction hazards by developing an EEG classifier based on the experiments conducted in an immersive virtual reality (VR) environment. Results show that the CatBoost classifier achieved the highest performance with 95.1% accuracy. In addition, three important channel locations (AF3, F3, and F4) and two frequency bands (beta and gamma) were found to be closely associated with hazard perception.

Introduction

The construction industry has recorded a large number of fatalities every year. According to surveys conducted by the U.S. Bureau of Labor Statistics (BLS) during the last five years (2015–2019), the number of annual fatalities in the construction industry has increased by 13.23%, from 937 in 2015 to 1061 in 2019 [1]. Also, the construction industry accounted for 20% of all fatalities in the private sector in 2019.

To improve construction workers' safety, hazard identification—a process to identify potential hazard factors that can pose adverse effects (e.g., accidents) on workers—has become a critical component in safety management [2,3]. In the current practice, safety managers usually perform manual inspections on a construction site based on the procedure, which is known as job hazard analysis [4]. However, previous studies revealed that a significant amount of construction hazards still remain unidentified or not well-assessed at the jobsite, continuously exposing workers to unsafe working conditions [2,5,6]. Such poor hazard identification performance is due to the subjectivity and inconsistency in the inspector's hazard identification abilities [7], a limited number of resources (e.g., inspectors) to inspect the large area [8], and the dynamic nature of construction environments which decrease the inspector's ability to recognize hazards [9].

Extensive research efforts have been made to improve hazard identification performance by analyzing the physical and physiological responses of humans measured via wearable sensors. The underlying rationale is that workers show unusual or abnormal responses when they encounter a hazard [[10], [11], [12]]. Physical responses (e.g., gait and stride) have been studied by employing wearable sensors (e.g., inertial measurement units), and the relationship between the existence of hazards and changes in physical responses has been demonstrated [[13], [14], [15]]. Similarly, physiological responses (e.g., eye movement, brain activity, and electrodermal activity) have been investigated to infer and locate the hazards in the surrounding environment [[16], [17], [18]].

Among them, analyzing electroencephalogram (EEG)—a method that can detect and evaluate brain activities—offers unique opportunities since it contains direct and critical information associated with human perception, cognition, and intention [19,20]. In addition, the recent advancement of wearable sensing technology has enabled the real-time collection of EEG signals and the application of wearable EEG in many real-world scenarios (e.g., construction jobsite). Within the construction safety domain, EEG has been increasingly used to measure, assess, and monitor varying mental aspects of construction workers in terms of vigilance, distraction, emotional state, stress, and mental workload [[21], [22], [23], [24], [25], [26]]. Several researchers have attempted to identify construction hazards by adopting wearable EEG technology [16,27].

Despite the contribution of previous research efforts, there remain two limitations in the current knowledge base. First, how EEG signals collected from the wearable device can be utilized to locate and identify construction hazards remains unclear. Most previous EEG studies focused on assessing the changes in human mental status (e.g., emotional states, fatigue, and stress) induced by the construction hazards. Although changes in mental status have the potential to be used to identify hazards, to what extent of such changes can be used to classify EEG signals associated with a hazard or no-hazard has not been revealed. Also, considering that EEG signal is the first response of human hazard perception, its direct application to classify hazards needs to be studied. Second, most previous studies are based on the experimental setting that uses planar stimulus; many researchers used static images to simulate construction hazards and displayed them to subjects to analyze their influence on the human risk perception. However, such an approach often leads to information distortion, resulting in a significant gap between the indoor experiment and the real-world construction site [28]. Considering that wearable EEG holds great potential to be applied to the real-world jobsite, there is a critical need to conduct an experiment in a more realistic setting (e.g., immersive virtual environment) to understand the impacts of hazards on workers' perception.

In this context, the paper investigates the feasibility of identifying construction hazards by developing a wearable EEG classifier that can classify EEG signals associated with perceived hazards in the virtual reality (VR) environment via the methodology, which comprises the following five steps. First, various types of construction hazards (e.g., fall, struck-by, etc.) are simulated in the VR environment. Second, EEG data is collected in VR-based laboratory experiments. Third, collected data is preprocessed to remove signal noise and artifacts. Fourth, EEG features are extracted from the preprocessed data. Fifth, classifiers are developed based on each of the fifteen most widely used machine learning (ML) algorithms. Then the performance of different classifiers are compared, and important observations are discussed. The results of this paper will contribute to the ubiquitous hazard identification based on wearable EEG devices and thereby improve the current hazard identification capability.

Section snippets

Current construction hazard identification practice

Hazard identification is one of the most effective approaches for safety management in the construction industry [2,29]. In the current practice, hazard identification relies on human inspection; safety managers perform manual inspections on construction sites [30]. To improve the inspector's hazard identification ability, job hazard analysis and safety programs have been developed and proposed by the Occupational Safety and Health Administration (OSHA) [31,32]. However, a large number of

Simulation of construction hazards in VR

Five categories of construction hazards—which were identified as the leading cause of injuries and fatalities in the construction industry by the Occupational Safety and Health Administration (OSHA) [45,46]—were simulated in the VR environment. Table 2 presents the five hazard categories (fall, slip and trip, struck-by, chemical and electrical, etc.), each of which contains a varying number of scenarios.

For example, five different scenarios from No. 1 to 5 in Table 2 were developed to

Comparison of 15 EEG classifiers' performance

To compare and evaluate the performance of fifteen EEG classifiers, four metrics (accuracy, precision, recall, and F1-score) were adopted. Classification accuracy, the percentage of correctly classified observations (classified vs. actual class) with respect to the total number of observations, is the primary measure of classification performance (Eq. (8)). Precision is defined as the fraction of EEG signals correctly classified as signals associated with the perceived hazard (Eq. (9)). Recall

Conclusion

This paper investigated the feasibility of identifying construction hazards by developing a wearable EEG classifier based on the experiments conducted in the immersive VR environment. Five types of hazards (fall, slip and trip, struck-by, chemical and electrical, etc.), identified by OSHA as majors construction hazards, were simulated in thirty VR scenes by consulting OSHA Safety and Health Regulations for Construction. EEG signals were collected from twenty-eight subjects who wore both

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (69)

  • D. Wang et al.

    Detecting and measuring construction workers’ vigilance through hybrid kinematic-EEG signals

    Autom. Constr.

    (2019)
  • D. Wang et al.

    Monitoring workers’ attention and vigilance in construction activities through a wireless and wearable electroencephalography system

    Autom. Constr.

    (2017)
  • J. Chen et al.

    Revealing the “invisible Gorilla” in construction: estimating construction safety through mental workload assessment

    Autom. Constr.

    (2016)
  • P.-C. Liao et al.

    A multimodal study to measure the cognitive demands of hazard recognition in construction workplaces

    Saf. Sci.

    (2021)
  • H. Guo et al.

    The availability of wearable-device-based physical data for the measurement of construction workers’ psychological status on site: from the perspective of safety management

    Autom. Constr.

    (2017)
  • S. Zhang et al.

    Building information modeling (BIM) and safety: automatic safety checking of construction models and schedules

    Autom. Constr.

    (2013)
  • H. Kim et al.

    Assessing the effects of slippery steel beam coatings to ironworkers’ gait stability

    Appl. Ergon.

    (2018)
  • X. Zhou et al.

    Hazard differentiation embedded in the brain: a near-infrared spectroscopy-based study

    Autom. Constr.

    (2021)
  • I. Jeelani et al.

    Automating and scaling personalized safety training using eye-tracking data

    Autom. Constr.

    (2018)
  • A. Heydarian et al.

    Immersive virtual environments versus physical built environments: a benchmarking study for building design and user-built environment explorations

    Autom. Constr.

    (2015)
  • A. Delorme et al.

    EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis

    J. Neurosci. Methods

    (2004)
  • M. Chaumon et al.

    A practical guide to the selection of independent components of the electroencephalogram for artifact correction

    J. Neurosci. Methods

    (2015)
  • H. Jebelli et al.

    EEG-based workers’ stress recognition at construction sites

    Autom. Constr.

    (2018)
  • T.R. Schneider et al.

    Modulation of neuronal oscillatory activity in the beta-and gamma-band is associated with current individual anxiety levels

    NeuroImage

    (2018)
  • L. Zhu et al.

    Mechanisms behind hazard perception of warning signs: an EEG study

    Transp. Res. Part F Traffic Psychol. Behav.

    (2020)
  • E.A.L. Teo et al.

    Framework for project managers to manage construction safety

    Int. J. Proj. Manag.

    (2005)
  • U.S. Bureau of Labor Statistics (BLS)

    Census of Fatal Occupational Injuries (CFOI)

  • G. Carter et al.

    Safety hazard identification on construction projects

    J. Constr. Eng. Manag.

    (2006)
  • Canadian Centre for Occupational Health and Safety (CCOHS), Hazard Identification

  • A. Albert et al.

    Enhancing construction hazard recognition and communication with energy-based cognitive mnemonics and safety meeting maturity model: multiple baseline study

    J. Constr. Eng. Manag.

    (2014)
  • P. Mitropoulos et al.

    Systems model of construction accident causation

    J. Constr. Eng. Manag.

    (2005)
  • A. Albert et al.

    Experimental field testing of a real-time construction hazard identification and transmission technique

    Constr. Manag. Econ.

    (2014)
  • D. Crundall et al.

    Eye movements and hazard perception in police pursuit and emergency response driving

    J. Exp. Psychol. Appl.

    (2003)
  • C.R. Ahn et al.

    Wearable sensing technology applications in construction safety and health

    J. Constr. Eng. Manag.

    (2019)
  • Cited by (0)

    View full text