Skip to main content
SpringerLink
Log in

Unsafe Construction Behavior Classification Using Deep Convolutional Neural Network

  • APPLICATION PROBLEMS
  • Published:
Pattern Recognition and Image Analysis Aims and scope Submit manuscript

Abstract

In the construction industry, about 80–90% of accidents are caused by the unsafe actions and behaviors of employees. Thus, behavior management plays a key role in enhancing safety. In particular, behavior observation is the most critical element for modifying workers’ behavior in a safe manner. However, there is a lack of practical methods to measure workers’ behavior in construction as current literature only focuses on a few unusual signs such as not wearing personal protective equipment. This paper proposes a system for recognizing workers’ dangerous behaviors. To that end, an image dataset has been collected, labeled for three such behaviors. Based on the dataset obtained, the transfer-learning approach is used with three pre-trained models, VGG19, Inception_V3 and InceptionResnet_V2. The results indicate that InceptionResnet_V2 performs better than VGG19_ and Inception_V3 for classifying unsafe behaviors and after 150 epochs, its accuracy reaches 92.44%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.
Fig. 13.
Fig. 14.
Fig. 15.
Fig. 16.
Fig. 17.
Fig. 18.
Fig. 19.
Fig. 20.
Fig. 21.

Similar content being viewed by others

REFERENCES

  1. H.W. Heinrich, D. Petersen, and N. Roos, Industrial Accident Prevention (McGraw-Hill, New York, 1980).

    Google Scholar 

  2. L. Helen and S. Rowlinson, Occupational Health and Safety in Construction Project Management (Spon Press, London, 2005), pp. 157–158.

    Google Scholar 

  3. T. R. Krause, The Behavior-Based Safety Process—Managing Involvement for an Injury-Free Culture (Wiley, New York, 1990).

    Google Scholar 

  4. J. Komaki, K. D. Barwick, and L. R. Scott, “A behavioral approach to occupational safety: Pinpointing and reinforcing safety performance in a food manufacturing plan,” J. Appl. Psych. 63 (4), 434–445 (1978).

    Article  Google Scholar 

  5. A. R. Duff, I. T. Robertson, R. A. Phillips, and M. D. Cooper, “Improving safety by the modification of behavior,” Constr. Manage. Econ. 12 (1), 67–78 (1994).

    Article  Google Scholar 

  6. T. R. Krause, K. J. Seymour, and C. M. Sloat, “Long-term evaluation of behavior-based method for improving safety performance: A meta-analysis of 73 interrupted time-series replications,” Saf. Sci. 32 (1), 1–18 (1999).

    Article  Google Scholar 

  7. T. E. McSween, Value-Based Safety Process: Improving Your Safety Culture with Behavior-Based Safety (Wiley, Hoboken, NJ, 2003).

    Book  Google Scholar 

  8. J. F. Godbey, “The effects of behavior-based safety techniques on behavior variation, targeted and non-targeted safety behaviors, and productivity and quality in manufacturing facilities,” PhD Dissertation (Auburn Univ., Auburn, AL, 2006).

  9. S. Han, S. Lee, S, and F. Peña-Mora, “Vision-based motion detection for safety behavior analysis in construction,” in Proc. of 2012 Construction Research Congress (CRC) (ASCE, Reston, VA, 2012), pp. 1032–1041.

  10. R. E. Levitt and N. M. Samelson, Construction Safety Management (McGraw-Hill, New York, 1987).

    Google Scholar 

  11. H. Laitinen, M. Marjamäki, and K. Päivärinta, “The validity of the TR safety observation method on building construction,” Accid. Anal. Prev. 31 (5), 463–472 (1999).

    Article  Google Scholar 

  12. S. Han, S. Lee, and F. Peña-Mora, “Application of dimension reduction techniques for motion recognition: Construction worker behavior monitoring,” in Proc. of 2011 ASCE Int. Workshop on Computing in Civil Engineering (ASCE, Reston, VA, 2011), pp. 102–109.

  13. J. S. Chhokar and J. A. Wallin, “Improving safety through applied behavior analysis,” J. Saf. Res. 15 (4), 141–151 (1984).

    Article  Google Scholar 

  14. J. L. Komaki, S. Zlotnick, and M. Jensen, “Development of an operant-based taxonomy and observational index of supervisory behavior,” J. Appl. Psych. 71 (2), 260–269 (1986).

    Article  Google Scholar 

  15. M. Park, E. Palinginis, and I. Brilakis, “Detection of construction workers in video frames for automatic initialization of vision trackers,” in Construction Research Congress (2012). https://doi.org/10.1061/9780784412329.095.

  16. I. Brilakis, M. W. Park, and G. Jog, “Automated vision tracking of project related entities,” Adv. Eng. Inf. 25 (2011). https://doi.org/10.1016/j.aei.2011.01.003

  17. V. Pətrəucean, I. Armeni, M. Nahangi, J. Yeung, I. Brilakis, and C. Haas, “State of research in automatic as-built modelling,” Adv. Eng. Inf. 29, 162–171 (2015). https://doi.org/10.1016/j.aei.2015.01.001

    Article  Google Scholar 

  18. M. W. Park and I. Brilakis, “Construction worker detection in video frames for initializing vision trackers,” Autom. Constr. 28, 15–25 (2012). https://doi.org/10.1016/j.autcon.2012.06.001

    Article  Google Scholar 

  19. Y. Ham, K. K. Han, J. J. Lin, and M. Golparvar-Fard, “Visual monitoring of civil infrastructure systems via camera-equipped unmanned aerial vehicles (UAVs): A review of related works,” Vis. Eng. 4 (1) (2016). https://doi.org/10.1186/s40327-015-0029-z

  20. J. Teizer, “Status quo and open challenges in vision-based sensing and tracking of temporary resources on infrastructure construction sites,” Adv. Eng. Inf. 29, 225–238 (2015). https://doi.org/10.1016/j.aei.2015.03.006

    Article  MathSciNet  Google Scholar 

  21. J. Yang, Z. Shi, and Z. Wu, “Vision-based action recognition of construction workers using dense trajectories,” Adv. Eng. Inf. 30, 327–336 (2016).https://doi.org/10.1016/j.aei.2016.04.009

    Article  Google Scholar 

  22. P. Viola and M. J. Jones, “Robust real-time face detection,” Int. J. Comput. Vis. 57, 137–154 (2004). https://doi.org/10.1023/B:VISI.0000013087.49260.fb

    Article  Google Scholar 

  23. S. Du, M. Shehata, and W. Badawy, “Hard hat detection in video sequences based on face features, motion and color information,” in Proc. 3rd International Conference on Computer Research and Development (2011), pp. 25–29. https://doi.org/10.1109/ICCRD.2011.5763846.

  24. H. Kim, K. Kim, and H. Kim, “Data-driven scene parsing method for recognizing construction site objects in the whole image,” Autom. Constr. 71, 271–282 (2016). https://doi.org/10.1016/j.autcon.2016.08.018

    Article  Google Scholar 

  25. W. Rawat and Z. Wang, “Deep convolutional neural networks for image classification: A comprehensive review,” Neural Comput. 29 (9), 2352–2449 (2017).

    Article  MathSciNet  Google Scholar 

  26. W. Fang, L. Ding, H. Luo, and P. E. D. Love, “Falls from heights: A computer vision-based approach for safety harness detection,” Autom. Constr. 91, 53–61 (2018). https://doi.org/10.1016/j.autcon.2018.02.018

    Article  Google Scholar 

  27. Q. Fang, H. Li, X. Luo, L. Ding, H. Luo, and C. Li, “Computer vision aided inspection on falling prevention measures for steeplejacks in an aerial environment,” Autom. Constr. 93, 148–164 (2018). https://doi.org/10.1016/j.autcon.2018.05.022

    Article  Google Scholar 

  28. Q. Fang, H. Li, X. Luo, L. Ding, H. Luo, T. M. Rose, and W. An, “Detecting non-hardhat-use by a deep learning method from far-field surveillance videos,” Autom. Constr. 85, 1–9 (2018). https://doi.org/10.1016/j.autcon.2017.09.018

    Article  Google Scholar 

  29. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv (2014). arXiv:1409.1556

  30. C. Szegedy, V. Vanhoucke, S. Ioffe, et al., Rethinking the inception architecture for computer vision. https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Szegedy_Rethinking_the_Inception_CVPR_2016_paper.pdf. Accessed June 22, 2020.

  31. K. He, X. Zhang, S. Ren, et al., Deep residual learning for image recognition. https://www.cv-foundation.org/openaccess/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html. Accessed June 22, 2020.

  32. J. Deng, W. Dong, R. Socher, et al., “ImageNet: A large-scale hierarchical image database,” in IEEE Conference on Computer Vision and Pattern Recognition (Miami, FL, 2009).

  33. K. Zdenek, C. Hainan, and L. Xiaowei, “Transfer learning and deep convolutional neural networks for safety guardrail detection in 2D images,” Autom. Constr. 89, 58–70 (2018). https://doi.org/10.1016/j.autcon.2018.01.003

    Article  Google Scholar 

  34. https://github.com/tiensu/Construction_Site_Unsafe_Actions_Dataset.git.

  35. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed. (Pearson, 2018), chap. 3.

    Google Scholar 

  36. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed. (Pearson, 2018), chap. 4.

    Google Scholar 

  37. H. Kim, S. Sohn, and J. S. Kim, “Revisiting Gist-PCA hashing for near duplicate image detection,” J. Sign. Process Syst. 91, 575–586 (2019). https://doi.org/10.1007/s11265-018-1360-0

    Article  Google Scholar 

  38. Keras-team, “Image data preprocessing.” https://keras.io/api/preprocessing/image/. Accessed June 22, 2020.

  39. S. Leslie, “Cyclical learning rates for training neural networks,” arXiv (2015). arXiv:1506.01186

  40. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Proceedings of the 25th International Conference on Neural Information Processing Systems (2012), pp. 1097–1105.

  41. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv (2014). arXiv:1409.1556

  42. K. Gopalakrishnan, S. K. Khaitan, A. Choudhary, et al., “Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection,” Constr. Build. Mater. 157, 322–330 (2017).

    Article  Google Scholar 

  43. Stanford Vision Lab, Large Scale Visual Recognition Challenge 2012 (VOC2012). http://image-net.org/challenges/LSVRC/2012/. Accessed June 22, 2020.

  44. K. He, X. Zhang, S. Ren, et al., “Deep residual learning for image recognition,” arXiv (2015). arXiv:1512.03385

  45. K. He, X. Zhang, S. Ren, et al., “Identity mappings in deep residual networks,” arXiv (2016). arXiv:1603.05027

  46. Uniqtech, Understand the Softmax Function in Minutes. https://medium.com/data-science-bootcamp/understand-the-softmax-function-in-minutes-f3a59641e86d. Accessed June 22, 2020.

  47. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv (2017). arXiv:1412.6980

  48. Keras-Team, Keras: The Python Deep Learning library. https://keras.io/. Accessed June 22, 2020.

Download references

ACKNOWLEDGMENTS

We would like to thank our colleagues in the Information Technology Specialization Department of FPT University, Hanoi, Vietnam for their critical and relevant comments on the manuscript; Colleagues in the English Department who have helped to polish the English text. We would also like to extend our gratitude to experts at the Vietnam Institute for Building Science and Technology (IBST) for helping us with the methodological aspects of this study and with reviewing part of the data.

Author information

Authors and Affiliations

  1. FPT University, Hoa Lac High Tech Park, 10000, Hanoi, Vietnam

    P. D. Hung & N. T. Su

Authors
  1. P. D. Hung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. N. T. Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to P. D. Hung or N. T. Su.

Ethics declarations

COMPLIANCE WITH ETHICAL STANDARDS

Conflict of interests. The authors declare no conflict of interest neither in financial nor in any other area.Statement of compliance with standards of research involving humans as subjects. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants involved in the study.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflict of interest neither in financial nor in any other area.

Additional information

Phan Duy Hung received his Ph.D. degree from INP Grenoble France, in 2008. Since 2009, he has worked as a Lecturer, and served as the Head of Department and the Director of Master Program in the Software engineering at FPT University, Hanoi, Vietnam.

His current research interests include Digital Signal and Image processing, Internet of Things, BigData, Artificial Intelligence, Measurement and Control Systems and Industrial networking.

Nguyen Tien Su received his B. Sc from Hanoi University of Science and Technology, Vietnam in 2012, and MCS from FPT University, Vietnam 2020. Since 2013, he has worked as an engineer in the fields of Machine Learning, Image Processing and Software engineering.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hung, P.D., Su, N.T. Unsafe Construction Behavior Classification Using Deep Convolutional Neural Network. Pattern Recognit. Image Anal. 31, 271–284 (2021). https://doi.org/10.1134/S1054661821020073

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1054661821020073

Keywords:

Search

Navigation