Abstract Today, computer vision object detection methods are used for safety inspections from site videos and images. These methods detect bounding boxes and use hand-made rules to enable personal protective equipment compliance checks. This paper presents a new method to improve the breadth and depth of vision-based safety compliance checking by explicitly classifying worker-tool interactions. A detection model is trained on a newly constructed image dataset for construction sites, achieving 52.9% average mean precision for 10 object categories and 89.4% average precision for detecting workers. Using this detector and new dataset, the proposed human-object interaction recognition model achieved 79.78% precision and 77.64% recall for hard hat checking; 79.11% precision and 75.29% recall for safety coloring checking. The new model also verifies hand protection for workers when tools are being used with 66.2% precision and 64.86% recall. The proposed model is superior in these checking tasks when compared with post-processing detected objects with hand-made rules, or applying detected objects only.

中文翻译：

---

工地安全自动检测人机交互识别

摘要 今天，计算机视觉对象检测方法被用于现场视频和图像的安全检查。这些方法检测边界框并使用手工规则来启用个人防护设备合规性检查。本文提出了一种新方法，通过明确分类工人-工具交互来提高基于视觉的安全合规检查的广度和深度。在新构建的建筑工地图像数据集上训练检测模型，10 个对象类别的平均精度为 52.9%，工人检测的平均精度为 89.4%。使用该检测器和新数据集，所提出的人-物交互识别模型在安全帽检查中实现了 79.78% 的准确率和 77.64% 的召回率；安全着色检查的准确率为 79.11%，召回率为 75.29%。新模型还以 66.2% 的准确率和 64.86% 的召回率验证使用工具时工人的手部保护。与使用手工规则对检测到的对象进行后处理或仅应用检测到的对象相比，所提出的模型在这些检查任务中更为优越。