A two-stage knowledge-based deep learning algorithm is presented for enabling automated damage detection in real-time using the augmented reality smart glasses. The first stage of the algorithm entails the identification of damage prone zones within the region of interest. This requires domain knowledge about the damage as well as the structure being inspected. In the second stage, automated damage detection is performed independently within each of the identified zones starting with the one that is the most damage prone. For real-time visual inspection enhancement using the augmented reality smart glasses, this two-stage approach not only ensures computational feasibility and efficiency but also significantly improves the probability of detection when dealing with structures with complex geometric features. A pilot study is conducted using hands-free Epson BT-300 smart glasses during which two distinct tasks are performed: First, using a single deep learning model deployed on the augmented reality smart glasses, automatic detection and classification of corrosion/fatigue, which is the most common cause of failure in high-strength materials, is performed. Then, in order to highlight the efficacy of the proposed two-stage approach, the more challenging task of defect detection in a multi-joint bolted region is addressed. The pilot study is conducted without any artificial control of external conditions like acquisition angles, lighting, and so on. While automating the visual inspection process is not a new concept for large-scale structures, in most cases, assessment of the collected data is performed offline. The algorithms/techniques used therein cannot be implemented directly on computationally limited devices such as the hands-free augmented reality glasses which could then be used by inspectors in the field for real-time assistance. The proposed approach serves to overcome this bottleneck.

中文翻译：

---

增强现实通过基于知识的深度学习增强视觉检查

提出了一种基于知识的两阶段深度学习算法，用于使用增强现实智能眼镜实时实现自动损坏检测。算法的第一阶段需要识别感兴趣区域内的易损坏区域。这需要有关损坏以及被检查结构的领域知识。在第二阶段，从最容易损坏的区域开始，在每个识别的区域内独立执行自动损坏检测。对于使用增强现实智能眼镜的实时视觉检测增强，这种两阶段方法不仅确保了计算可行性和效率，而且在处理具有复杂几何特征的结构时显着提高了检测概率。使用免提 Epson BT-300 智能眼镜进行试点研究，在此期间执行两项不同的任务：首先，使用部署在增强现实智能眼镜上的单一深度学习模型，自动检测和分类腐蚀/疲劳，这是高强度材料失效的最常见原因是执行。然后，为了突出所提出的两阶段方法的有效性，解决了多关节螺栓区域中更具挑战性的缺陷检测任务。试点研究是在没有对采集角度、照明等外部条件进行任何人工控制的情况下进行的。虽然自动化视觉检查过程对于大型结构来说并不是一个新概念，但在大多数情况下，对收集到的数据的评估是离线进行的。其中使用的算法/技术不能直接在计算受限的设备上实现，例如免提增强现实眼镜，然后现场检查员可以使用这些设备进行实时协助。所提出的方法用于克服这个瓶颈。