Abstract Quality and efficiency are crucial indicators of any manufacturing company. Many companies are suffering from a shortage of experienced workers across the production line to perform complex assembly tasks. To reduce time and error in an assembly task, a worker-centered system consisting of multi-modal Augmented Reality (AR) instructions with the support of a deep learning network for tool detection is introduced. The integrated AR is designed to provide on-site instructions including various visual renderings with a fine-tuned Region-based Convolutional Neural Network, which is trained on a synthetic tool dataset. The dataset is generated using CAD models of tools and displayed onto a 2D scene without using real tool images. By experimenting the system to a mechanical assembly of a CNC carving machine, the result of a designed experiment shows that the system helps reduce the time and errors of the given assembly tasks by 33.2 % and 32.4 %, respectively. With the integrated system, an efficient, customizable smart AR instruction system capable of sensing, characterizing requirements, and enhancing worker’s performance has been built and demonstrated.

中文翻译：

---

面向以工人为中心的智能制造的机械装配智能增强现实教学系统

摘要 质量和效率是任何制造企业的关键指标。许多公司都面临着生产线上缺乏经验丰富的工人来执行复杂的装配任务的困扰。为了减少装配任务中的时间和错误，引入了一个以工人为中心的系统，该系统由多模态增强现实 (AR) 指令组成，并支持用于工具检测的深度学习网络。集成的 AR 旨在提供现场指令，包括各种视觉渲染，以及经过微调的基于区域的卷积神经网络，该网络在合成工具数据集上进行训练。该数据集是使用工具的 CAD 模型生成的，并在不使用真实工具图像的情况下显示到 2D 场景中。通过将系统试验到 CNC 雕刻机的机械装配中，设计实验的结果表明，该系统有助于将给定装配任务的时间和错误分别减少 33.2% 和 32.4%。借助集成系统，构建并展示了一个高效、可定制的智能 AR 指令系统，该系统能够感知、表征需求并提高工人的绩效。