Abstract Human-Robot Collaboration (HRC), which enables a workspace where human and robot can dynamically and safely collaborate for improved operational efficiency, has been identified as a key element in smart manufacturing. Human action recognition plays a key role in the realization of HRC, as it helps identify current human action and provides the basis for future action prediction and robot planning. While Deep Learning (DL) has demonstrated great potential in advancing human action recognition, effectively leveraging the temporal information of human motions to improve the accuracy and robustness of action recognition has remained as a challenge. Furthermore, it is often difficult to obtain a large volume of data for DL network training and optimization, due to operational constraints in a realistic manufacturing setting. This paper presents an integrated method to address these two challenges, based on the optical flow and convolutional neural network (CNN)-based transfer learning. Specifically, optical flow images, which encode the temporal information of human motion, are extracted and serve as the input to a two-stream CNN structure for simultaneous parsing of spatial-temporal information of human motion. Subsequently, transfer learning is investigated to transfer the feature extraction capability of a pretrained CNN to manufacturing scenarios. Evaluation using engine block assembly confirmed the effectiveness of the developed method.

中文翻译：

---

用于人体动作识别的可转移两流卷积神经网络

摘要 人机协作 (HRC) 已被确定为智能制造的关键要素，它使人与机器人可以动态、安全地协作以提高运营效率的工作空间成为可能。人体动作识别在 HRC 的实现中起着关键作用，因为它有助于识别当前的人体动作并为未来的动作预测和机器人规划提供基础。虽然深度学习 (DL) 在推进人类动作识别方面表现出巨大潜力，但有效利用人类动作的时间信息来提高动作识别的准确性和鲁棒性仍然是一个挑战。此外，由于现实制造环境中的操作限制，通常很难获得大量数据用于 DL 网络训练和优化。本文基于光流和基于卷积神经网络 (CNN) 的迁移学习，提出了一种综合方法来解决这两个挑战。具体来说，提取对人体运动时间信息进行编码的光流图像，并将其作为双流 CNN 结构的输入，用于同时解析人体运动的时空信息。随后，研究转移学习以将预训练的 CNN 的特征提取能力转移到制造场景中。使用发动机缸体组件的评估证实了所开发方法的有效性。对人体运动的时间信息进行编码，提取并作为双流 CNN 结构的输入，用于同时解析人体运动的时空信息。随后，研究转移学习以将预训练的 CNN 的特征提取能力转移到制造场景中。使用发动机缸体组件的评估证实了所开发方法的有效性。对人体运动的时间信息进行编码，提取并作为双流 CNN 结构的输入，用于同时解析人体运动的时空信息。随后，研究转移学习以将预训练的 CNN 的特征提取能力转移到制造场景中。使用发动机缸体组件的评估证实了所开发方法的有效性。