In this paper, using the ball and beam system as an illustration, a control scheme is developed on human-robot collaboration, i.e., a two-level hierarchical framework is proposed to establish a robust human-robot collaboration (HRC) policy. On the high level, a deep reinforcement learning (DRL) algorithm is presented to plan the desired beam rotational velocity. The low level is constructed by a human-intention perception module and a robust collaboration policy design module. For the first module, a probabilistic model is fitted by using the Gaussian process regression (GPR) approach to predict human-hand velocities, and prediction results follow Gaussian distributions where mean values and variances represent predicted human-hand velocities and corresponding prediction confidences, respectively. For the second module, a robust collaboration policy is established by fusing a proactive policy and a conservative policy, where the proactive policy is used to control the robot to achieve the desired beam rotational velocity by using the predicted human-hand velocities. The conservative policy is designed to ensure the collaboration safety. The weighted parameters for fusion are adaptively tuned based on the prediction precision and confidence. Experiments are conducted on controlling ball position on a beam jointly by a human and a robot with vision data, and experimental results show the effectiveness of the designed robust collaboration policy. Note to Practitioners—Predicting human future behaviors and moderating robot behaviors accordingly is a long-standing problem for human-robot collaboration (HRC) tasks, such as assembling, transporting, etc. Existing approaches generally regard human behaviors as noises or only build simple human models without prediction confidence. This paper proposes a learning-based hierarchical framework that will derive a robust and safe HRC policy considering human behaviors, prediction confidence, and task-related optimality. The framework is validated by a representative experiment where human and robot are asked to jointly control a ball and beam system.

中文翻译：

---

一种基于学习的人机协作分层控制框架

在本文中，以球和梁系统为例，开发了一种人机协作控制方案，即提出了一个两级层次框架来建立鲁棒的人机协作（HRC）策略。在高层次上，提出了一种深度强化学习 (DRL) 算法来规划所需的光束旋转速度。低层由人类意图感知模块和强大的协作策略设计模块构建。对于第一个模块，使用高斯过程回归（GPR）方法拟合概率模型来预测人手速度，预测结果服从高斯分布，其中均值和方差分别代表预测的人手速度和相应的预测置信度. 对于第二个模块，通过融合主动策略和保守策略建立稳健的协作策略，其中主动策略用于控制机器人通过使用预测的人手速度来实现所需的光束旋转速度。保守策略旨在确保协作安全。融合的加权参数根据预测精度和置信度进行自适应调整。利用视觉数据对人和机器人联合控制横梁上的球位置进行了实验，实验结果表明了所设计的鲁棒协作策略的有效性。从业者注意事项——预测人类未来行为并相应地调节机器人行为是人机协作 (HRC) 任务（例如组装、运输等）中长期存在的问题。现有方法通常将人类行为视为噪声，或者仅建立没有预测置信度的简单人类模型。本文提出了一个基于学习的层次框架，该框架将在考虑人类行为、预测置信度和任务相关最优性的情况下推导出一个稳健且安全的 HRC 策略。该框架通过一个具有代表性的实验进行了验证，在该实验中，人类和机器人被要求共同控制一个球和梁系统。