Physical human-robot collaboration is increasingly required in many contexts (such as industrial and rehabilitation applications). The robot needs to interact with the human to perform the target task while relieving the user from the workload. To do that, the robot should be able to recognize the human's intentions and guarantee safe and adaptive behavior along the intended motion directions. The robot-control strategies with such attributes are particularly demanded in the industrial field, where the operator guides the robot manually to manipulate heavy parts (e.g., while teaching a specific task). With this aim, this work proposes a Q-Learning-based Model Predictive Variable Impedance Control (Q-LMPVIC) to assist the operators in a physical human-robot collaboration (pHRC) tasks. A Cartesian impedance control loop is designed to implement a decoupled compliant robot dynamics. The impedance control parameters (i.e., setpoint and damping parameters) are then optimized online in order to maximize the performance of the pHRC. For this purpose, an ensemble of neural networks is designed to learn the modeling of the human-robot interaction dynamics while capturing the associated uncertainties. The derived modeling is then exploited by the model predictive controller (MPC), enhanced with the stability guarantees by means of Lyapunov constraints. The MPC is solved by making use of a Q-Learning method that, in its online implementation, uses an actor-critic algorithm to approximate the exact solution. Indeed, the Q-learning method provides an accurate and highly efficient solution (in terms of computational time and resources). The proposed approach has been validated through experimental tests, in which a Franka EMIKA panda robot has been used as a test platform. Each user was asked to interact with the robot along the controlled vertical z Cartesian direction. The proposed controller has been compared with a model-based reinforcement learning variable impedance controller (MBRLC) previously developed by some of the authors in order to evaluate the performance. As highlighted in the achieved results, the proposed controller is able to improve the pHRC performance. Additionally, two industrial tasks (a collaborative assembly and a collaborative deposition task) have been demonstrated to prove the applicability of the proposed solution in real industrial scenarios.

在许多情况下（例如工业和康复应用）越来越需要物理人机协作。机器人需要与人类交互以执行目标任务，同时减轻用户的工作量。为此，机器人应该能够识别人类的意图，并保证沿预期运动方向的安全和自适应行为。具有这种属性的机器人控制策略在工业领域尤其需要，在工业领域中，操作员手动引导机器人操作重型部件（例如，同时教授特定任务）。为此，本工作提出了一种基于 Q-Learning 的模型预测可变阻抗控制 (Q-LMPVIC) 来协助操作员完成物理人机协作 (pHRC) 任务。笛卡尔阻抗控制回路旨在实现解耦的顺应机器人动力学。阻抗控制参数（即，设定点和阻尼参数）然后在线优化，以最大限度地提高 pHRC 的性能。为此，设计了一组神经网络来学习人机交互动力学的建模，同时捕捉相关的不确定性。然后，模型预测控制器 (MPC) 使用派生的建模，并通过 Lyapunov 约束增强稳定性保证。MPC 是通过使用 Q-Learning 方法来解决的，该方法在其在线实现中使用 actor-critic 算法来近似精确的解决方案。事实上，Q-learning 方法提供了一个准确且高效的解决方案（在计算时间和资源方面）。所提出的方法已通过实验测试得到验证，其中以 Franka EMIKA 熊猫机器人作为测试平台。z笛卡尔方向。所提出的控制器已与一些作者先前开发的基于模型的强化学习可变阻抗控制器 (MBRLC) 进行了比较，以评估其性能。正如所取得的结果所强调的那样，所提出的控制器能够提高 pHRC 性能。此外，已经证明了两个工业任务（协同组装和协同沉积任务），以证明所提出的解决方案在实际工业场景中的适用性。