Aiming to solve the problem that the contact force at a robot end effector when tracking an unknown curved-surface workpiece is difficult to keep constant, a robot force control algorithm based on reinforcement learning is proposed. In this paper, a contact model and force mapping relationship are established for a robot end effector and surface. For the problem that the tangential angle of the workpiece surface is difficult to obtain in the mapping relationship, a neural network is used to identify the tangential angle of the unknown curved-surface workpiece. To keep the normal force of the robot end effector constant, a compensation term is added to a traditional explicit force controller to adapt to the robot constant force tracking scenario. For the problem that the compensation term parameters are difficult to select, the reinforcement learning algorithm A2C (advantage actor critic) is used to find the optimal parameters, and the return function and state values are modified in the A2C algorithm to satisfy the robot tracking scenario. The results show that the neural network algorithm has a good recognition effect on the tangential angle of the curved surface. The force error between the normal force and the expected force is substantially within ± 2 N after 60 iterations of the robot force control algorithm based on A2C; additionally, the variance of the force error decreases by 50.7%, 34.05% and 79.41%, respectively, compared with the force signals obtained by a fuzzy iterative algorithm and an explicit force control with two sets of fixed control parameters.

为解决机器人跟踪未知曲面工件时机器人末端执行器的接触力难以保持恒定的问题，提出了一种基于强化学习的机器人力控制算法。在本文中，建立了机器人末端执行器和表面的接触模型和力映射关系。针对映射关系难以求得工件表面切向角的问题，采用神经网络识别未知曲面工件的切向角。为了使机器人末端执行器的法向力保持恒定，将补偿项添加到传统的显式力控制器中，以适应机器人恒力跟踪情况。对于补偿项参数难以选择的问题，使用强化学习算法A2C（优势演员评论家）找到最佳参数，并在A2C算法中修改返回函数和状态值以满足机器人的跟踪情况。结果表明，神经网络算法对曲面切向角具有良好的识别效果。在基于A2C的机器人力控制算法进行60次迭代后，法向力与预期力之间的力误差基本在±2 N之内；此外，与通过模糊迭代算法和带有两组固定控制参数的显式力控制所获得的力信号相比，力误差的方差分别降低了50.7％，34.05％和79.41％。并在A2C算法中修改了返回函数和状态值，以满足机器人跟踪的情况。结果表明，神经网络算法对曲面切向角具有良好的识别效果。在基于A2C的机器人力控制算法进行60次迭代后，法向力和预期力之间的力误差基本在±2 N之内；此外，与通过模糊迭代算法和带有两组固定控制参数的显式力控制所获得的力信号相比，力误差的方差分别降低了50.7％，34.05％和79.41％。并在A2C算法中修改了返回函数和状态值，以满足机器人跟踪的情况。结果表明，神经网络算法对曲面切向角具有良好的识别效果。在基于A2C的机器人力控制算法进行60次迭代后，法向力和预期力之间的力误差基本在±2 N之内；此外，与通过模糊迭代算法和带有两组固定控制参数的显式力控制所获得的力信号相比，力误差的方差分别降低了50.7％，34.05％和79.41％。结果表明，神经网络算法对曲面切向角具有良好的识别效果。在基于A2C的机器人力控制算法进行60次迭代后，法向力和预期力之间的力误差基本在±2 N之内；此外，与通过模糊迭代算法和具有两组固定控制参数的显式力控制获得的力信号相比，力误差的方差分别减少了50.7％，34.05％和79.41％。结果表明，神经网络算法对曲面切向角具有良好的识别效果。在基于A2C的机器人力控制算法进行60次迭代后，法向力和预期力之间的力误差基本在±2 N之内；此外，与通过模糊迭代算法和带有两组固定控制参数的显式力控制所获得的力信号相比，力误差的方差分别降低了50.7％，34.05％和79.41％。