In this paper, an imitation learning approach of vision guided reaching skill is proposed for robotic precision manipulation, which enables the robot to adapt its end-effector’s nonlinear motion with the awareness of collision-avoidance. The reaching skill model firstly uses the raw images of objects as inputs, and generates the incremental motion command to guide the lower-level vision-based controller. The needle’s tip is detected in image space and the obstacle region is extracted by image segmentation. A neighborhood-sampling method is designed for needle component collision perception, which includes a neural networks based attention module. The neural network based policy module infers the desired motion in the image space according to the neighborhood-sampling result, goal and current positions of the needle’s tip. A refinement module is developed to further improve the performance of the policy module. In three dimensional (3D) manipulation tasks, typically two cameras are used for image-based vision control. Therefore, considering the epipolar constraint, the relative movements in two cameras’ views are refined by optimization. Experimental are conducted to validate the effectiveness of the proposed methods.

本文提出了一种视觉引导到达技术的模仿学习方法，用于机器人的精确操纵，使机器人能够在避免碰撞的情况下适应其末端执行器的非线性运动。到达技能模型首先使用对象的原始图像作为输入，并生成增量运动命令以指导下层基于视觉的控制器。在图像空间中检测出针尖，并通过图像分割提取障碍物区域。设计了一种用于针组件碰撞感知的邻域采样方法，该方法包括基于神经网络的注意力模块。基于神经网络的策略模块根据邻域采样结果，目标和针尖的当前位置来推断图像空间中的所需运动。开发了改进模块以进一步改善策略模块的性能。在三维（3D）操纵任务中，通常使用两个摄像机进行基于图像的视觉控制。因此，考虑到极线约束，可以通过优化来优化两个摄像机视图中的相对运动。进行实验以验证所提出方法的有效性。