Vision-based manipulation has been largely used in various robot applications. Normally, in order to obtain the spatial information of the operated target, a carefully calibrated stereo vision system is required. However, it limits the application of robots in the unstructured environment which limits both the number and the pose of the camera. In this study, a calibration-free monocular vision-based robot manipulation approach is proposed based on domain randomization and deep reinforcement learning (DRL). Firstly, a learning strategy combined domain randomization is developed to estimate the spatial information of the target from a single monocular camera arbitrarily mounted in a large area of the manipulation environment. Secondly, to address the monocular occlusion problem which regularly happens during robot manipulations, an occlusion awareness DRL policy has been designed to control the robot to avoid occlusions actively in the manipulation tasks. The performance of our method has been evaluated on two common manipulation tasks, reaching and lifting of a target building block, which show the efficiency and effectiveness of our proposed approach.

中文翻译：

---

具有遮挡感知功能的免校准单目视觉机器人操作


基于视觉的操纵已广泛应用于各种机器人应用中。通常，为了获得操作目标的空间信息，需要仔细校准的立体视觉系统。然而，它限制了机器人在非结构化环境中的应用，从而限制了相机的数量和姿态。在本研究中，提出了一种基于域随机化和深度强化学习（DRL）的免校准单目视觉机器人操纵方法。首先，开发了一种结合域随机化的学习策略，以从任意安装在大面积操作环境中的单个单目相机估计目标的空间信息。其次，为了解决机器人操作过程中经常发生的单眼遮挡问题，设计了遮挡感知DRL策略来控制机器人在操作任务中主动避免遮挡。我们的方法的性能已经在两个常见的操作任务（到达和提升目标构建块）上进行了评估，这表明了我们提出的方法的效率和有效性。