Industrial robots can perform delicate operations, and they have good stability and durability. However, robots adapt with difficulty to changes in tasks and environments, and they basically can only perform operations in a fixed logic sequence. In contrast, humans can adapt to changes in the environment at any time due to their strong hand–eye coordination. Humans can quickly adjust their limbs to adapt to changes in targets, distances, and directions, as observed by their visual system; this produces a perfect closed-loop control process called the sensor–actor process. This paper studies the robotic hand–eye coordination problem using a robot playing an eight-puzzle game. First, the robot’s system analyzes changes in the position, angle, and layout of the puzzle board in the scene through an image recognition algorithm. It then formulates a more optimized operation sequence. Next, the system transforms the instructions of moving tiles into the physical coordinate values in the world coordinate system according to the image-understanding result. The robot then moves its hand to accurately point at tiles and move them in the correct direction. These tasks require the robot to perceive various changes in the position, posture, and initial layout of the puzzle board in the scene, and to calculate the motion vector parameters. To validate the proposed approach, field experiments are conducted. The success rate of the move operation was over 96%, which shows that this system based on visual perception can greatly improve the adaptability of the robot, making it more flexible and autonomous, and laying the foundation for expanding the robot’s ability to work in spontaneous scenes.

工业机器人可以执行精细的操作，并且具有良好的稳定性和耐用性。但是，机器人很难适应任务和环境的变化，基本上只能按照固定的逻辑顺序进行操作。相比之下，人类由于手眼协调能力强，可以随时适应环境的变化。人类可以快速调整四肢以适应视觉系统观察到的目标、距离和方向的变化；这产生了一个完美的闭环控制过程，称为传感器-行动者过程。本文使用机器人玩八拼游戏来研究机器人手眼协调问题。首先，机器人的系统通过图像识别算法分析场景中拼图板的位置、角度和布局的变化。然后制定更优化的操作序列。接下来，系统根据图像理解结果将移动瓷砖的指令转化为世界坐标系中的物理坐标值。然后机器人移动它的手以准确地指向瓷砖并沿正确的方向移动它们。这些任务需要机器人感知场景中拼图板的位置、姿态和初始布局的各种变化，并计算运动矢量参数。为了验证所提出的方法，进行了现场实验。移动操作的成功率超过96%，说明这套基于视觉感知的系统可以大大提高机器人的适应性，使其更加灵活和自主，