This paper investigates the use of Deep Reinforcement Learning (DRL) applied to the humanoid robot soccer environment, where a robot must learn from basic to complex skills while it interacts with the environment through images received by its own camera. To do so, the Dueling Double DQN algorithm is used: it receives the images from the robot’s camera and decides on which discrete action should be performed, such as walk forward, turn to the left or kick the ball. The first experiments were performed in a robotic simulator in which the robot could learn, with DRL, three different tasks: to walk towards the ball, to act like a penalty taker and to act like a goalkeeper. In the second experiment, the learning obtained in the task to walk towards the ball was transferred to a real humanoid robot and a similar behavior could be observed, even though the environment was not exactly the same when the domain was changed. Results showed that it is possible to use DRL to learn tasks related to the role of a humanoid robot-soccer player, such as goalkeeper and penalty taker.

本文研究了深度强化学习 (DRL) 在人形机器人足球环境中的应用，在该环境中，机器人必须从基本技能到复杂技能进行学习，同时通过自己的相机接收到的图像与环境进行交互。为此，使用了 Duling Double DQN 算法：它从机器人的摄像头接收图像并决定应该执行哪个离散动作，例如向前走、向左转或踢球。第一个实验是在机器人模拟器中进行的，其中机器人可以通过 DRL 学习三种不同的任务：走向球、表现得像点球手和像守门员一样。在第二个实验中，在向球行走的任务中获得的学习被转移到一个真正的人形机器人上，并且可以观察到类似的行为，即使更改域时环境并不完全相同。结果表明，可以使用 DRL 来学习与类人机器人足球运动员角色相关的任务，例如守门员和罚球者。