Robots are extending their presence in domestic environments every day, being more common to see them carrying out tasks in home scenarios. In the future, robots are expected to increasingly perform more complex tasks and, therefore, be able to acquire experience from different sources as quickly as possible. A plausible approach to address this issue is interactive feedback, where a trainer advises a learner on which actions should be taken from specific states to speed up the learning process. Moreover, deep reinforcement learning has been recently widely utilized in robotics to learn the environment and acquire new skills autonomously. However, an open issue when using deep reinforcement learning is the excessive time needed to learn a task from raw input images. In this work, we propose a deep reinforcement learning approach with interactive feedback to learn a domestic task in a human-robot scenario. We compare three different learning methods using a simulated robotic arm for the task of organizing different objects; the proposed methods are (i) deep reinforcement learning (DeepRL); (ii) interactive deep reinforcement learning using a previously trained artificial agent as an advisor (agent-IDeepRL); and (iii) interactive deep reinforcement learning using a human advisor (human-IDeepRL). We demonstrate that interactive approaches provide advantages for the learning process. The obtained results show that a learner agent, using either agent-IDeepRL or human-IDeepRL, completes the given task earlier and has fewer mistakes compared to the autonomous DeepRL approach.

中文翻译：

---

人机环境中交互式反馈的深度强化学习

机器人每天都在扩大他们在家庭环境中的存在，更常见的是他们在家庭场景中执行任务。未来，预计机器人将越来越多地执行更复杂的任务，因此能够尽快从不同来源获取经验。解决此问题的一种可行方法是交互式反馈，其中培训师建议学习者应从特定状态采取哪些行动以加快学习过程。此外，深度强化学习最近已广泛应用于机器人技术中，以自主学习环境和获取新技能。然而，使用深度强化学习时的一个悬而未决的问题是从原始输入图像中学习任务所需的时间过长。在这项工作中，我们提出了一种具有交互式反馈的深度强化学习方法，以在人机场景中学习家庭任务。我们使用模拟机械臂来比较三种不同的学习方法，用于组织不同对象的任务；提出的方法是（i）深度强化学习（DeepRL）；(ii) 使用先前训练的人工代理作为顾问的交互式深度强化学习 (agent-IDeepRL)；(iii) 使用人类顾问 (human-IDEepRL) 进行交互式深度强化学习。我们证明交互式方法为学习过程提供了优势。获得的结果表明，与自主 DeepRL 方法相比，使用 agent-IDeepRL 或 human-IDeepRL 的学习者代理更早完成给定的任务并且错误更少。