We address the problem of solving complex bimanual robot manipulation tasks on multiple objects with sparse rewards. Such complex tasks can be decomposed into sub-tasks that are accomplishable by different robots concurrently or sequentially for better efficiency. While previous reinforcement learning approaches primarily focus on modeling the compositionality of sub-tasks, two fundamental issues are largely ignored particularly when learning cooperative strategies for two robots: (i) domination, i.e., one robot may try to solve a task by itself and leaves the other idle; (ii) conflict, i.e., one robot can easily interrupt another's workspace when executing different sub-tasks simultaneously. To tackle these two issues, we propose a novel technique called disentangled attention, which provides an intrinsic regularization for two robots to focus on separate sub-tasks and objects. We evaluate our method on four bimanual manipulation tasks. Experimental results show that our proposed intrinsic regularization successfully avoids domination and reduces conflicts for the policies, which leads to significantly more effective cooperative strategies than all the baselines. Our project page with videos is at https://mehooz.github.io/bimanual-attention.

中文翻译：

---

解开注意力作为双手多对象操作的内在正则化

我们解决了在具有稀疏奖励的多个对象上解决复杂的双手机器人操作任务的问题。这些复杂的任务可以分解为子任务，由不同的机器人同时或顺序完成，以提高效率。虽然以前的强化学习方法主要侧重于对子任务的组合进行建模，但在学习两个机器人的合作策略时，两个基本问题在很大程度上被忽略了：(i) 支配，即一个机器人可能试图自己解决任务并离开另一个空闲；(ii) 冲突，即一个机器人在同时执行不同的子任务时很容易打断另一个机器人的工作空间。为了解决这两个问题，我们提出了一种称为分离注意力的新技术，它为两个机器人提供了内在的正则化，以专注于单独的子任务和对象。我们在四个双手操作任务上评估我们的方法。实验结果表明，我们提出的内在正则化成功地避免了支配并减少了政策的冲突，这导致了比所有基线更有效的合作策略。我们的视频项目页面位于 https://mehooz.github.io/bimanual-attention。