While deep learning and deep reinforcement learning (RL) systems have demonstrated impressive results in domains such as image classification, game playing, and robotic control, data efficiency remains a major challenge. Multi-task learning has emerged as a promising approach for sharing structure across multiple tasks to enable more efficient learning. However, the multi-task setting presents a number of optimization challenges, making it difficult to realize large efficiency gains compared to learning tasks independently. The reasons why multi-task learning is so challenging compared to single-task learning are not fully understood. In this work, we identify a set of three conditions of the multi-task optimization landscape that cause detrimental gradient interference, and develop a simple yet general approach for avoiding such interference between task gradients. We propose a form of gradient surgery that projects a task's gradient onto the normal plane of the gradient of any other task that has a conflicting gradient. On a series of challenging multi-task supervised and multi-task RL problems, this approach leads to substantial gains in efficiency and performance. Further, it is model-agnostic and can be combined with previously-proposed multi-task architectures for enhanced performance.

中文翻译：

---

多任务学习的梯度手术

虽然深度学习和深度强化学习 (RL) 系统在图像分类、游戏和机器人控制等领域取得了令人瞩目的成果，但数据效率仍然是一个重大挑战。多任务学习已成为跨多个任务共享结构以实现更有效学习的有前途的方法。然而，多任务设置提出了许多优化挑战，与独立学习任务相比，难以实现大的效率提升。与单任务学习相比，多任务学习如此具有挑战性的原因尚不完全清楚。在这项工作中，我们确定了多任务优化环境中导致有害梯度干扰的一组三个条件，并开发一种简单而通用的方法来避免任务梯度之间的这种干扰。我们提出了一种梯度手术形式，将任务的梯度投影到任何其他具有冲突梯度的任务的梯度的法线平面上。在一系列具有挑战性的多任务监督和多任务 RL 问题上，这种方法可以显着提高效率和性能。此外，它与模型无关，可以与先前提出的多任务架构相结合以提高性能。这种方法大大提高了效率和性能。此外，它与模型无关，可以与先前提出的多任务架构相结合以提高性能。这种方法大大提高了效率和性能。此外，它与模型无关，可以与先前提出的多任务架构相结合以提高性能。