当前位置: X-MOL 学术arXiv.cs.LG › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
TAG: Task-based Accumulated Gradients for Lifelong learning
arXiv - CS - Machine Learning Pub Date : 2021-05-11 , DOI: arxiv-2105.05155
Pranshu Malviya, Balaraman Ravindran, Sarath Chandar

When an agent encounters a continual stream of new tasks in the lifelong learning setting, it leverages the knowledge it gained from the earlier tasks to help learn the new tasks better. In such a scenario, identifying an efficient knowledge representation becomes a challenging problem. Most research works propose to either store a subset of examples from the past tasks in a replay buffer, dedicate a separate set of parameters to each task or penalize excessive updates over parameters by introducing a regularization term. While existing methods employ the general task-agnostic stochastic gradient descent update rule, we propose a task-aware optimizer that adapts the learning rate based on the relatedness among tasks. We utilize the directions taken by the parameters during the updates by accumulating the gradients specific to each task. These task-based accumulated gradients act as a knowledge base that is maintained and updated throughout the stream. We empirically show that our proposed adaptive learning rate not only accounts for catastrophic forgetting but also allows positive backward transfer. We also show that our method performs better than several state-of-the-art methods in lifelong learning on complex datasets with a large number of tasks.

中文翻译:

TAG:终身学习的基于任务的累积渐变

当代理在终身学习环境中遇到新的连续任务流时,它会利用从较早任务中获得的知识来帮助更好地学习新任务。在这种情况下,识别有效的知识表示形式将成为一个具有挑战性的问题。大多数研究工作都建议将过去任务的示例子集存储在重播缓冲区中,为每个任务分配单独的参数集,或者通过引入正则化术语来惩罚过度的参数更新。虽然现有方法采用与任务无关的通用随机梯度下降更新规则,但我们提出了一种任务感知优化器,该优化器根据任务之间的相关性来调整学习率。通过累积特定于每个任务的梯度,我们利用了更新期间参数所采用的方向。这些基于任务的累积渐变充当知识库,在整个流中对其进行维护和更新。我们凭经验表明,我们提出的自适应学习率不仅解决了灾难性的遗忘,而且还允许积极的向后转移。我们还表明,在对具有大量任务的复杂数据集进行终生学习时,我们的方法比几种最先进的方法性能更好。
更新日期:2021-05-12
down
wechat
bug