Deep neural networks (DNNs) are known to perform well when deployed to test distributions that shares high similarity with the training distribution. Feeding DNNs with new data sequentially that were unseen in the training distribution has two major challenges -- fast adaptation to new tasks and catastrophic forgetting of old tasks. Such difficulties paved way for the on-going research on few-shot learning and continual learning. To tackle these problems, we introduce Attentive Independent Mechanisms (AIM). We incorporate the idea of learning using fast and slow weights in conjunction with the decoupling of the feature extraction and higher-order conceptual learning of a DNN. AIM is designed for higher-order conceptual learning, modeled by a mixture of experts that compete to learn independent concepts to solve a new task. AIM is a modular component that can be inserted into existing deep learning frameworks. We demonstrate its capability for few-shot learning by adding it to SIB and trained on MiniImageNet and CIFAR-FS, showing significant improvement. AIM is also applied to ANML and OML trained on Omniglot, CIFAR-100 and MiniImageNet to demonstrate its capability in continual learning. Code made publicly available at https://github.com/huang50213/AIM-Fewshot-Continual.

中文翻译：

---

具有细心独立机制的小样本和持续学习

众所周知，深度神经网络 (DNN) 在部署以测试与训练分布具有高度相似性的分布时表现良好。将训练分布中看不到的新数据按顺序提供给 DNN 有两个主要挑战——快速适应新任务和灾难性地忘记旧任务。这些困难为正在进行的小样本学习和持续学习的研究铺平了道路。为了解决这些问题，我们引入了注意力独立机制（AIM）。我们将使用快速和慢速权重的学习思想与 DNN 的特征提取和高阶概念学习的解耦结合起来。AIM 是为高阶概念学习而设计的，由专家混合建模，这些专家竞争学习独立概念以解决新任务。AIM 是一个模块化组件，可以插入到现有的深度学习框架中。我们通过将其添加到 SIB 并在 MiniImageNet 和 CIFAR-FS 上进行训练，展示了它的小样本学习能力，显示出显着的改进。AIM 还应用于在 Omniglot、CIFAR-100 和 MiniImageNet 上训练的 ANML 和 OML，以展示其在持续学习方面的能力。代码在 https://github.com/huang50213/AIM-Fewshot-Continual 上公开可用。