Meta-learning has been proposed as a framework to address the challenging few-shot learning setting. The key idea is to leverage a large number of similar few-shot tasks in order to learn how to adapt a base-learner to a new task for which only a few labeled samples are available. As deep neural networks (DNNs) tend to overfit using a few samples only, typical meta-learning models use shallow neural networks, thus limiting its effectiveness. In order to achieve top performance, some recent works tried to use the DNNs pre-trained on large-scale datasets but mostly in straight-forward manners, e.g., (1) taking their weights as a warm start of meta-training, and (2) freezing their convolutional layers as the feature extractor of base-learners. In this paper, we propose a novel approach called meta-transfer learning (MTL), which learns to transfer the weights of a deep NN for few-shot learning tasks. Specifically, meta refers to training multiple tasks, and transfer is achieved by learning scaling and shifting functions of DNN weights (and biases) for each task. To further boost the learning efficiency of MTL, we introduce the hard task (HT) meta-batch scheme as an effective learning curriculum of few-shot classification tasks. We conduct experiments for five-class few-shot classification tasks on three challenging benchmarks, miniImageNet, tieredImageNet, and Fewshot-CIFAR100 (FC100), in both supervised and semi-supervised settings. Extensive comparisons to related works validate that our MTL approach trained with the proposed HT meta-batch scheme achieves top performance. An ablation study also shows that both components contribute to fast convergence and high accuracy.

中文翻译：

---

通过困难任务进行元迁移学习


元学习被提议作为一个框架来解决具有挑战性的小样本学习环境。关键思想是利用大量类似的小样本任务来学习如何使基础学习器适应只有少数标记样本可用的新任务。由于深度神经网络（DNN）往往仅使用少量样本就会过度拟合，因此典型的元学习模型使用浅层神经网络，从而限制了其有效性。为了实现最佳性能，最近的一些工作尝试使用在大规模数据集上预训练的 DNN，但大多采用直接的方式，例如，（1）将其权重作为元训练的热启动，并且（ 2）将其卷积层冻结为基础学习器的特征提取器。在本文中，我们提出了一种称为元迁移学习（MTL）的新方法，它学习为少样本学习任务迁移深度神经网络的权重。具体来说，元是指训练多个任务，而迁移是通过学习每个任务的 DNN 权重（和偏差）的缩放和移位函数来实现的。为了进一步提高 MTL 的学习效率，我们引入了硬任务（HT）元批次方案作为少样本分类任务的有效学习课程。我们在监督和半监督设置中，在三个具有挑战性的基准（miniImageNet、tieredImageNet 和 Fewshot-CIFAR100 (FC100)）上进行了五类少样本分类任务的实验。与相关工作的广泛比较验证了我们用所提出的 HT 元批次方案训练的 MTL 方法实现了最佳性能。消融研究还表明，这两个组件都有助于快速收敛和高精度。