A major gap between few-shot and many-shot learning is the data distribution empirically oserved by the model during training. In few-shot learning, the learned model can easily become over-fitted based on the biased distribution formed by only a few training examples, while the ground-truth data distribution is more accurately uncovered in many-shot learning to learn a well-generalized model. In this paper, we propose to calibrate the distribution of these few-sample classes to be more unbiased to alleviate such an over-fitting problem. The distribution calibration is achieved by transferring statistics from the classes with sufficient examples to those few-sample classes. After calibration, an adequate number of examples can be sampled from the calibrated distribution to expand the inputs to the classifier. Specifically, we assume every dimension in the feature representation from the same class follows a Gaussian distribution so that the mean and the variance of the distribution can borrow from that of similar classes whose statistics are better estimated with an adequate number of samples. Extensive experiments on three datasets, miniImageNet, tieredImageNet, and CUB, show that a simple linear classifier trained using the features sampled from our calibrated distribution can outperform the state-of-the-art accuracy by a large margin. Besides the favorable performance, the proposed method also exhibits high flexibility by showing consistent accuracy improvement when it is built on top of any off-the-shelf pretrained feature extractors and classification models without extra learnable parameters. The visualization of these generated features demonstrates that our calibrated distribution is an accurate estimation thus the generalization ability gain is convincing. We also establish a generalization error bound for the proposed distribution-calibration-based few-shot learning, which consists of the distribution assumption error, the distribution approximation error, and the estimation error. This generalization error bound theoretically justifies the effectiveness of the proposed method.

中文翻译：

---

通过分布校准弥合少样本学习和多样本学习之间的差距


少样本学习和多样本学习之间的主要差距是模型在训练过程中根据经验观察到的数据分布。在少样本学习中，基于仅由少数训练样本形成的偏差分布，学习到的模型很容易变得过度拟合，而在多样本学习中，可以更准确地揭示真实数据分布，以学习泛化良好的模型。模型。在本文中，我们建议将这些少样本类的分布校准为更加公正，以缓解这种过度拟合问题。分布校准是通过将统计数据从具有足够示例的类转移到那些样本较少的类来实现的。校准后，可以从校准分布中采样足够数量的示例，以扩展分类器的输入。具体来说，我们假设同一类的特征表示中的每个维度都遵循高斯分布，以便分布的均值和方差可以借鉴类似类的平均值和方差，这些类的统计数据可以通过足够数量的样本更好地估计。对三个数据集（miniImageNet、tieredImageNet 和 CUB）的广泛实验表明，使用从我们的校准分布中采样的特征训练的简单线性分类器可以大幅超越最先进的精度。除了良好的性能之外，所提出的方法还表现出高度的灵活性，当它建立在任何现成的预训练特征提取器和分类模型之上而无需额外的可学习参数时，它表现出一致的精度改进。这些生成的特征的可视化表明我们的校准分布是准确的估计，因此泛化能力的增益是令人信服的。 我们还为所提出的基于分布校准的小样本学习建立了泛化误差界，其中包括分布假设误差、分布近似误差和估计误差。这种泛化误差界限从理论上证明了所提出方法的有效性。