Learning from one or few visual examples is one of the key capabilities of humans since early infancy, but is still a significant challenge for modern AI systems. While considerable progress has been achieved in few-shot learning from a few image examples, much less attention has been given to the verbal descriptions that are usually provided to infants when they are presented with a new object. In this paper, we focus on the role of additional semantics that can significantly facilitate few-shot visual learning. Building upon recent advances in few-shot learning with additional semantic information, we demonstrate that further improvements are possible by combining multiple and richer semantics (category labels, attributes, and natural language descriptions). Using these ideas, we offer the community new results on the popular miniImageNet and CUB few-shot benchmarks, comparing favorably to the previous state-of-the-art results for both visual only and visual plus semantics-based approaches. We also performed an ablation study investigating the components and design choices of our approach. Code available on github.com/EliSchwartz/mutiple-semantics.

从一个或几个视觉示例中学习是人类自婴儿早期以来的关键能力之一，但对于现代人工智能系统来说仍然是一项重大挑战。虽然在从几个图像示例中进行小样本学习方面取得了相当大的进展，但对于通常提供给婴儿的新对象时的口头描述却很少关注。在本文中，我们专注于可以显着促进少镜头视觉学习的附加语义的作用。基于最近在使用附加语义信息的少样本学习方面取得的进展，我们证明了通过结合多种和更丰富的语义（类别标签、属性和自然语言描述）。使用这些想法，我们在流行的mini ImageNet 和 CUB 少样本基准上为社区提供了新结果，与之前仅视觉和基于视觉加语义的方法的最新结果相比，具有优势。我们还进行了一项消融研究，调查我们方法的组件和设计选择。代码可在 github.com/EliSchwartz/mutiple-semantics 上找到。