Due to the importance of zero-shot learning, i.e., classifying images where there is a lack of labeled training data, the number of proposed approaches has recently increased steadily. We argue that it is time to take a step back and to analyze the status quo of the area. The purpose of this paper is three-fold. First, given the fact that there is no agreed upon zero-shot learning benchmark, we first define a new benchmark by unifying both the evaluation protocols and data splits of publicly available datasets used for this task. This is an important contribution as published results are often not comparable and sometimes even flawed due to, e.g., pre-training on zero-shot test classes. Moreover, we propose a new zero-shot learning dataset, the Animals with Attributes 2 (AWA2) dataset which we make publicly available both in terms of image features and the images themselves. Second, we compare and analyze a significant number of the state-of-the-art methods in depth, both in the classic zero-shot setting but also in the more realistic generalized zero-shot setting. Finally, we discuss in detail the limitations of the current status of the area which can be taken as a basis for advancing it.

中文翻译：

---

零样本学习__好、坏、丑综合评价


由于零样本学习（即对缺乏标记训练数据的图像进行分类）的重要性，提出的方法的数量最近稳步增加。我们认为现在是退一步分析该地区现状的时候了。本文的目的有三个。首先，鉴于没有商定的零样本学习基准，我们首先通过统一用于此任务的公开可用数据集的评估协议和数据分割来定义一个新基准。这是一个重要的贡献，因为发布的结果通常不具有可比性，有时甚至由于零样本测试类的预训练等原因而存在缺陷。此外，我们提出了一个新的零样本学习数据集，即 Animals with Attributes 2 (AWA2) 数据集，我们在图像特征和图像本身方面都公开了该数据集。其次，我们深入比较和分析了大量最先进的方法，无论是在经典的零样本设置中，还是在更现实的广义零样本设置中。最后，我们详细讨论了该领域现状的局限性，可以作为推进该领域的基础。