We extend the theory of PAC learning in a way which allows to model a rich variety of learning tasks where the data satisfy special properties that ease the learning process. For example, tasks where the distance of the data from the decision boundary is bounded away from zero. The basic and simple idea is to consider partial concepts: these are functions that can be undefined on certain parts of the space. When learning a partial concept, we assume that the source distribution is supported only on points where the partial concept is defined. This way, one can naturally express assumptions on the data such as lying on a lower dimensional surface or margin conditions. In contrast, it is not at all clear that such assumptions can be expressed by the traditional PAC theory. In fact we exhibit easy-to-learn partial concept classes which provably cannot be captured by the traditional PAC theory. This also resolves a question posed by Attias, Kontorovich, and Mansour 2019. We characterize PAC learnability of partial concept classes and reveal an algorithmic landscape which is fundamentally different than the classical one. For example, in the classical PAC model, learning boils down to Empirical Risk Minimization (ERM). In stark contrast, we show that the ERM principle fails in explaining learnability of partial concept classes. In fact, we demonstrate classes that are incredibly easy to learn, but such that any algorithm that learns them must use an hypothesis space with unbounded VC dimension. We also find that the sample compression conjecture fails in this setting. Thus, this theory features problems that cannot be represented nor solved in the traditional way. We view this as evidence that it might provide insights on the nature of learnability in realistic scenarios which the classical theory fails to explain.

中文翻译：

---

部分概念类的PAC可学习性理论

我们以某种方式扩展了 PAC 学习的理论，允许对丰富多样的学习任务进行建模，其中数据满足简化学习过程的特殊属性。例如，数据与决策边界的距离远离零的任务。基本而简单的想法是考虑部分概念：这些是可以在空间的某些部分未定义的函数。在学习部分概念时，我们假设仅在定义部分概念的点上支持源分布。通过这种方式，人们可以自然地表达对数据的假设，例如位于较低维度的表面或边缘条件。相比之下，这些假设能否用传统的 PAC 理论表达，则完全不清楚。事实上，我们展示了易于学习的部分概念类，传统的 PAC 理论无法证明这些类。这也解决了 Attias、Kontorovich 和 Mansour 2019 年提出的一个问题。我们描述了部分概念类的 PAC 可学习性，并揭示了一种与经典算法根本不同的算法格局。例如，在经典的 PAC 模型中，学习归结为经验风险最小化 (ERM)。与此形成鲜明对比的是，我们表明 ERM 原则无法解释部分概念类的可学习性。事实上，我们展示了非常容易学习的类，但是任何学习它们的算法都必须使用具有无界 VC 维的假设空间。我们还发现样本压缩猜想在此设置中失败。因此，该理论的特点是无法以传统方式表示或解决的问题。我们将此视为证据，表明它可能会提供有关经典理论无法解释的现实场景中可学习性本质的见解。