In many real-world classification problems, accurate prediction of membership probabilities is critical for further decision making. The probability calibration problem studies how to map scores obtained from one classification algorithm to membership probabilities. The requirement of non-decreasingness for this mapping involves an infinite number of inequality constraints, which makes its estimation computationally intractable. For the sake of this difficulty, existing methods failed to achieve four desiderata of probability calibration: universal flexibility, non-decreasingness, continuousness and computational tractability. This paper proposes a method with shape-restricted polynomial regression, which satisfies all four desiderata. In the method, the calibrating function is approximated with monotone polynomials, and the continuously-constrained requirement of monotonicity is equivalent to some semidefinite constraints. Thus, the calibration problem can be solved with tractable semidefinite programs. This estimator is both strongly and weakly universally consistent under a trivial condition. Experimental results on both artificial and real data sets clearly show that the method can greatly improve calibrating performance in terms of reliability-curve related measures.

中文翻译：

---

使用形状限制多项式回归校准分类概率


在许多现实世界的分类问题中，准确预测隶属概率对于进一步决策至关重要。概率校准问题研究如何将从一种分类算法获得的分数映射到隶属概率。该映射的非递减性要求涉及无限数量的不等式约束，这使得其估计在计算上变得困难。由于这个困难，现有的方法无法实现概率校准的四个要求：通用灵活性、非递减性、连续性和计算易处理性。本文提出了一种形状限制多项式回归的方法，它满足所有四个需求。该方法中，标定函数用单调多项式来逼近，单调性的连续约束要求等价于一些半定约束。因此，标定问题可以用易于处理的半定程序来解决。该估计量在平凡条件下具有强普遍一致和弱普遍一致。人工和真实数据集的实验结果清楚地表明，该方法可以极大地提高可靠性曲线相关措施的校准性能。