Abstract

We consider the common setting where one observes probability estimates for a large number of events, such as default risks for numerous bonds. Unfortunately, even with unbiased estimates, selecting events corresponding to the most extreme probabilities can result in systematically underestimating the true level of uncertainty. We develop an empirical Bayes approach “excess certainty adjusted probabilities” (ECAP), using a variant of Tweedie’s formula, which updates probability estimates to correct for selection bias. ECAP is a flexible nonparametric method, which directly estimates the score function associated with the probability estimates, so it does not need to make any restrictive assumptions about the prior on the true probabilities. ECAP also works well in settings where the probability estimates are biased. We demonstrate through theoretical results, simulations, and an analysis of two real world datasets, that ECAP can provide significant improvements over the original probability estimates. Supplementary materials for this article are available online.

摘要

我们考虑观察大量事件的概率估计的常见设置，例如大量债券的违约风险。不幸的是，即使采用无偏估计，选择与最极端概率相对应的事件也可能导致系统地低估不确定性的真实水平。我们开发了一种经验贝叶斯方法“过度确定性调整概率”（ECAP），使用 Tweedie 公式的变体，它更新概率估计以纠正选择偏差。ECAP 是一种灵活的非参数方法，它直接估计与概率估计相关的得分函数，因此它不需要对真实概率的先验做出任何限制性假设。ECAP 在概率估计有偏差的情况下也很有效。我们通过理论结果、模拟和对两个现实世界数据集的分析证明，ECAP 可以比原始概率估计提供显着改进。本文的补充材料可在线获取。