Strictly proper scoring rules (SPSR) are incentive compatible for eliciting information about random variables from strategic agents when the principal can reward agents after the realization of the random variables. They also quantify the quality of elicited information, with more accurate predictions receiving higher scores in expectation. In this paper, we extend such scoring rules to settings where a principal elicits private probabilistic beliefs but only has access to agents' reports. We name our solution \emph{Surrogate Scoring Rules} (SSR). SSR build on a bias correction step and an error rate estimation procedure for a reference answer defined using agents' reports. We show that, with a single bit of information about the prior distribution of the random variables, SSR in a multi-task setting recover SPSR in expectation, as if having access to the ground truth. Therefore, a salient feature of SSR is that they quantify the quality of information despite the lack of ground truth, just as SPSR do for the setting \emph{with} ground truth. As a by-product, SSR induce \emph{dominant truthfulness} in reporting. Our method is verified both theoretically and empirically using data collected from real human forecasters.

中文翻译：

---

代理评分规则

当委托人可以在随机变量实现后奖励代理人时，严格适当的评分规则 (SPSR) 是激励兼容的，用于从战略代理人那里获取有关随机变量的信息。他们还量化了引出信息的质量，更准确的预测会获得更高的期望分数。在本文中，我们将此类评分规则扩展到委托人引发私人概率信念但只能访问代理报告的设置。我们将我们的解决方案命名为 \emph{Surrogate Scoring Rules} (SSR)。SSR 建立在偏差校正步骤和使用代理报告定义的参考答案的错误率估计程序之上。我们表明，通过有关随机变量先验分布的一点信息，多任务设置中的 SSR 可以在期望中恢复 SPSR，就好像可以访问基本事实一样。因此，SSR 的一个显着特征是，尽管缺乏基本事实，它们仍能量化信息的质量，就像 SPSR 对设置 \emph{with} 基本事实所做的那样。作为副产品，SSR 在报告中诱导\emph {显性真实}。我们的方法使用从真实的人类预测者收集的数据在理论上和经验上得到验证。