Performance weighted aggregation of expert judgments, using calibration questions, has been advocated to improve pooled quantitative judgments for ecological questions. However, there is little discussion or practical advice in the ecological literature regarding the application, advantages or challenges of performance weighting. In this paper we (1) illustrate how the IDEA protocol with four‐step question format can be extended to include performance weighted aggregation from the Classical Model, and (2) explore the extent to which this extension improves pooled judgments for a range of performance measures. Our case study demonstrates that performance weights can improve judgments derived from the IDEA protocol with four‐step question format. However, there is no a‐priori guarantee of improvement. We conclude that the merits of the method lie in demonstrating that the final aggregation of judgments provides the best representation of uncertainty (i.e., validation), whether that be via equally weighted or performance weighted aggregation. Whether the time and effort entailed in performance weights can be justified is a matter for decision‐makers. Our case study outlines the rationale, challenges, and benefits of performance weighted aggregations. It will help to inform decisions about the deployment of performance weighting and avoid common pitfalls in its application.

中文翻译：

---

加权和汇总专家生态判断。

提倡使用校准问题对专家判断进行性能加权聚合，以改进对生态问题的汇总定量判断。然而，生态学文献中几乎没有关于性能加权的应用、优势或挑战的讨论或实用建议。在本文中，我们 (1) 说明了如何将具有四步问题格式的 IDEA 协议扩展为包括来自经典模型的性能加权聚合，以及 (2) 探讨这种扩展在多大程度上改进了一系列性能的汇总判断措施。我们的案例研究表明，性能权重可以通过四步问题格式改进从 IDEA 协议得出的判断。然而，没有先验改进的保证。我们得出结论，该方法的优点在于证明判断的最终聚合提供了不确定性（即验证）的最佳表示，无论是通过同等加权聚合还是性能加权聚合。绩效权重所花费的时间和精力是否合理是决策者的问题。我们的案例研究概述了性能加权聚合的基本原理、挑战和好处。它将有助于为有关性能加权部署的决策提供信息，并避免其应用程序中的常见陷阱。