The Defense Advanced Research Projects Agency (DARPA) programme ‘Systematizing Confidence in Open Research and Evidence' (SCORE) aims to generate confidence scores for a large number of research claims from empirical studies in the social and behavioural sciences. The confidence scores will provide a quantitative assessment of how likely a claim will hold up in an independent replication. To create the scores, we follow earlier approaches and use prediction markets and surveys to forecast replication outcomes. Based on an initial set of forecasts for the overall replication rate in SCORE and its dependence on the academic discipline and the time of publication, we show that participants expect replication rates to increase over time. Moreover, they expect replication rates to differ between fields, with the highest replication rate in economics (average survey response 58%), and the lowest in psychology and in education (average survey response of 42% for both fields). These results reveal insights into the academic community's views of the replication crisis, including for research fields for which no large-scale replication studies have been undertaken yet.

国防高级研究计划局 (DARPA) 计划“开放研究和证据的置信度系统化”(SCORE) 旨在为社会和行为科学领域的实证研究中的大量研究主张生成置信度分数。置信度分数将定量评估某个主张在独立重复中成立的可能性。为了创建分数，我们遵循早期的方法并使用预测市场和调查来预测复制结果。根据对 SCORE 整体复制率的初步预测及其对学科和出版时间的依赖性，我们表明参与者预计复制率会随着时间的推移而增加。此外，他们预计不同领域的重复率会有所不同，其中经济学领域的重复率最高（调查平均响应率为 58%），心理学和教育领域的重复率最低（两个领域的平均调查响应均为 42%）。这些结果揭示了学术界对复制危机的看法，包括尚未进行大规模复制研究的研究领域。