ABSTRACT A risk assessor’s confidence has been shown to influence both the rater as well as those evaluating the risk assessment. It is important to consider the impact of confidence on accuracy in the risk assessment field given the significant implications of risk assessments for the assessed (e.g. available treatment options vs. restrictions of liberties) and the safety of the public. While prior research in the risk assessment field has used correlations to investigate the confidence-accuracy (C-A) relationship, a number of fields in psychology have introduced additional techniques, including calibration analysis, to understand this relationship. In this study, we examined the C-A relationship across various adverse clinical outcomes using the Short-Term Assessment of Risk and Treatability (START) and compared and contrasted the C-A relationship using conventional methods (i.e. correlations/ROC analysis) and calibration. Raters completed START assessments for a sample of 106 civil psychiatric inpatients. Overall, calibration provided greater detail into the C-A relationship compared to correlations/ROC analysis. Our results also suggested that the C-A relationship varied as a function of the outcome assessed (e.g. violence, substance abuse, unauthorized leave). These results provide insights into the C-A relationship in medico-legal settings and can inform best practices for risk assessment training and implementation.

中文翻译：

---

使用 START 评估风险评估中的置信度-准确度关系：引入校准分析

摘要 风险评估者的信心已被证明会影响评估者和评估风险评估的人。考虑到风险评估对被评估者（例如可用的治疗方案与自由的限制）和公众安全的重大影响，考虑信心对风险评估领域准确性的影响是很重要的。虽然风险评估领域的先前研究使用相关性来研究置信度 - 准确度 (CA) 关系，但心理学中的许多领域引入了额外的技术，包括校准分析，以了解这种关系。在这项研究中，我们使用风险和可治疗性的短期评估 (START) 检查了各种不良临床结果的 CA 关系，并使用传统方法（即相关性/ROC 分析）和校准比较和对比了 CA 关系。评估者完成了对 106 名平民精神病住院患者样本的 START 评估。总体而言，与相关性/ROC 分析相比，校准为 CA 关系提供了更多细节。我们的结果还表明，CA 关系随评估结果（例如暴力、药物滥用、未经授权的休假）而变化。这些结果提供了对法医学环境中 CA 关系的见解，并可以为风险评估培训和实施的最佳实践提供信息。评估者完成了对 106 名平民精神病住院患者样本的 START 评估。总体而言，与相关性/ROC 分析相比，校准为 CA 关系提供了更多细节。我们的结果还表明，CA 关系随评估结果（例如暴力、药物滥用、未经授权的休假）而变化。这些结果提供了对法医学环境中 CA 关系的见解，并可以为风险评估培训和实施的最佳实践提供信息。评估者完成了对 106 名平民精神病住院患者样本的 START 评估。总体而言，与相关性/ROC 分析相比，校准为 CA 关系提供了更多细节。我们的结果还表明，CA 关系随评估结果（例如暴力、药物滥用、未经授权的休假）而变化。这些结果提供了对法医学环境中 CA 关系的见解，并可以为风险评估培训和实施的最佳实践提供信息。