Abstract

In most states and jurisdictions, risk assessments are incorporated into justice system practice. Despite decades of use, the methods of tool development are rarely translated to the field. Many agencies implement ‘off-the-shelf’ versions, where a tool developed with a unique set of methods and subjects demonstrates prediction shrinkage when applied to a new jurisdiction. Using a large, 10-state sample of assessed youth (N=494,050), we isolate, test, and evaluate the relative impact of notable risk assessment variations, including: item selection, response weighting, outcome definition/duration, and jurisdiction. We further combined approaches to evaluate an ‘optimized’ development approach. Findings revealed substantial gains with each variation tested, where optimized models provided a full effect size predictive improvement. We discuss best practices for the future of risk assessment, noting the predictive accuracy lost when implementing tools off-the-shelf, and describe how optimization techniques substantially improve risk prediction, specifying a given tool to an agency’s needs.

摘要

在大多数州和司法管辖区，风险评估被纳入司法系统实践。尽管使用了数十年，但工具开发方法很少被转化为该领域。许多机构实施“现成”版本，其中使用一组独特的方法和主题开发的工具在应用于新司法管辖区时显示出预测收缩。我们使用来自 10 个州的被评估青年的大型样本（N=494,050），分离、测试和评估显着风险评估变化的相对影响，包括：项目选择、响应权重、结果定义/持续时间和管辖权。我们进一步结合了评估“优化”开发方法的方法。研究结果显示，每个测试变体都有显着的收益，其中优化的模型提供了完整的效果大小预测改进。