In the criminal legal context, risk assessment algorithms are touted as data-driven, well-tested tools. Studies known as validation tests are typically cited by practitioners to show that a particular risk assessment algorithm has predictive accuracy, establishes legitimate differences between risk groups, and maintains some measure of group fairness in treatment. To establish these important goals, most tests use a one-shot, single-point measurement. Using a Polya Urn model, we explore the implication of feedback effects in sequential scoring-decision processes. We show through simulation that risk can propagate over sequential decisions in ways that are not captured by one-shot tests. For example, even a very small or undetectable level of bias in risk allocation can amplify over sequential risk-based decisions, leading to observable group differences after a number of decision iterations. Risk assessment tools operate in a highly complex and path-dependent process, fraught with historical inequity. We conclude from this study that these tools do not properly account for compounding effects, and require new approaches to development and auditing.

中文翻译：

---

重复使用刑事风险评估中的反馈效应

在刑事法律环境中，风险评估算法被吹捧为数据驱动的，经过良好测试的工具。从业人员通常会引用称为验证测试​​的研究，以表明特定的风险评估算法具有预测准确性，可在风险组之间建立合理的差异，并在治疗中维持组公平性的某种衡量标准。为了建立这些重要目标，大多数测试使用单次单点测量。使用Polya Urn模型，我们探索了反馈效应在顺序评分决策过程中的含义。我们通过仿真表明，风险可以通过一次性测试无法捕获的方式在顺序决策中传播。例如，即使是很小或无法检测到的风险分配偏差水平，也可能会放大基于顺序风险的决策，在多次决策迭代后导致可观察到的组差异。风险评估工具在高度复杂且依赖路径的过程中运作，充满了历史的不平等性。我们从这项研究得出的结论是，这些工具不能适当地考虑到复合效应，需要开发和审计的新方法。