We study a unique natural experiment, during which 5-10% of draft opinions by judges of the Board of Veterans Appeals (BVA) were randomly selected for “quality review” by a team of full-time staff attorneys for nearly 15 years. This performance program had the express goals of measuring accuracy and reducing reversal rates on appeal. In cases of legal error, the quality review team wrote memoranda to judges to permit correction before opinions were issued. We use rich internal administrative data on nearly 600,000 cases from 2002-2016 to provide the first rigorous study of this review process. With precise estimates, we show that the program had no appreciable effect on reducing appeals or reversals. Based on internal records, we demonstrate that this inefficacy is likely by design, as meeting the performance measure of “accuracy” was at cross-purposes with error correction. These findings inform longstanding questions of law, organization, and bureaucracy, including performance management, standards of review, and the institutional design of mass adjudication. ∗This manuscript is authors’ original version. A revised version will be published in the Journal of Law, Economics, and Organization. We thank Reid Whitaker for research assistance and Daryl Levinson, James Ridgway, and Bill Simon for helpful comments and conversations. †Corresponding Author, William Benjamin Scott and Luna M. Scott Professor of Law, Professor (by courtesy) of Political Science; & Senior Fellow at Stanford Institute for Economic Policy Research; Stanford University, 559 Nathan Abbott Way, Stanford, CA 94305; Tel: 650-723-9560; Fax: 650-725-0253; Email: dho@law.stanford.edu ‡Research Fellow, Stanford Law School, 559 Nathan Abbott Way, Stanford, CA 94305; Ph.D. student, Department of Political Science, Stanford University §Director, Appellate Operations at Bergmann and Moore, LLC; Former-Chief of Office of Quality Assurance, Board of Veterans’ Appeals, U.S. Department of Veterans Affairs; 7920 Norfolk Ave #700, Bethesda, MD 20814 ¶Professor of Law, UCLA School of Law, 385 Charles E. Young Drive East, Los Angeles, CA 90095

中文翻译：

---

大规模裁决的质量审查：退伍军人上诉委员会的随机自然实验，2003–16年

我们研究了一个独特的自然实验，在此过程中，由资深专职律师团队随机挑选了退伍军人上诉委员会（BVA）法官的意见草案的5-10％进行“质量审查”，历时近15年。这项绩效计划的明确目标是衡量准确性并降低上诉的撤销率。如果发生法律错误，质量审查小组会向法官写备忘录，以便在发表意见之前进行更正。我们使用2002-2016年间近60万个案例的丰富内部行政数据，对该审查过程进行了首次严格的研究。通过精确的估算，我们表明该程序在减少上诉或撤销请求方面没有明显的效果。根据内部记录，我们证明这种无效性可能是设计使然，因为要达到“准确性”的绩效衡量标准与纠错是互用的。这些发现为长期存在的法律，组织和官僚主义问题提供了信息，包括绩效管理，审查标准和大规模裁决的制度设计。*此手稿是作者的原始版本。修订版将在《法律，经济学和组织杂志》上发表。我们感谢里德·惠特克（Reid Whitaker）的研究协助，以及达里尔·莱文森（Daryl Levinson），詹姆斯·里奇韦（James Ridgway）和比尔·西蒙（Bill Simon）的有益评论和对话。†通讯作者，威廉·本杰明·斯科特（William Benjamin Scott）和卢娜·M·斯科特（Luna M.Scott）法学教授，政治学教授（由礼貌提供）；斯坦福大学经济政策研究所高级研究员；斯坦福大学（Stanford University），地址：559 Nathan Abbott Way，斯坦福，CA 94305；电话：650-723-9560；传真：650-725-0253；电子邮件：dho@law.stanford.edu‡斯坦福大学法学院研究学者，地址：559 Nathan Abbott Way，Stanford，CA 94305；博士 斯坦福大学政治学系学生§伯格曼和摩尔有限责任公司上诉业务总监；美国退伍军人事务部退伍军人上诉委员会质量保证办公室前任主任；马里兰州贝塞斯达市7920 Norfolk Ave＃700，¶加州大学洛杉矶分校法学院法学教授，Charles E.Young Drive East 385，洛杉矶，CA 90095