Abstract Early Warning Systems (EWS) and Early Warning Indictors (EWI) have recently emerged as an attractive domain for states and school districts interested in predicting student outcomes using data that schools already collect with the intention to better time and tailor interventions. However, current diagnostic measures used across the domain do not consider the dual issues of sensitivity and specificity of predictors, key components for considering accuracy. We apply signal detection theory using Receiver Operating Characteristic (ROC) Area Under the Curve (AUC) analysis adapted from the engineering and medical domains, and using the pROC package in R. Using nationally generalizable data from the Education Longitudinal Study of 2002 (ELS:2002) we provide examples of applying ROC accuracy analysis to a variety of predictors of student outcomes, such as dropping out of high school, college enrollment, and postsecondary STEM degrees and careers.

中文翻译：

---

曲线下的接收器工作特征（ROC）区域（AUC）：一种用于评估教育成果预测因子准确性的诊断措施

摘要早期预警系统（EWS）和早期预警指标（EWI）已成为对州和学区感兴趣的领域，这些州和学区希望利用学校已经收集的数据来预测学生的成绩，以期更好地安排时间和量身定制干预措施。但是，当前在整个领域中使用的诊断措施并未考虑到预测因子的敏感性和特异性这两个双重问题，而预测因子是考虑准确性的关键组成部分。我们将信号检测理论应用于工程和医学领域，采用曲线下的接收器工作特征（ROC）分析区域（AUC），并在R中使用pROC软件包。使用2002年教育纵向研究（ELS：