This study was motivated by a need to understand the extent to which behavioral indicators of engagement from digital log data are associated with various student learning outcomes above and beyond self-reported levels of engagement, and whether the strength of these associations vary depending on the type of learning outcome. Student learning was assessed by way of four distinct learning outcomes that varied according to stakes (low-v. high-stakes) and span (one-time v. aggregated). Participants included high school students between 14 and 18 years of age enrolled in an AP Statistics course (N = 320, M age = 16.76 years, SD age = 0.85; 60.2% female) who had consented to use an online assessment system over the course of an academic year that was designed to provide personalized performance reports. While largely uncorrelated with self-report measures, certain process data variables were significantly correlated with learning outcomes. In particular, students’ frequency of score report checking, an indication of feedback-seeking behavior, while uncorrelated with self-reported student engagement, was associated with all learning outcomes. Other behaviors, such as the number of log-in sessions and the duration of sessions, were not. These findings suggest that process data from online assessment systems can help broaden and deepen our understanding of student behavior above and beyond self-report. That said, given that the volume and complexity of process data can make it challenging to mine and interpret, researchers must consider theory when identifying process data variables that are critical to the understanding of constructs of interest.

这项研究的动机是需要了解数字日志数据中参与的行为指标在多大程度上超出自我报告的参与水平之外的各种学生学习成果，以及这些联系的强度是否因类型而异学习成果。通过四种不同的学习成果对学生的学习进行评估，这些学习成果根据风险（低风险与高风险）和跨度（一次性风险与汇总风险）的不同而变化。参加者包括14到18岁之间的高中学生，参加了AP统计课程（N  = 320，M年龄= 16.76岁，SD年龄= 0.85; 60.2％的女性）同意在一学年内使用旨在提供个性化绩效报告的在线评估系统。虽然很大程度上与自我报告测度不相关，但某些过程数据变量与学习成果显着相关。尤其是，学生的成绩报告检查频率，即寻求反馈行为的指标，尽管与自我报告的学生参与度无关，但与所有学习成果相关。其他行为，例如登录会话数和会话持续时间则没有。这些发现表明，来自在线评估系统的过程数据可以帮助拓宽和加深我们对超出自我报告之外的学生行为的理解。那就是