Abstract

Recent years have witnessed an increasing interest in the use of automated writing evaluation (AWE) in second language writing classrooms. This increase is partially due to the belief that AWE can assist teachers by allowing them to devote more feedback to higher-level (HL) writing skills, such as content and organization, while the technology addresses lower-level (LL) skills, such as grammar. As is speculated, student revisions will then be positively impacted. However, little evidence has supported these claims, calling into question the impact of AWE on teaching and learning. The current study explored these claims by comparing two second language writing classes that were assigned to either an AWE + teacher feedback condition or a teacher-only-feedback condition. Findings suggest that using AWE as a complement to teacher feedback did not have a significant impact on the amount of HL teacher feedback, but the teacher who did not use AWE tended to provide a greater amount of LL feedback than AWE alone. Furthermore, students seemed to revise the teacher’s LL feedback more frequently than LL feedback from the computer. Interestingly, students retained their improvement in accuracy in the long-term when they had access to AWE, but students who did not have access appeared to have lower retention. We explain the relevance of our findings in relation to an argument-based validation framework to align our work with state-of-the-art research in the field and contribute to a broader discussion about how AWE can be best provided to support second language writing development.

摘要

近年来，人们越来越关注在第二语言写作课堂中使用自动写作评估 (AWE)。这种增长的部分原因是相信 AWE 可以帮助教师通过允许他们将更多的反馈投入到更高级别 (HL) 的写作技能上，例如内容和组织，而该技术解决了较低级别 (LL) 的技能，例如语法。正如推测的那样，学生的修订将受到积极影响。然而，几乎没有证据支持这些说法，从而质疑 AWE 对教学和学习的影响。目前的研究通过比较分配给 AWE + 教师反馈条件或仅教师反馈条件的两个第二语言写作课程来探索这些主张。研究结果表明，使用 AWE 作为教师反馈的补充对 HL 教师反馈的数量没有显着影响，但不使用 AWE 的教师往往比单独使用 AWE 提供更多的 LL 反馈。此外，与来自计算机的 LL 反馈相比，学生似乎更频繁地修改老师的 LL 反馈。有趣的是，当学生可以访问 AWE 时，他们的准确性长期保持在提高，但没有访问权限的学生的保留率似乎较低。我们解释了我们的发现与基于论证的验证框架的相关性，以使我们的工作与该领域的最新研究保持一致，并有助于更广泛地讨论如何最好地提供 AWE 以支持第二语言写作发展。