ABSTRACT Machine learning (ML) is an emergent computerised technology that relies on algorithms built by ‘learning’ from training data rather than ‘instruction’, which holds great potential to revolutionise science assessment. This study systematically reviewed 49 articles regarding ML-based science assessment through a triangle framework with technical, validity, and pedagogical features on three vertices. We found that a majority of the studies focused on the validity vertex, as compared to the other two vertices. The existing studies primarily involve text recognition, classification, and scoring with an emphasis on constructing scientific explanations, with a vast range of human-machine agreement measures. To achieve the agreement measures, most of the studies employed a cross-validation method, rather than self- or split-validation. ML allows complex assessments to be used by teachers without the burden of human scoring, saving both time and cost. Most studies used supervised ML, which relies on extraction of attributes from student work that was first coded by humans to achieve automaticity, rather than semi- or unsupervised ML. We found that 24 studies were explicitly embedded in science learning activities, such as scientific inquiry and argumentation, to provide feedback or learning guidance. This study identifies existing research gaps and suggests that all three vertices of the ML triangle should be addressed in future assessment studies, with an emphasis on the pedagogy and technology features.

中文翻译：

---

机器学习在科学评估中的应用：系统评价

摘要 机器学习 (ML) 是一种新兴的计算机化技术，它依赖于通过从训练数据中“学习”而不是“指令”构建的算法，它具有彻底改变科学评估的巨大潜力。本研究通过在三个顶点上具有技术、有效性和教学特征的三角形框架系统地审查了 49 篇关于基于机器学习的科学评估的文章。我们发现，与其他两个顶点相比，大多数研究都集中在有效性顶点上。现有的研究主要涉及文本识别、分类和评分，重点是构建科学解释，具有广泛的人机一致性措施。为了实现一致性措施，大多数研究采用了交叉验证方法，而不是自我验证或拆分验证。ML 允许教师使用复杂的评估，而无需人工评分，从而节省时间和成本。大多数研究使用监督机器学习，它依赖于从最初由人类编码的学生作业中提取属性来实现自动化，而不是半监督或无监督机器学习。我们发现 24 项研究明确嵌入到科学学习活动中，例如科学探究和论证，以提供反馈或学习指导。这项研究确定了现有的研究空白，并建议在未来的评估研究中应解决 ML 三角形的所有三个顶点，重点放在教学法和技术特征上。它依赖于从最初由人类编码的学生作业中提取属性来实现自动化，而不是半监督或无监督的机器学习。我们发现 24 项研究明确嵌入到科学学习活动中，例如科学探究和论证，以提供反馈或学习指导。这项研究确定了现有的研究空白，并建议在未来的评估研究中应解决 ML 三角形的所有三个顶点，重点放在教学法和技术特征上。它依赖于从最初由人类编码的学生作业中提取属性来实现自动化，而不是半监督或无监督的机器学习。我们发现 24 项研究明确嵌入到科学学习活动中，例如科学探究和论证，以提供反馈或学习指导。这项研究确定了现有的研究空白，并建议在未来的评估研究中应解决 ML 三角形的所有三个顶点，重点放在教学法和技术特征上。