Assessing students’ participation in science practices presents several challenges, especially when aiming to differentiate meaningful (vs. rote) forms of participation. In this study, we sought to use machine learning (ML) for a novel purpose in science assessment: developing a construct map for students’ consideration of generality, a key epistemic understanding that undergirds meaningful participation in knowledge-building practices. We report on our efforts to assess the nature of 845 students’ ideas about the generality of their model-based explanations through the combination of an embedded written assessment and a novel data analytic approach that combines unsupervised and supervised machine learning methods and human-driven, interpretive coding. We demonstrate how unsupervised machine learning methods, when coupled with qualitative, interpretive coding, were used to revise our construct map for generality in a way that allowed for a more nuanced evaluation that was closely tied to empirical patterns in the data. We also explored the application of the construct map as a framework for coding used as a part of supervised machine learning methods, finding that it demonstrates some viability for use in future analyses. We discuss implications for the assessment of students’ meaningful participation in science practices in terms of their considerations of generality, the role of unsupervised methods in science assessment, and combining machine learning and human-driven approach for understanding students’ complex involvement in science practices.

评估学生对科学实践的参与提出了若干挑战，尤其是在试图区分有意义的（相对于死记硬背）参与形式时。在这项研究中，我们试图将机器学习（ML）用于科学评估的新目的：为学生考虑普遍性而开发构造图，这是一种重要的认识论理解，有助于有意义地参与知识构建实践。我们报告了通过整合嵌入式书面评估和新颖的数据分析方法（结合无监督和监督的机器学习方法以及人为驱动的方法）来评估845名学生基于模型的解释的一般性的努力，解释性编码。我们展示了无监督的机器学习方法，结合定性，解释性编码，如何用于修正通用性的构造图，从而允许更细微的评估与数据的经验模式密切相关。我们还探讨了构造图作为编码框架的应用，该框架被用作监督机器学习方法的一部分，发现它证明了在将来的分析中使用的可行性。我们从普遍性的考虑，无监督方法在科学评估中的作用以及将机器学习和人为驱动的方法相结合来理解学生对科学实践的复杂参与等方面，讨论对评估学生有意义地参与科学实践的意义。