Contribution: A rubric for assessing the systems thinking expressed in conceptual models of technological systems has been constructed and assessed using a formal methodology. The rubric, a synthesis of prior findings in science and engineering education, forms a framework for improving communication between science and engineering educators. Background: Systems thinking is an important skill in engineering, but to date no rubric for assessing this skill based on a formal methodology has been published with reliability and validity measures. Research Questions: (a) What attributes should a rubric for assessing the systems thinking of engineering students comprise? (b) To what extent can such a rubric serve in assessing the systems thinking level of engineering students, as expressed in their conceptual system models? Methodology: Based on a literature review of systems thinking assessment in science and engineering education, the authors classified the systems thinking attributes they had identified into system function, structure, and behavior. Scoring instructions were developed for conceptual system models based on Object-Process Methodology ISO 19450, a formal methodology and language for model-based systems engineering. A total of 142 undergraduate engineering students in 32 teams modeled Web-based systems using the methodology. Each team submitted two models of the same system, at the middle and end of semester. Models were scored using the rubric, and its reliability and validity were evaluated. Findings: Indications of interrater reliability, internal consistency, and construct validity were acceptable, excluding the system function aspect, implying the rubric may be used reliably for its intended purpose.

中文翻译：

---

基于模型的系统思维：评估工程学生团队

贡献：已使用正式方法构建并评估了用于评估技术系统概念模型中表达的系统思维的量表。该标题综合了科学和工程教育的先前发现，形成了一个框架，以改善科学和工程教育者之间的交流。背景：系统思维是工程中的一项重要技能，但迄今为止，尚未发布基于正式方法论评估此技能的标准，其中包含可靠性和有效性措施。研究问题： (a) 评估工程专业学生系统思维的量规应该包括哪些属性？(b) 这种量规在多大程度上可以用于评估工程学生的系统思维水平，正如他们的概念系统模型所表达的那样？方法：基于对科学和工程教育中系统思维评估的文献回顾，作者将他们确定的系统思维属性分类为系统功能、结构和行为。评分说明是为基于对象过程方法 ISO 19450 的概念系统模型开发的，这是一种用于基于模型的系统工程的正式方法和语言。共有 32 个团队的 142 名本科工程专业学生使用该方法对基于 Web 的系统进行建模。每个团队在学期中和学期末提交了相同系统的两个模型。使用量规对模型进行评分，并评估其信度和效度。结果：评价者间信度、内部一致性和结构效度的指标是可以接受的，不包括系统功能方面，