Students’ experience is used in empirical software engineering research as well as in software engineering education to group students in either homogeneous or heterogeneous groups. To do so, students are commonly asked to self-rate their experience, as self-rated experience has been shown to be a good predictor for performance in programming tasks. Another experience-related measurement is participants’ confidence (i.e., how confident is the person that their given answer is correct). Hence, self-rated experience and confidence are used as selector or control variables throughout empirical software engineering research and software engineering education. In this paper, we analyze data from several student experiments conducted in the past years to investigate whether self-rated experience and confidence are also good predictors for students’ performance in model comprehension tasks. Our results show that while students can somewhat assess the correctness of a particular answer to one concrete question regarding a conceptual model (i.e., their confidence), their overall self-rated experience does not correlate with their actual performance. Hence, the use of the commonly used measurement of self-rated experience as a selector or control variable must be considered unreliable for model comprehension tasks.

学生的经验用于实证软件工程研究以及软件工程教育，以将学生分为同质或异质组。为此，通常要求学生对他们的经验进行自我评价，因为自我评价的经验已被证明是编程任务表现的良好预测指标。另一个与经验相关的衡量标准是参与者的信心（即，人们对他们给出的答案是正确的信心程度）。因此，在整个经验软件工程研究和软件工程教育中，自评经验和信心被用作选择器或控制变量。在本文中，我们分析了过去几年进行的几个学生实验的数据，以调查自评经验和信心是否也是学生在模型理解任务中表现的良好预测指标。我们的结果表明，虽然学生可以在一定程度上评估一个关于概念模型的具体问题的特定答案的正确性（即他们的信心），但他们的整体自评经验与他们的实际表现并不相关。因此，对于模型理解任务，必须认为使用常用的自评经验测量作为选择器或控制变量是不可靠的。他们的整体自评体验与他们的实际表现不相关。因此，对于模型理解任务，必须认为使用常用的自评经验测量作为选择器或控制变量是不可靠的。他们的整体自评体验与他们的实际表现不相关。因此，对于模型理解任务，必须认为使用常用的自评经验测量作为选择器或控制变量是不可靠的。