当前位置: X-MOL 学术Journal of Educational Measurement › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Conceptualizing Rater Judgments and Rating Processes for Rater‐Mediated Assessments
Journal of Educational Measurement ( IF 1.188 ) Pub Date : 2019-07-16 , DOI: 10.1111/jedm.12226
Jue Wang 1 , George Engelhard 2
Affiliation  

Rater‐mediated assessments exhibit scoring challenges due to the involvement of human raters. The quality of human ratings largely determines the reliability, validity, and fairness of the assessment process. Our research recommends that the evaluation of ratings should be based on two aspects: a theoretical model of human judgment and an appropriate measurement model for evaluating these judgments. In rater‐mediated assessments, the underlying constructs and response processes may require the use of different rater judgment models and the application of different measurement models. We describe the use of Brunswik's lens model as an organizing theme for conceptualizing human judgments in rater‐mediated assessments. The constructs vary depending on which distal variables are identified in the lens models for the underlying rater‐mediated assessment. For example, one lens model can be developed to emphasize the measurement of student proficiency, while another lens model can stress the evaluation of rater accuracy. Next, we describe two measurement models that reflect different response processes (cumulative and unfolding) from raters: Rasch and hyperbolic cosine models. Future directions for the development and evaluation of rater‐mediated assessments are suggested.

中文翻译:

概念化评估者判断和评估过程,以评估者为中介的评估

由于人类评估者的参与,评估者介导的评估显示出计分挑战。人工评估的质量在很大程度上决定了评估过程的可靠性,有效性和公平性。我们的研究建议对评级的评估应基于两个方面:人类判断的理论模型和评估这些判断的适当度量模型。在评估者介导的评估中,潜在的构建和响应过程可能需要使用不同的评估者判断模型和不同的度量模型。我们将Brunswik的镜头模型描述为组织者,用于概念化评估者介导的评估中的人类判断。结构的变化取决于在晶状体模型中识别出哪些远端变量,以进行基础的评估者介导的评估。例如,可以开发一种镜片模型来强调学生熟练程度的测量,而另一种镜片模型可以强调评估者准确性的评估。接下来,我们描述两个反映评估者不同响应过程(累积和展开)的测量模型:Rasch模型和双曲余弦模型。建议了评估者介导的评估的发展和评估的未来方向。Rasch和双曲余弦模型。建议了评估者介导的评估的发展和评估的未来方向。Rasch和双曲余弦模型。建议了评估者介导的评估的发展和评估的未来方向。
更新日期:2019-07-16
down
wechat
bug