Rater‐mediated assessments are a common methodology for measuring persons, investigating rater behavior, and/or defining latent constructs. The purpose of this article is to provide a pedagogical framework for examining rater variability in the context of rater‐mediated assessments using three distinct models. The first model is the observation model, which includes ecological/environmental considerations for the evaluation system. The second model is the measurement model, which includes the transformation of observed, rater response data to linear measures using a measurement model with specific requirements of rater‐invariant measurement in order to examine raters’ construct‐relevant variability stemming from the evaluative system. The third model is the interaction model, which includes an interaction parameter to allow for the investigation into raters’ systematic, construct‐irrelevant variability stemming from the evaluative system. Implications for measurement outcomes and validity are discussed.

中文翻译：

---

在评估者介导的评估中检查评估者变异性的教学考虑：一个三模型框架

评估者介导的评估是衡量人员，调查评估者行为和/或定义潜在构造的常用方法。本文的目的是提供一种教学框架，用于使用三种不同的模型在评估者介导的评估中检查评估者的变异性。第一个模型是观测模型，其中包括评估系统的生态/环境因素。第二个模型是度量模型，它包括使用对评估者不变测量有特定要求的测量模型，将观察到的评估者响应数据转换为线性度量，以便检查评估者源自评估系统的与构造相关的变异性。第三个模型是交互模型，其中包括一个交互参数，可用于调查评估者源自评估系统的系统性，与结构无关的变异性。讨论了对测量结果和有效性的影响。