Abstract Researchers have recommended involving domain experts in the design of scoring rubrics of language for specific purpose tests by eliciting profession-relevant, indigenous criteria and applying these to test performances (see, e.g., Douglas, 2001; Jacoby, 1998; Pill, 2016). However, these indigenous criteria, derived as they are from people outside the assessment field, may be difficult to apply by the non-domain expert raters typically employed to rate performances on language tests. This paper addresses this question with reference to the writing component of the Occupational English Test (OET), a test designed to assess the English communication skills of overseas-trained health professionals. The paper describes the development of a set of professionally-relevant writing descriptors and then explores how well language-trained raters (N = 15) were able to apply these to a set of OET writing samples. All raters were interviewed and the rating data were analysed statistically. The findings show that while the statistical properties of the score data were generally satisfactory, some of the raters felt that they were not able to apply the scale confidently due to their perceived lack of medical knowledge. The study has implications for scale design, rater training and the use of professionally-relevant rating scales for LSP testing purposes.

中文翻译：

---

“我将为语法而战”：探索受过语言培训的评分者实施与专业相关的写作评分量表的能力

摘要 研究人员建议让领域专家参与设计特定目的测试的语言评分标准，方法是引出与职业相关的本土标准并将这些标准应用于测试表现（参见，例如，Douglas，2001；Jacoby，1998；Pill，2016） . 然而，这些源自评估领域之外的人的本土标准可能难以被通常用于评估语言测试表现的非领域专家评估者应用。本文参考职业英语考试 (OET) 的写作部分来回答这个问题，该考试旨在评估在海外培训的卫生专业人员的英语沟通技巧。该论文描述了一组与专业相关的写作描述符的开发，然后探讨了受过语言训练的评分者（N = 15）如何能够将这些应用于一组 OET 写作样本。对所有评分者进行了访谈，并对评分数据进行了统计分析。结果表明，虽然评分数据的统计特性总体上令人满意，但一些评分者认为由于缺乏医学知识，他们无法自信地应用该量表。该研究对量表设计、评估员培训和将专业相关的评估量表用于 LSP 测试具有影响。结果表明，虽然评分数据的统计特性总体上令人满意，但一些评分者认为由于缺乏医学知识，他们无法自信地应用该量表。该研究对量表设计、评估员培训和将专业相关的评估量表用于 LSP 测试具有影响。结果表明，虽然评分数据的统计特性总体上令人满意，但一些评分者认为由于缺乏医学知识，他们无法自信地应用该量表。该研究对量表设计、评估员培训和将专业相关的评估量表用于 LSP 测试具有影响。