Abstract

The accuracy of peer ratings on students’ performance has attracted much attention from higher education researchers. In this study, we attempted to explore the accuracy of peer ratings on the quality of spoken-language interpreting in the context of tertiary-level interpreter training. We sought to understand how different types of peer raters (experienced versus novice) would affect the accuracy of their ratings on three quality dimensions (fidelity, fluency and expression) of bi-directional interpretation (English-to-Chinese and Chinese-to-English). Our data analysis, based on a rater accuracy model, shows that: (a) overall, it was easier for the peer raters to score the English-to-Chinese interpretation accurately than the opposite direction and to assess fidelity accurately than fluency and expression; (b) the experienced peer raters scored the interpretations more accurately than their novice counterparts; and (c) the higher-quality interpretations were easier to be scored accurately than those of lower quality. However, the directionality of interpreting may moderate these relationships. These findings are expected to enrich and deepen our understanding of rater accuracy as a function of distal and proximal factors in the Brunswik’s lens model, and to help us better organize peer assessment in the higher educational context.

摘要

同伴评价对学生成绩的准确性引起了高等教育研究人员的广泛关注。在这项研究中，我们试图在高等教育口译培训的背景下探索同行评价对口语口译质量的准确性。我们试图了解不同类型的同行评估者（经验丰富的与新手）如何影响他们在双向口译（英译中和中译英）的三个质量维度（保真度、流畅度和表达）上的评分准确性）。我们基于评分者准确率模型的数据分析表明：(a) 总体而言，同行评分者对英汉口译的准确评分比反方向更容易，准确评估保真度比流利度和表达更容易；(b) 有经验的同行评估者比他们的新手同行更准确地对解释进行评分；(c) 高质量的解释比低质量的解释更容易准确评分。然而，口译的方向性可能会缓和这些关系。这些发现有望丰富和加深我们对作为 Brunswik 镜头模型中远端和近端因素函数的评分者准确性的理解，并帮助我们在高等教育环境中更好地组织同行评估。