Abstract

A fundamental requirement of language assessments which is underresearched in computerized assessments is impartiality (fairness) or equal treatment of test takers regardless of background. The present study aimed to evaluate fairness in the Pearson Test of English (PTE) Academic Reading test, which is a computerized reading assessment, by investigating differential item functioning (DIF) across Indo-European (IE) and Non-Indo-European (NIE) language families. Previous research has shown that similarities between readers’ mother tongue and the second language being learned can advantage some test takers. To test this hypothesis, we analyzed data from 783 international test takers who took the PTE Academic test, using the partial credit model in Rasch measurement. We examined two main types of DIF: uniform DIF (UDIF), which occurs when an item consistently gives a particular group of test takers an advantage across all levels of ability, and non-uniform DIF (NUDIF), which occurs when the performance of test takers varies across the ability continuum. The results showed no statistically significant UDIF (p > 0.05), but identified 3 NUDIF items out of 10 items across the language families. A mother tongue advantage was not observed. Similarity in test takers’ level of computer and Internet skills, test preparation, and language policies could contribute to the finding of no UDIF. Post-hoc content analysis of items suggested that the decrease of mother tongue advantage for IE groups in high-proficiency groups and lucky guesses of low-ability groups may have contributed to the emergence of NUDIF items. Lastly, recommendations for investigating social and contextual factors are proposed.

摘要

在计算机化评估中未充分研究的语言评估的一个基本要求是公正（公平）或无论背景如何都应平等对待应试者。本研究旨在通过调查印欧语 (IE) 和非印欧语 (NIE) 的差异项目功能 (DIF) 来评估皮尔逊英语考试 (PTE) 学术阅读测试的公平性，这是一种计算机化的阅读评估。 ) 语系。以前的研究表明，读者的母语和正在学习的第二语言之间的相似性可以使一些考生受益。为了验证这一假设，我们使用 Rasch 测量中的部分学分模型分析了 783 名参加 PTE Academic 考试的国际考生的数据。我们检查了两种主要类型的 DIF：统一 DIF (UDIF)，当一个项目在所有能力水平上始终为一组特定的应试者提供优势时，就会发生这种情况，而当应试者的表现在能力连续体中发生变化时，就会发生非均匀 DIF (NUDIF)。结果显示没有统计学上显着的UDIF（p  > 0.05），但在整个语言系列的 10 个项目中识别了 3 个 NUDIF 项目。没有观察到母语优势。应试者的计算机和互联网技能水平、考试准备和语言政策的相似性可能有助于发现没有 UDIF。项目的事后内容分析表明，高水平组的 IE 组的母语优势下降和低能力组的幸运猜测可能是 NUDIF 项目出现的原因。最后，提出了调查社会和背景因素的建议。