ABSTRACT

A long-standing debate in the testing of listening concerns the authenticity of the listening input. On the one hand, listening texts produced by item writers often lack spoken language characteristics. On the other hand, real-life recordings are often too context-specific to stand alone, or not suitable for item generation. In this study, we explored the effectiveness of an existing item-writing training course to produce authentic-sounding listening texts within the constraints of test specifications. Twenty-five trainees took an online item-writing course including training on creating authentic-sounding listening texts. Prior to and after the course, they developed a listening task. The resulting listening texts were judged on authenticity by three professional item reviewers and analysed linguistically by the researchers. Additionally, we interviewed the trainees following each item writing event and analysed their online discussions from during the course. Statistical comparison of the pre-and post-course authenticity scores revealed a positive effect of the training on item-writers’ ability to produce authentic-sounding listening texts, while the linguistic analysis demonstrated that the texts produced after the training contained more instances of spoken language. The interviews and discussions revealed that item writers’ awareness of spoken language features and their text production techniques influenced their ability to develop authentic-sounding texts.

摘要

听力测试中的一个长期争论涉及听力输入的真实性。一方面，项目作者制作的听力文本往往缺乏口语特征。另一方面，现实生活中的录音通常过于特定于上下文而不能独立存在，或者不适合项目生成。在这项研究中，我们探讨了现有项目编写培训课程在测试规范的约束下生成听起来真实的听力文本的有效性。25 名学员参加了在线项目编写课程，包括创建听起来真实的听力文本的培训。在课程之前和之后，他们制定了一项听力任务。所产生的听力文本由三位专业项目审查员根据真实性进行判断，并由研究人员进行语言分析。此外，我们在每个项目写作活动之后采访了受训者，并分析了他们在课程期间的在线讨论。课前和课后真实性分数的统计比较显示，培训对项目编写者生成听起来真实的听力文本的能力有积极影响，而语言分析表明，培训后生成的文本包含更多的口语实例语。访谈和讨论表明，项目作者对口语特征的认识和他们的文本制作技术影响了他们开发真实发声文本的能力。课前和课后真实性分数的统计比较显示，培训对项目编写者生成听起来真实的听力文本的能力有积极影响，而语言分析表明，培训后生成的文本包含更多的口语实例语。访谈和讨论表明，项目作者对口语特征的认识和他们的文本制作技术影响了他们开发真实发声文本的能力。课前和课后真实性分数的统计比较显示，培训对项目编写者生成听起来真实的听力文本的能力有积极影响，而语言分析表明，培训后生成的文本包含更多的口语实例语。访谈和讨论表明，项目作者对口语特征的认识和他们的文本制作技术影响了他们开发真实发声文本的能力。