Abstract

Artificial Intelligence (AI) and Extended Reality (XR) have been employed in several foreign language education applications to increase the availability of experiential learning methods akin to international immersion programs. However, research in multi-modal spoken dialogue in L2 combined with immersive technologies and collaborative learning is thin, limiting students’ experiences to solo interactions focused mostly on vocabulary and grammar in such settings. We intend to fill this gap as we present the Cognitive Immersive Language Learning Environment (CILLE). The AI in CILLE can hear, see, and understand its users and can engage with them in non-dyadic multimodal conversations. The XR offers students a feeling of being somewhere else without the use of intrusive devices and supports multi-party, multi-modal interactions. Together, AI and XR create naturalistic conversational interactions targeted towards comprehensive foreign language acquisition. We evaluate CILLE as a Chinese-as-a-foreign-language (CFL) education tool through a seven-week, mixed-methods study with university students (N = 10). Results display statistical significance and retained improvement in CFL vocabulary, comprehension, and conversation skills. Coupled with an analysis of student feedback and researcher observations, we show how CILLE is designed and experienced by students to learn CFL.

摘要

人工智能 (AI) 和扩展现实 (XR) 已被用于多种外语教育应用程序，以增加类似于国际沉浸式课程的体验式学习方法的可用性。然而，结合沉浸式技术和协作学习的 L2 多模态口语对话的研究还很薄弱，这将学生的体验仅限于在此类环境中主要集中在词汇和语法上的单独互动。我们打算在展示认知沉浸式语言学习环境 (CILLE) 时填补这一空白。CILLE 中的 AI 可以听到、看到和理解其用户，并可以与他们进行非双向多模式对话。XR 为学生提供身临其境的感觉，无需使用侵入性设备，并支持多方、多模式交互。一起，AI 和 XR 创建了针对全面外语习得的自然对话交互。我们通过对大学生 (N = 10) 进行为期七周的混合方法研究，将 CILLE 作为一种汉语作为外语 (CFL) 教育工具进行评估。结果显示统计意义和 CFL 词汇量、理解力和会话技巧的持续改进。结合对学生反馈和研究人员观察的分析，我们展示了学生如何设计和体验 CILLE 来学习 CFL。结果显示统计意义和 CFL 词汇量、理解力和会话技巧的持续改进。结合对学生反馈和研究人员观察的分析，我们展示了学生如何设计和体验 CILLE 来学习 CFL。结果显示统计意义和 CFL 词汇量、理解力和会话技巧的持续改进。结合对学生反馈和研究人员观察的分析，我们展示了学生如何设计和体验 CILLE 来学习 CFL。