ABSTRACT

The rapid evolution of virtual reality (VR) technologies and their adoption for learning opens up new possibilities for shifting semiotic content across modes, with underexplored scope for transmediating content in visual, haptic, and auditory ways in immersive media literacy practices. This research investigated users’ creative digital designing involving a popular, three-dimensional virtual painting program with upper elementary students who used a VR head mounted display and sensors. The analysis attended to how students transmediated the same story across written, verbal, and virtual painting modes, tracing key themes of the students’ virtual experience: (i) immersion and three-dimensionality, (ii) subjective presence, (iii) sensory illusion, and (iv) interactivity with motion tracking. Students reassembled and shifted narrative content, sometimes seamlessly, while experiencing ambiguity and complexity about three-dimensional representation in an immersive world. Producing stories across modes invited adaptation and generative thinking to overcome the lack of equivalence between writing, drawing, and virtual painting modes.

摘要

虚拟现实 (VR) 技术的快速发展及其在学习中的采用为跨模式转换符号内容开辟了新的可能性，在沉浸式媒体素养实践中以视觉、触觉和听觉方式传递内容的空间尚未得到充分探索。这项研究调查了用户的创意数字设计，包括一个流行的 3D 虚拟绘画程序，以及使用 VR 头戴式显示器和传感器的高年级小学生。该分析关注学生如何通过书面、口头和虚拟绘画模式转换同一个故事，追踪学生虚拟体验的关键主题：(i) 沉浸感和三维度，(ii) 主观存在，(iii) 感官错觉，以及（iv）与运动跟踪的交互性。学生重新组合和转换叙事内容，有时是无缝的，同时在沉浸式世界中体验三维表示的模糊性和复杂性。跨模式制作故事需要适应和生成思维，以克服写作、绘画和虚拟绘画模式之间缺乏对等的问题。