ABSTRACT

In everyday group conversations, we must decide whom to pay attention to and when. This process of dynamic social attention is important for goals both perceptual and social. The present study investigated gaze during a conversation in a realistic group and in a controlled laboratory study where third-party observers watched videos of the same group. In both contexts, we explore how gaze allocation is related to turn-taking in speech. Experimental video clips were edited to either remove the sound, freeze the video, or transition to a blank screen, allowing us to determine how shifts in attention between speakers depend on visual or auditory cues. Gaze behaviour in the real, interactive situation was similar to the fixations made by observers watching a video. Eyetracked participants often fixated the person speaking and shifted gaze in response to changes in speaker, even when sound was removed or the video freeze-framed. These findings suggest we sometimes fixate the location of speakers even when no additional visual information can be gained. Our novel approach offers both a comparison of interactive and third-party viewing and the opportunity for controlled experimental manipulations. This delivers a rich understanding of gaze behaviour and multimodal attention during a conversation following.

摘要

在日常的小组对话中，我们必须决定关注谁以及何时关注。这种动态的社会关注过程对于感知和社会目标都很重要。本研究调查了现实小组对话期间的凝视，以及第三方观察者观看同一小组视频的受控实验室研究。在这两种情况下，我们都探讨了注视分配与语音轮换的关系。实验性视频剪辑被编辑为去除声音、冻结视频或过渡到空白屏幕，使我们能够确定扬声器之间的注意力转移如何依赖于视觉或听觉线索。在真实的、交互的情况下，注视行为类似于观察者观看视频时的注视。眼动追踪的参与者经常盯着说话的人，并根据说话者的变化转移视线，即使声音被移除或视频被定格。这些发现表明，即使无法获得额外的视觉信息，我们有时也会固定扬声器的位置。我们的新颖方法提供了交互式和第三方查看的比较以及受控实验操作的机会。这在随后的对话中提供了对凝视行为和多模态注意力的丰富理解。我们的新颖方法提供了交互式和第三方查看的比较以及受控实验操作的机会。这在随后的对话中提供了对凝视行为和多模态注意力的丰富理解。我们的新颖方法提供了交互式和第三方查看的比较以及受控实验操作的机会。这在随后的对话中提供了对凝视行为和多模态注意力的丰富理解。