In this paper, we develop a robotic telepresence system to provide remote users with immersive embodiment in local environments through a custom-designed mobile robot. The proposed telepresence system uses a virtual reality (VR) device to connect a remote user to the robot. Three dimensional visual data from a RGB-D camera are rendered for real-time stereoscopic display in the VR device, which forms a deeply-coupled human machine system and creates an immersive experience of telepresence. Based on a user study, it is found that better user experience can be achieved by allowing the robot to track the speaker while being aware of the intention of the remote user. To this end we propose a human-robot collaborative control framework based on human intention recognition and sound localization. The intentions of head movement of the remote user are inferred based on the motion of the VR device using hidden Markov models. The speaker is tracked through sound source localization using a microphone array. A collaborative control scheme is developed to fuse the control from the robot and the remote user. Experiments are conducted in both one-to-one and one-to-two remote conversation scenarios. The results show that the proposed system can significantly improve the immersiveness and performance of robotic telepresence systems, therefore greatly enhancing the user experience of such telepresence systems.

在本文中，我们开发了一种机器人远程呈现系统，通过定制设计的移动机器人为远程用户在本地环境中提供身临其境的体现。拟议的网真系统使用虚拟现实（VR）设备将远程用户连接到机器人。渲染来自RGB-D摄像机的三维视觉数据，以在VR设备中进行实时立体显示，从而形成了深度耦合的人机系统，并创造了身临其境的网真体验。根据用户研究，发现通过允许机器人在意识到远程用户的意图的同时跟踪扬声器，可以实现更好的用户体验。为此，我们提出了一种基于人类意图识别和声音本地化的人机协作控制框架。使用隐藏的马尔可夫模型基于VR设备的运动来推断远程用户的头部运动的意图。使用麦克风阵列通过声音源定位来跟踪扬声器。开发了一种协作控制方案，以融合来自机器人和远程用户的控制。实验是在一对一和一对二远程对话场景中进行的。结果表明，所提出的系统可以显着提高机器人远程呈现系统的沉浸性和性能，从而大大增强了这种远程呈现系统的用户体验。开发了一种协作控制方案，以融合来自机器人和远程用户的控制。实验是在一对一和一对二远程对话场景中进行的。结果表明，所提出的系统可以显着提高机器人远程呈现系统的沉浸性和性能，从而大大增强了这种远程呈现系统的用户体验。开发了一种协作控制方案，以融合来自机器人和远程用户的控制。实验是在一对一和一对二远程对话场景中进行的。结果表明，所提出的系统可以显着提高机器人远程呈现系统的沉浸性和性能，从而大大增强了这种远程呈现系统的用户体验。