Abstract This study proposes a new shared mixed reality (MR)-bilateral telerobotic system. The main contribution of this study is to combine MR teleoperation and bilateral teleoperation, which takes advantage of the two types of teleoperation and compensates for each other’s drawbacks. With this combination, the proposed system can address the asymmetry issues in bilateral teleoperation, such as kinematic redundancy and workspace inequality, and provide force feedback, which is lacking in MR teleoperation. In addition, this system effectively supports long-distance movements and fine movements. In this system, a new MR interface is developed to provide the operator with an immersive visual feedback of the workspace, in which a useful virtual controller known as an interaction proxy—is designed. Compared with previous virtual reality-based teleoperation systems, this interaction proxy can freely decouple the operator from the control loop, such that the operational burden can be substantially alleviated. Additionally, the force feedback provided by the bilateral teleoperation gives the operator an advanced perception about the remote workspace and can improve task performance. Experiments on multiple pick-and-place tasks are provided to demonstrate the feasibility and effectiveness of the proposed system.

中文翻译：

---

共享混合现实-双边遥控机器人系统

摘要 本研究提出了一种新的共享混合现实 (MR)-双边遥控机器人系统。本研究的主要贡献是将MR遥操作和双边遥操作结合起来，利用了两种遥操作的优势，弥补了彼此的不足。通过这种组合，所提出的系统可以解决双边遥操作中的不对称问题，例如运动冗余和工作空间不平等，并提供 MR 遥操作所缺乏的力反馈。此外，该系统有效支持长距离动作和精细动作。在该系统中，开发了一个新的 MR 界面，为操作员提供沉浸式的工作空间视觉反馈，其中设计了一个有用的虚拟控制器，称为交互代理。与以往基于虚拟现实的遥操作系统相比，这种交互代理可以将操作者与控制回路自由解耦，从而大大减轻操作负担。此外，双边遥操作提供的力反馈使操作员对远程工作空间有了更深入的了解，并可以提高任务性能。提供了多个拾取和放置任务的实验，以证明所提出系统的可行性和有效性。