ABSTRACT

Deictic expressions – such as uttering ”this tool” and pointing at a hammer – are important and frequent elements of everyday communication. They are used, for example, when conveying location and orientation information, or when identifying objects. Pointing gestures exist among all known cultures, languages and ethnicities, but they depend on parties to be collocated. When communicating across distances, deictic expressions are usually lost. In this paper, we introduce an approach for communicating deictic pointing across distances by simulating the sender hand posture in the receiver’s hand using electrical muscle stimulation (EMS). We describe our prototypical implementation of this approach and report on a lab-based user study evaluating it. Our results indicate that it is technically feasible to use EMS for communicating deictic pointing gestures across distances. Participants in our study were successful in resolving deictic expressions consisting of verbal and gestural components. Furthermore, our results provide initial insights into user behavior, user satisfaction and specific challenges in this context. Our contributions can inform the design of future interfaces for naturally transmitting pointing gestures across distances.

摘要

明确的表达方式（例如说出“此工具”并指着锤子）是日常交流的重要且频繁的元素。例如，在传送位置和方向信息或识别对象时使用它们。指向手势存在于所有已知的文化，语言和族裔中，但是它们取决于聚会对象。在远距离交流时，通常会失去言语表达。在本文中，我们通过电肌肉刺激（EMS）模拟接收者手中的发送者手部姿势，介绍了一种跨距离传达定向指向的方法。我们描述了这种方法的原型实现，并报告了基于实验室的用户研究对其进行评估。我们的结果表明，使用EMS跨距离传达专用的指向手势在技术上是可行的。我们研究的参与者成功地解决了由言语和手势组成的言语表达。此外，我们的结果提供了在这种情况下对用户行为，用户满意度和特定挑战的初步见解。我们的贡献可以为将来界面的设计提供信息，以便自然地跨距离传输指向手势。