ABSTRACT

Mobile Multimodal Interaction aims at exploiting complementary aspects of human communication capacities and new mobile sensors. Recently, most mobile applications are limited to a basic interaction modality, namely touchscreen, which is subject to restricted interaction under certain situations. In this paper, we present On-the-Fly Interaction Editor (OFIE), an application that allows mobile end-users to define and integrate sensor-based unimodal and multimodal input interactions in their already installed applications according to their contexts. OFIE is based on the Event-Condition-Action rules and Programming By Demonstration approach that allows end-users to demonstrate their expected action simply by performing it on the application’s interface. We evaluated OFIE through a controlled user study. Our evaluation involves 15 participants distributed on 4 groups based on their programming experience. Each participant was invited to integrate six input interactions (three multimodal inputs). The initial results show that end-users are able to successfully integrate sensor-based input interactions using OFIE.

摘要

移动多式联运旨在利用人类通信能力和新型移动传感器的互补方面。最近，大多数移动应用程序仅限于基本的交互方式，即触摸屏，在某些情况下会受到受限的交互作用。在本文中，我们介绍了即时互动编辑器（OFIE），该应用程序允许移动最终用户根据其上下文在其已安装的应用程序中定义和集成基于传感器的单峰和多峰输入交互。OFIE基于事件条件操作规则和按演示编程方法，该方法允许最终用户仅通过在应用程序的界面上执行操作即可演示其预期的操作。我们通过受控用户研究评估了OFIE。我们的评估涉及15位参与者，根据他们的编程经验，他们分为4个小组。邀请每个参与者整合六个输入交互（三个多模式输入）。初步结果表明，最终用户能够使用OFIE成功集成基于传感器的输入交互。