Designing of touchless user interface is gaining popularity in various contexts. Using such interfaces, users can interact with electronic devices even when the hands are dirty or non-conductive. Also, user with partial physical disability can interact with electronic devices using such systems. Research in this direction has got major boost because of the emergence of low-cost sensors such as Leap Motion, Kinect or RealSense devices. In this paper, we propose a Leap Motion controller-based methodology to facilitate rendering of 2D and 3D shapes on display devices. The proposed method tracks finger movements while users perform natural gestures within the field of view of the sensor. In the next phase, trajectories are analyzed to extract extended Npen++ features in 3D. These features represent finger movements during the gestures and they are fed to unidirectional left-to-right Hidden Markov Model (HMM) for training. A one-to-one mapping between gestures and shapes is proposed. Finally, shapes corresponding to these gestures are rendered over the display using MuPad interface. We have created a dataset of 5400 samples recorded by 10 volunteers. Our dataset contains 18 geometric and 18 non-geometric shapes such as "circle", "rectangle", "flower", "cone", "sphere" etc. The proposed methodology achieves an accuracy of 92.87% when evaluated using 5-fold cross validation method. Our experiments revel that the extended 3D features perform better than existing 3D features in the context of shape representation and classification. The method can be used for developing useful HCI applications for smart display devices.

中文翻译：

---

由手势引导的 2D 显示设备上形状的视觉渲染

非接触式用户界面的设计在各种环境中越来越受欢迎。使用此类界面，即使手脏或不导电，用户也可以与电子设备进行交互。此外，部分身体残疾的用户可以使用此类系统与电子设备进行交互。由于 Leap Motion、Kinect 或 RealSense 设备等低成本传感器的出现，这方面的研究得到了重大推动。在本文中，我们提出了一种基于 Leap Motion 控制器的方法，以促进在显示设备上渲染 2D 和 3D 形状。当用户在传感器的视野内执行自然手势时，所提出的方法会跟踪手指运动。在下一阶段，分析轨迹以在 3D 中提取扩展的 Npen++ 特征。这些特征代表手势期间的手指运动，并将它们馈送到单向从左到右的隐马尔可夫模型 (HMM) 进行训练。提出了手势和形状之间的一对一映射。最后，使用 MuPad 界面在显示器上呈现与这些手势对应的形状。我们创建了一个由 10 名志愿者记录的 5400 个样本的数据集。我们的数据集包含 18 个几何形状和 18 个非几何形状，如“圆形”、“矩形”、“花”、“圆锥”、“球体”等。当使用 5 折交叉评估时，所提出的方法达到了 92.87% 的准确率验证方法。我们的实验表明，在形状表示和分类的背景下，扩展的 3D 特征比现有的 3D 特征表现更好。