Abstract In recent years gesture recognition has become an increasingly interesting topic for both research and industry. While interaction with a device through a gestural interface is a promising idea in several applications especially in the industrial field, some of the issues related to the task are still considered a challenge. In the scientific literature, a relevant amount of work has been recently presented on the problem of detecting and classifying gestures from 3D hands’ joints trajectories that can be captured by cheap devices installed on head-mounted displays and desktop computers. The methods proposed so far can achieve very good results on benchmarks requiring the offline supervised classification of segmented gestures of a particular kind but are not usually tested on the more realistic task of finding gestures execution within a continuous hand tracking session. In this paper, we present a novel benchmark, SFINGE 3D, aimed at evaluating online gesture detection and recognition. The dataset is composed of a dictionary of 13 segmented gestures used as a training set and 72 trajectories each containing 3–5 of the 13 gestures, performed in continuous tracking, padded with random hand movements acting as noise. The presented dataset, captured with a head-mounted Leap Motion device, is particularly suitable to evaluate gesture detection methods in a realistic use-case scenario, as it allows the analysis of online detection performance on heterogeneous gestures, characterized by static hand pose, global hand motions, and finger articulation. We exploited SFINGE 3D to compare two different approaches for the online detection and classification, one based on visual rendering and Convolutional Neural Networks and the other based on geometry-based handcrafted features and dissimilarity-based classifiers. We discuss the results, analyzing strengths and weaknesses of the methods, and deriving useful hints for their improvement.

中文翻译：

---

SFINGE 3D：一种从 3D 手指轨迹在线检测和识别异构手势的新基准

摘要 近年来，手势识别已成为研究和工业界越来越感兴趣的话题。虽然通过手势界面与设备交互在一些应用中是一个很有前途的想法，尤其是在工业领域，但与任务相关的一些问题仍然被认为是一个挑战。在科学文献中，最近提出了大量关于从 3D 手关节轨迹检测和分类手势的问题，这些手势可以由安装在头戴式显示器和台式计算机上的廉价设备捕获。迄今为止提出的方法可以在需要对特定类型的分段手势进行离线监督分类的基准测试上取得非常好的结果，但通常不会在连续手部跟踪会话中寻找手势执行的更现实任务上进行测试。在本文中，我们提出了一种新颖的基准测试 SFINGE 3D，旨在评估在线手势检测和识别。该数据集由用作训练集的 13 个分段手势和 72 个轨迹组成，每个轨迹包含 13 个手势中的 3-5 个，在连续跟踪中执行，填充随机手部动作作为噪声。所提供的数据集是用头戴式 Leap Motion 设备捕获的，特别适合评估现实用例场景中的手势检测方法，因为它允许分析异构手势的在线检测性能，其特征是静态手部姿势、全局手部运动和手指关节。我们利用 SFINGE 3D 来比较两种不同的在线检测和分类方法，一种基于视觉渲染和卷积神经网络，另一种基于基于几何的手工特征和基于差异的分类器。我们讨论结果，分析方法的优点和缺点，并为它们的改进得出有用的提示。一种基于视觉渲染和卷积神经网络，另一种基于基于几何的手工特征和基于差异的分类器。我们讨论结果，分析方法的优点和缺点，并为它们的改进得出有用的提示。一种基于视觉渲染和卷积神经网络，另一种基于基于几何的手工特征和基于差异的分类器。我们讨论结果，分析方法的优点和缺点，并为它们的改进得出有用的提示。