Online hand gesture recognition (HGR) techniques are essential in augmented reality (AR) applications for enabling natural human-to-computer interaction and communication. In recent years, the consumer market for low-cost AR devices has been rapidly growing, while the technology maturity in this domain is still limited. Those devices are typical of low prices, limited memory, and resource-constrained computational units, which makes online HGR a challenging problem. To tackle this problem, we propose a lightweight and computationally efficient HGR framework, namely LE-HGR, to enable real-time gesture recognition on embedded devices with low computing power. We also show that the proposed method is of high accuracy and robustness, which is able to reach high-end performance in a variety of complicated interaction environments. To achieve our goal, we first propose a cascaded multi-task convolutional neural network (CNN) to simultaneously predict probabilities of hand detection and regress hand keypoint locations online. We show that, with the proposed cascaded architecture design, false-positive estimates can be largely eliminated. Additionally, an associated mapping approach is introduced to track the hand trace via the predicted locations, which addresses the interference of multi-handedness. Subsequently, we propose a trace sequence neural network (TraceSeqNN) to recognize the hand gesture by exploiting the motion features of the tracked trace. Finally, we provide a variety of experimental results to show that the proposed framework is able to achieve state-of-the-art accuracy with significantly reduced computational cost, which are the key properties for enabling real-time applications in low-cost commercial devices such as mobile devices and AR/VR headsets.

中文翻译：

---

LE-HGR：一种轻量级、高效的基于 RGB 的嵌入式 AR 设备在线手势识别网络

在线手势识别 (HGR) 技术在增强现实 (AR) 应用程序中必不可少，可实现自然的人机交互和通信。近年来，低成本AR设备的消费市场快速增长，但该领域的技术成熟度仍然有限​​。这些设备价格低廉、内存有限、计算单元资源受限，这使得在线 HGR 成为一个具有挑战性的问题。为了解决这个问题，我们提出了一种轻量级且计算效率高的 HGR 框架，即 LE-HGR，以在低计算能力的嵌入式设备上实现实时手势识别。我们还表明，所提出的方法具有高精度和鲁棒性，能够在各种复杂的交互环境中达到高端性能。为了实现我们的目标，我们首先提出了一个级联多任务卷积神经网络（CNN）来同时在线预测手部检测的概率和回归手部关键点位置。我们表明，通过所提出的级联架构设计，可以在很大程度上消除误报估计。此外，还引入了相关的映射方法来通过预测位置跟踪手部轨迹，从而解决了多手性的干扰。随后，我们提出了一个轨迹序列神经网络（TraceSeqNN），通过利用跟踪轨迹的运动特征来识别手势。最后，我们提供了各种实验结果，以表明所提出的框架能够在显着降低计算成本的情况下实现最先进的准确性，