In skeleton-based action recognition, graph convolutional networks (GCNs) have achieved remarkable success. However, there are two shortcomings of current GCN-based methods. Firstly, the computation cost is pretty heavy, typically over 15 GFLOPs for one action sample. Some recent works even reach ~100 GFLOPs. Secondly, the receptive fields of both spatial graph and temporal graph are inflexible. Although recent works introduce incremental adaptive modules to enhance the expressiveness of spatial graph, their efficiency is still limited by regular GCN structures. In this paper, we propose a shift graph convolutional network (ShiftGCN) to overcome both shortcomings. ShiftGCN is composed of novel shift graph operations and lightweight point-wise convolutions, where the shift graph operations provide flexible receptive fields for both spatial graph and temporal graph. To further boost the efficiency, we introduce four techniques and build a more lightweight skeleton-based action recognition model named ShiftGCN++. ShiftGCN++ is an extremely computation-efficient model, which is designed for low-power and low-cost devices with very limited computing power. On three datasets for skeleton-based action recognition, ShiftGCN notably exceeds the state-of-the-art methods with over $10\times $ less FLOPs and $4\times $ practical speedup. ShiftGCN++ further boosts the efficiency of ShiftGCN, which achieves comparable performance with $6\times $ less FLOPs and $2\times $ practical speedup.

中文翻译：

---

使用 ShiftGCN++ 实现基于骨架的极轻量级动作识别

在基于骨架的动作识别中，图卷积网络（GCN）取得了显着的成功。然而，当前基于 GCN 的方法有两个缺点。首先，计算成本非常高，一个动作样本通常超过 15 GFLOP。最近的一些作品甚至达到了~100 GFLOPs。其次，空间图和时间图的感受野都是不灵活的。尽管最近的工作引入了增量自适应模块来增强空间图的表达能力，但它们的效率仍然受到常规 GCN 结构的限制。在本文中，我们提出了一种移位图卷积网络（ShiftGCN）来克服这两个缺点。ShiftGCN 由新颖的移位图操作和轻量级的逐点卷积组成，其中移位图操作为空间图和时间图提供了灵活的接受域。为了进一步提高效率，我们引入了四种技术并构建了一个名为 ShiftGCN++ 的更轻量级的基于骨架的动作识别模型。ShiftGCN++ 是一种计算效率极高的模型，专为计算能力非常有限的低功耗和低成本设备而设计。在基于骨架的动作识别的三个数据集上，ShiftGCN 明显超过了最先进的方法 $10\times $ 更少的 FLOP 和 $4\times $ 实际加速。ShiftGCN++ 进一步提高了 ShiftGCN 的效率，实现了与 $6\times $ 更少的 FLOP 和 $2\times $ 实际加速。