Human action recognition (HAR) is one of the most challenging tasks in the field of computer vision due to complex backgrounds and ambiguity action, etc. To tackle these issues, we propose a novel action recognition framework called Semantic Feature and High-order Physical Feature Fusion (SF-HPFF). Concretely, we first calculate attention pooling module with a low-rank approximation to remove the information of irrelevant complex backgrounds and thus capture the interested target motion region. On this basis, motion features based on the physical characteristics of flow field and semantic features based on word embedding are developed to distinguish ambiguity behaviors. These features are of low dimension and high discrimination, which help to reduce computation burden significantly while maintaining an excellent recognition performance. Finally, cascaded convolutional fusion network is adopted to fuse features and accomplish classification. Multiple experiment results validate that the proposed SF-HPFF outperforms the state-of-art action recognition methods.

由于复杂的背景和模糊的动作等，人类动作识别（HAR）是计算机视觉领域最具挑战性的任务之一。为了解决这些问题，我们提出了一种新的动作识别框架，称为语义特征和高阶物理特征融合（SF-HPFF）。具体而言，我们首先计算具有低秩近似的注意力池模块，以去除无关复杂背景的信息，从而捕获感兴趣的目标运动区域。在此基础上，开发了基于流场物理特征的运动特征和基于词嵌入的语义特征来区分歧义行为。这些特征具有低维度和高辨别力，有助于在保持出色识别性能的同时显着减少计算负担。最后，采用级联卷积融合网络融合特征完成分类。多项实验结果验证了所提出的 SF-HPFF 优于最先进的动作识别方法。