Video-based action recognition in realistic scenes is a core technology for human–computer interaction and smart surveillance. Although the trajectory features with the bag of visual words have confirmed promising performance, spatiotemporal interactive information cannot be effectively encoded which is valuable for classification. To address this issue, we propose a spatiotemporal semantic feature (ST-SF) and implement the conversion of it to the auxiliary criterion based on the information entropy theory. First, we present a text-based relevance analysis method to estimate the textual labels of objects most relevant to actions, which are employed to train the more targeted detectors based on the deep network. False detections are optimized by the inter-frame cooperativity and dynamic programming to construct the valid tubes. Then, we design the ST-SF to encode the interactive information, and the concept and calculation of feature entropy are defined based on the spatial distribution of ST-SFs on the training set. Finally, we achieve a two-stage classification strategy using the resulting decision gains. Experimental results on three publicly available datasets demonstrate that our method is robust and improves upon the state-of-the-art algorithms.

中文翻译：

---

基于时空语义编码和特征熵的辅助标准转换用于动作识别

基于视频的现实场景动作识别是人机交互和智能监控的核心技术。尽管带有视觉词袋的轨迹特征已经证实了良好的性能，但时空交互信息不能被有效地编码，这对分类很有价值。为了解决这个问题，我们提出了一种时空语义特征（ST-SF），并基于信息熵理论将其转换为辅助标准。首先，我们提出了一种基于文本的相关性分析方法来估计与动作最相关的对象的文本标签，用于训练基于深度网络的更有针对性的检测器。通过帧间协同性和动态编程优化错误检测以构建有效管。然后，我们设计了 ST-SF 来编码交互信息，并根据 ST-SF 在训练集上的空间分布定义特征熵的概念和计算。最后，我们使用由此产生的决策增益实现了两阶段分类策略。在三个公开可用的数据集上的实验结果表明，我们的方法是稳健的，并且改进了最先进的算法。