The goal of early action recognition is to predict action label when the sequence is partially observed. The existing methods treat the early action recognition task as sequential classification problems on different observation ratios of an action sequence. Since these models are trained by differentiating positive category from all negative classes, the diverse information of different negative categories is ignored, which we believe can be collected to help improve the recognition performance. In this paper, we step towards to a new direction by introducing category exclusion to early action recognition. We model the exclusion as a mask operation on the classification probability output of a pre-trained early action recognition classifier. Specifically, we use policy-based reinforcement learning to train an agent. The agent generates a series of binary masks to exclude interfering negative categories during action execution and hence help improve the recognition accuracy. The proposed method is evaluated on three benchmark recognition datasets, NTU-RGBD, First-Person Hand Action, as well as UCF-101. The proposed method enhances the recognition accuracy consistently over all different observation ratios on the three datasets, where the accuracy improvements on the early stages are especially significant.

中文翻译：

---

使用基于策略的强化学习进行类别排除的早期行动识别

早期动作识别的目标是在部分观察到序列时预测动作标签。现有方法将早期动作识别任务视为对动作序列的不同观察率的顺序分类问题。由于这些模型是通过区分正类别和所有负类别来训练的，因此忽略了不同负类别的不同信息，我们认为可以收集这些信息以帮助提高识别性能。在本文中，我们通过将类别排除引入早期动作识别来迈向新的方向。我们将排除建模为对预训练的早期动作识别分类器的分类概率输出的掩码操作。具体来说，我们使用基于策略的强化学习来训练代理。代理生成一系列二进制掩码以在动作执行期间排除干扰负面类别，从而有助于提高识别准确性。所提出的方法在三个基准识别数据集 NTU-RGBD、第一人称手部动作以及 UCF-101 上进行了评估。所提出的方法在三个数据集上的所有不同观察率上一致地提高了识别准确度，其中早期阶段的准确度提高尤其显着。