Early activity recognition is a classification task before the completion of activity. The study of early activity recognition is beneficial to avoid serious result. Previous studies have focused on extracting effective activity features and modeling for quick and accurate classification. It is challenging because of lack of available information. In order to get a firm basis for judgment, this paper adds an activity prediction module prior to recognition module. The main task of the module is to predict subsequent motions according to observed motions. To avoid motion blur, the structure of GAN (Generative Adversarial Networks) is used to generate the predicted motions. Compared with the traditional deep learning model, dilated neural network has advantages in large-span spatiotemporal feature modeling. The dilated RNN (Recurrent Neural Networks) and CNN (Convolutional Neural Networks) are introduced to the recognition module. In order to make the activity prediction and recognition modules work together, this paper designs and introduces a hard class mining mechanism to improve the learning ability of hard class samples. The proposed method is validated on four skeletal activity datasets and achieves state-of-the-art accuracy.

早期活动识别是活动完成之前的分类任务。早期活动识别的研究有助于避免严重的后果。先前的研究集中在提取有效活动特征和建模以进行快速准确的分类。由于缺乏可用信息，因此具有挑战性。为了获得坚实的判断基础，本文在识别模块之前增加了活动预测模块。该模块的主要任务是根据观察到的运动来预测后续运动。为了避免运动模糊，可使用GAN（生成对抗网络）的结构来生成预测的运动。与传统的深度学习模型相比，膨胀神经网络在大跨度时空特征建模方面具有优势。将扩张后的RNN（递归神经网络）和CNN（卷积神经网络）引入识别模块。为了使活动预测和识别模块协同工作，本文设计并介绍了一种硬类挖掘机制，以提高硬类样本的学习能力。该方法在四个骨骼活动数据集上得到了验证，并达到了最新的准确性。