Recognizing human activities has become a trend in smart surveillance that contains several challenges, such as performing effective analyses of huge video data streams, while maintaining low computational complexity, and performing this task in real-time. Current activity recognition techniques are using convolutional neural network (CNN) models with computationally complex classifiers, creating hurdles in obtaining quick responses for abnormal activities. To address these challenges in real-time surveillance, this paper proposes a lightweight deep learning-assisted framework for activity recognition. First, we detect a human in the surveillance stream using an effective CNN model, which is trained on two surveillance datasets. The detected individual is tracked throughout the video stream via an ultra-fast object tracker called the ‘minimum output sum of squared error’ (MOSSE). Next, for each tracked individual, pyramidal convolutional features are extracted from two consecutive frames using the efficient LiteFlowNet CNN. Finally, a novel deep skip connection gated recurrent unit (DS-GRU) is trained to learn the temporal changes in the sequence of frames for activity recognition. Experiments are conducted over five benchmark activity recognition datasets, and the results indicate the efficiency of the proposed technique for real-time surveillance applications compared to the state-of-the-art.

识别人类活动已成为智能监控的趋势，其中包含了许多挑战，例如对大型视频数据流进行有效分析，同时保持较低的计算复杂度，并实时执行此任务。当前的活动识别技术正在使用具有计算复杂分类器的卷积神经网络（CNN）模型，从而在获得对异常活动的快速响应方面遇到障碍。为了解决实时监控中的这些挑战，本文提出了一种用于活动识别的轻量级深度学习辅助框架。首先，我们使用有效的CNN模型在监视流中检测到一个人，该模型在两个监视数据集上进行训练。通过称为“最小输出平方误差的总和”的超快速对象跟踪器，在整个视频流中跟踪检测到的个体。接下来，对于每个跟踪的个体，使用高效的LiteFlowNet CNN从两个连续的帧中提取金字塔卷积特征。最后，训练一种新型的深度跳过连接门控循环单元（DS-GRU），以学习帧序列中的时间变化以进行活动识别。在五个基准活动识别数据集上进行了实验，结果表明，与最新技术相比，该技术在实时监视应用中的效率更高。最后，训练一种新型的深度跳过连接门控循环单元（DS-GRU），以学习帧序列中的时间变化以进行活动识别。在五个基准活动识别数据集上进行了实验，结果表明，与最新技术相比，该技术在实时监视应用中的效率更高。最后，训练一种新型的深度跳过连接门控循环单元（DS-GRU），以学习帧序列中的时间变化以进行活动识别。在五个基准活动识别数据集上进行了实验，结果表明，与最新技术相比，该技术在实时监视应用中的效率更高。