The dual stream-based human action recognition model offers the advantage of high recognition accuracy, but the algorithm is less robust in case of lighting changes. The human skeleton has a strong ability to express human behavior and actions; however, the scene information is ignored. Drawing on the idea of the dual-stream model, this paper proposes a human skeleton and scene image-based dual-stream model for human action recognition. The motion features are extracted through the spatio-temporal graph convolution of the human skeleton, and a scene recognition model is proposed based on the sparse frame sampling of video and video-level consensus strategy to process the scene video and gather the visual scene information. The proposed model exploits the advantages of skeleton information in motion expression and the superiority of the image in scene presentation. The scene information and spatio-temporal graph convolution-based human skeleton limbs are fused complementarily to achieve human action recognition. Compared to the conventional optical flow-based dual-stream action recognition method, this model is verified by experimenting under unstable light conditions, and the performance of human action recognition is robust and promising.

基于双流的人体动作识别模型具有识别准确率高的优点，但算法在光照变化的情况下鲁棒性较差。人体骨骼具有很强的表达人类行为和动作的能力；然而，场景信息被忽略。借鉴双流模型的思想，本文提出了一种基于人体骨骼和场景图像的双流模型进行人体动作识别。通过人体骨骼的时空图卷积提取运动特征，提出一种基于视频稀疏帧采样和视频级共识策略的场景识别模型，对场景视频进行处理，收集视觉场景信息。所提出的模型利用了骨骼信息在运动表达中的优势和图像在场景呈现中的优势。场景信息和基于时空图卷积的人体骨骼四肢互补融合，实现人体动作识别。与传统的基于光流的双流动作识别方法相比，该模型通过在不稳定光照条件下的实验验证，人体动作识别的性能稳健且有前景。