In recent years, great progress has been made in recognizing human activities in complete image sequences. However, predicting human activity earlier in a video is still a challenging task. In this paper, a novel framework named weighted long short-term memory network (WLSTM) with saliency-aware motion enhancement (SME) is proposed for video activity prediction. First, a boundary-prior based motion segmentation method is introduced to use shortest geodesic distance in an undirected weighted graph. Next, a dynamic contrast segmentation strategy is proposed to segment the moving object in a complex environment. Then, the SME is constructed to enhance the moving object by suppressing irrelevant background in each frame. Moreover, an effective long-range attention mechanism is designed to further deal with the long-term dependency of complex non-periodic activities by automatically focusing more on the semantic critical frames instead of processing all sampled frames equally. Thus, the learned weights can highlight the discriminative frames and reduce the temporal redundancy. Finally, we evaluate our framework on the UT-Interaction and sub-JHMDB datasets. The experimental results show that WLSTM with SME statistically outperforms a number of state-of-the-art methods on both datasets.

近年来，在识别完整图像序列中的人类活动方面取得了长足的进步。但是，在视频中更早地预测人类活动仍然是一项艰巨的任务。在本文中，提出了一种具有显着性运动增强（SME）的加权长期短期记忆网络（WLSTM）的新颖框架，用于视频活动预测。首先，引入基于边界优先的运动分割方法，以在无向加权图中使用最短测地距离。接下来，提出了一种动态对比度分割策略来分割复杂环境中的运动物体。然后，构造SME以通过抑制每个帧中无关的背景来增强运动对象。此外，一种有效的远程关注机制旨在通过自动将更多注意力集中在语义关键帧上，而不是平等地处理所有采样帧来进一步处理复杂的非周期性活动的长期依赖性。因此，学习到的权重可以突出区分帧并减少时间冗余。最后，我们在UT-Interaction和sub-JHMDB数据集上评估我们的框架。实验结果表明，带有SME的WLSTM在两个数据集上的性能均优于许多最新方法。我们在UT-Interaction和sub-JHMDB数据集上评估我们的框架。实验结果表明，带有SME的WLSTM在两个数据集上的性能均优于许多最新方法。我们在UT-Interaction和sub-JHMDB数据集上评估我们的框架。实验结果表明，带有SME的WLSTM在两个数据集上的性能均优于许多最新方法。