This study proposes a three-stream model using two different types of deep convolutional neural networks (CNNs): (i) a spatial stream with a CNN on images; (ii) a ResNet (residual network) on optical flows; and, (iii) a ResNet on the concatenation of motion features. This model is applied to four datasets: (i) UCF Sports; (ii) Youtube Sports; (iii) SBU action interaction; and (iv) a subset of the UCF-1M Sports. Using two optical flow estimation methods: (i) a motion boundary emphasised Epicflow (Edge Preserving Interpolation Correspondences for Optical Flow) method, (MBEpicflow); and (ii) the Flownet 2 method, a learning optical flow estimation method. It was found that (i) the proposed MBEpicflow outperforms the Flownet 2 method on the SBU dataset, while the Flownet 2 performs equally well or better than the MBEpicflow method on the other three datasets, and these results are the best when compared with those obtained using other approaches on all datasets evaluated. These results showed the importance of accurate optical flow plays in human action recognition, an aspect which has been seldom addressed. Moreover, it showed that if some measure of the global behaviours of motion is incorporated, the generalisation performance is often improved by 1–2%.

中文翻译：

---

运动边界强调光流方法用于人体动作识别

这项研究提出了一种使用两种不同类型的深度卷积神经网络（CNN）的三流模型：（i）在图像上带有CNN的空间流；（ii）关于光流的ResNet（残留网络）；（iii）关于运动特征串联的ResNet。该模型应用于四个数据集：（i）UCF体育；（ii）YouTube体育；（iii）SBU行动互动；（iv）UCF-1M运动的子集。使用两种光流估计方法：（i）强调运动边界的Epicflow（光流的边缘保留插值对应）方法（MBEpicflow）；（ii）Flownet 2方法，一种学习光流量估计方法。发现（i）拟议的MBEpicflow优于SBU数据集上的Flownet 2方法，而Flownet 2在其他三个数据集上的性能与MBEpicflow方法相同或更好，并且与在所有评估的数据集上使用其他方法获得的结果相比，这些结果是最佳的。这些结果表明，准确的光流运动在人体动作识别中的重要性，这一方面很少得到解决。此外，它表明，如果合并了一些整体运动行为的度量，则泛化性能通常会提高1-2％。