Various action recognition approaches have recently been proposed with the aid of three-dimensional (3D) convolution and a multiple stream structure. However, existing methods are sensitive to background and optical flow noise, which prevents from learning the main object in a video frame. Furthermore, they cannot reflect the accuracy of each stream in the process of combining multiple streams. In this paper, we present a novel action recognition method that improves the existing method using optical flow and a multi-stream structure. The proposed method consists of two parts: (i) optical flow enhancement process using image segmentation and (ii) score fusion process by applying weighted sum of the accuracy. The enhancement process can help the network to efficiently analyze the flow information of the main object in the optical flow frame, thereby improving accuracy. A different accuracy of each stream can be reflected to the fused score while using the proposed score fusion method. We achieved an accuracy of 98.2% on UCF-101 and 82.4% on HMDB-51. The proposed method outperformed many state-of-the-art methods without changing the network structure and it is expected to be easily applied to other networks.

中文翻译：

---

使用具有光流和加权总和的多流深度学习增强的动作识别。

最近已经借助于三维（3D）卷积和多流结构提出了各种动作识别方法。但是，现有方法对背景噪声和光流噪声敏感，这妨碍了学习视频帧中的主要对象。此外，它们不能反映在合并多个流的过程中每个流的准确性。在本文中，我们提出了一种新颖的动作识别方法，该方法使用光流和多流结构改进了现有方法。所提出的方法包括两部分：（i）使用图像分割的光流增强处理和（ii）通过应用精度的加权和进行分数融合处理。增强过程可以帮助网络有效地分析光流帧中主要物体的流信息，从而提高准确性。使用建议的分数融合方法时，每个流的不同精度可以反映到融合分数中。我们在UCF-101上达到98.2％的精度，在HMDB-51上达到82.4％的精度。所提出的方法在不改变网络结构的情况下胜过许多最新技术，并且有望轻松应用于其他网络。