Analysis of first-person (egocentric) videos involving human actions could help in the solutions of many problems. These videos include a large number of fine-grained action categories with hand–object interactions. In this paper, a compositional verb–noun model including two complementary temporal streams is proposed with various fusion strategies to recognize egocentric actions. The first step is based on construction of verb and object video models as decomposition of actions with a special attention on hands. Particularly, the verb video model that is the spatial–temporal encoding of hand actions and the object video model that is the object scores with hand–object layout are represented as two separate pathways. The second step is the fusion stage to identify action category, where distinct verb and object models are combined to give their action judgments. We propose fusion strategies with recurrent steps collecting verb and object label judgments along a temporal video sequence. We evaluate recognition performances for individual verb and object models; and we present extensive experimental evaluations for action recognition over recurrent-based fusion approaches on the EGTEA Gaze+ dataset.

分析涉及人类行为的第一人称（以自我为中心）视频可以帮助解决许多问题。这些视频包括大量具有手-对象交互的细粒度动作类别。在本文中，提出了一种包含两个互补时间流的组合动词-名词模型，该模型具有各种融合策略来识别以自我为中心的行为。第一步是基于动词和对象视频模型的构建，作为动作的分解，特别注意手。特别地，作为手部动作的空间-时间编码的动词视频模型和作为具有手-对象布局的对象分数的对象视频模型被表示为两个独立的路径。第二步是融合阶段，识别动作类别，其中不同的动词和宾语模型组合在一起以给出它们的动作判断。我们提出了具有循环步骤的融合策略，沿着时间视频序列收集动词和对象标签判断。我们评估单个动词和对象模型的识别性能；我们对 EGTEA Gaze+ 数据集上基于循环的融合方法的动作识别进行了广泛的实验评估。