Social relations and interactions between persons form the foundation of human society. Effective recognition of social relationships has great potential for understanding and improving people’s psychology and behaviors, e.g., mental health and activity analysis, and further improving social resilience. Existing work of social relation recognition (SRR) mainly focuses on exploiting two or three types of features to recognize social relations without considering the relations between features. In this paper, we proposed a new framework for extraction and fusion of the hybrid features, namely Social Relation Graph Reasoning model driven by Hybrid-Features (HF-SRGR). For the proposed method, a social relation graph was constructed first using relation and scene features as nodes. An attention mechanism was then designed to incorporate into graph neural networks (GNNs), generating inter-pair features and interactions between relation nodes and the scene node, respectively. Besides, the propagation of scene information further strengthens the rationality of interaction reasoning. Extensive experiments on PISC and PIPA datasets show that our proposed approach achieves better performance over the state-of-the-art methods in terms of accuracy.

人与人之间的社会关系和互动构成了人类社会的基础。对社会关系的有效识别对于理解和改善人们的心理和行为（例如心理健康和活动分析）以及进一步提高社会适应力具有巨大潜力。社会关系识别（SRR）的现有工作主要集中在利用两种或三种类型的特征来识别社会关系，而不考虑特征之间的关系。在本文中，我们提出了一种用于提取和融合混合特征的新框架，即混合特征驱动的社会关系图推理模型（HF-SRGR）。对于所提出的方法，首先使用关系和场景特征作为节点构建社交关系图。然后设计了一种注意力机制以结合到图神经网络 (GNN) 中，分别生成关系节点和场景节点之间的对间特征和交互。此外，场景信息的传播进一步加强了交互推理的合理性。在 PISC 和 PIPA 数据集上的大量实验表明，我们提出的方法在准确性方面比最先进的方法具有更好的性能。