As an important and challenging problem in computer vision, scene graph generation (SGG) aims to find out the underlying semantic relationships among objects from a given image for scene understanding. Usually, prevalent SGG approaches adopt a learning pipeline with the assumption that there exists only a single relationship for a particular object pair. Considering the common phenomenon that a pair of objects can be attached by multiple relationships, we propose a multi-label scene graph generation pipeline with multi-grained features (MLMG-SGG), which formulates the relationship detection as a multi-label classification problem during training while generating multigraphs at inference time. In order to better model the fine-grained relationships, the proposed pipeline encodes the feature representation of SGG on different spatial scales by a specially designed Multi-Grained Module (MGM), resulting in the multi-grained (i.e., object-level and region-level) features of objects. Experimental results over the benchmark dataset demonstrate the significant performance gain of the proposed pipeline used as a plug-in for the state-of-the-art methods.

中文翻译：

---

MLMG-SGG：具有多粒度特征的多标签场景图生成


作为计算机视觉中一个重要且具有挑战性的问题，场景图生成（SGG）旨在从给定图像中找出对象之间的潜在语义关系以进行场景理解。通常，流行的 SGG 方法采用学习管道，并假设特定对象对仅存在单一关系。考虑到一对对象可以通过多种关系连接的常见现象，我们提出了一种具有多粒度特征的多标签场景图生成管道（MLMG-SGG），它将关系检测表述为多标签分类问题在推理时生成多重图的同时进行训练。为了更好地建模细粒度关系，所提出的管道通过专门设计的多粒度模块（MGM）对不同空间尺度上的 SGG 特征表示进行编码，从而产生多粒度（即对象级和区域级） -level）对象的特征。基准数据集的实验结果证明了所提出的管道用作最先进方法的插件的显着性能增益。