Weakly supervised semantic instance segmentation with only image-level supervision, instead of relying on expensive pixel-wise masks or bounding box annotations, is an important problem to alleviate the data-hungry nature of deep learning. In this article, we tackle this challenging problem by aggregating the image-level information of all training images into a large knowledge graph and exploiting semantic relationships from this graph. Specifically, our effort starts with some generic segment-based object proposals (SOP) without category priors. We propose a multiple instance learning (MIL) framework, which can be trained in an end-to-end manner using training images with image-level labels. For each proposal, this MIL framework can simultaneously compute probability distributions and category-aware semantic features, with which we can formulate a large undirected graph. The category of background is also included in this graph to remove the massive noisy object proposals. An optimal multi-way cut of this graph can thus assign a reliable category label to each proposal. The denoised SOP with assigned category labels can be viewed as pseudo instance segmentation of training images, which are used to train fully supervised models. The proposed approach achieves state-of-the-art performance for both weakly supervised instance segmentation and semantic segmentation. The code is available at https://github.com/yun-liu/LIID .

中文翻译：

---

利用实例、图像和数据集级别的信息进行弱监督实例分割

仅具有图像级监督的弱监督语义实例分割，而不是依赖于昂贵的像素级掩码或边界框注释，是缓解深度学习的数据饥渴性质的重要问题。在本文中，我们通过将所有训练图像的图像级信息聚合到一个大型知识图中并利用该图中的语义关系来解决这个具有挑战性的问题。具体来说，我们的工作从一些没有类别先验的通用基于分段的对象建议（SOP）开始。我们提出了一个多实例学习 (MIL) 框架，该框架可以使用带有图像级标签的训练图像以端到端的方式进行训练。对于每个提议，这个 MIL 框架可以同时计算概率分布和类别感知语义特征，我们可以用它来制定一个大的无向图。该图中还包括背景类别，以去除大量嘈杂的对象建议。因此，该图的最佳多路切割可以为每个提议分配一个可靠的类别标签。具有分配类别标签的去噪 SOP 可以看作是训练图像的伪实例分割，用于训练完全监督的模型。所提出的方法在弱监督实例分割和语义分割方面都达到了最先进的性能。该代码可在 具有分配类别标签的去噪 SOP 可以看作是训练图像的伪实例分割，用于训练完全监督的模型。所提出的方法在弱监督实例分割和语义分割方面都达到了最先进的性能。该代码可在 具有分配类别标签的去噪 SOP 可以看作是训练图像的伪实例分割，用于训练完全监督的模型。所提出的方法在弱监督实例分割和语义分割方面都达到了最先进的性能。该代码可在https://github.com/yun-liu/LIID .