Visual relationship can capture essential information for images, like the interactions between pairs of objects. Such relationships have become one prominent component of knowledge within sparse image data collected by multimedia sensing devices. Both the latent information and potential privacy can be included in the relationships. However, due to the high combinatorial complexity in modeling all potential relation triplets, previous studies on visual relationship detection have used the mixed visual and semantic features separately for each object, which is incapable for sparse data in IoT systems. Therefore, this paper proposes a new deep learning model for visual relationship detection, which is a novel attempt for cooperating computational intelligence (CI) methods with IoTs. The model imports the knowledge graph and adopts features for both entities and connections among them as extra information. It maps the visual features extracted from images into the knowledge-based embedding vector space, so as to benefit from information in the background knowledge domain and alleviate the impacts of data sparsity. This is the first time that visual features are projected and combined with prior knowledge for visual relationship detection. Moreover, the complexity of the network is reduced by avoiding the learning of redundant features from images. Finally, we show the superiority of our model by evaluating on two datasets.

中文翻译：

---

物联网中稀疏图像的鲁棒视觉关系检测

视觉关系可以捕获图像的基本信息，例如对象对之间的交互。这种关系已成为多媒体传感设备收集的稀疏图像数据中知识的重要组成部分。潜在信息和潜在隐私都可以包含在关系中。然而，由于建模所有潜在关系三元组的高组合复杂性，以前关于视觉关系检测的研究对每个对象分别使用混合的视觉和语义特征，这无法处理物联网系统中的稀疏数据。因此，本文提出了一种新的视觉关系检测深度学习模型，这是将计算智能（CI）方法与物联网合作的一种新尝试。该模型导入知识图并采用实体和它们之间的连接的特征作为额外信息。它将从图像中提取的视觉特征映射到基于知识的嵌入向量空间，从而受益于背景知识域中的信息并减轻数据稀疏性的影响。这是第一次将视觉特征投影并结合先验知识进行视觉关系检测。此外，通过避免从图像中学习冗余特征，降低了网络的复杂性。最后，我们通过评估两个数据集来展示我们模型的优越性。从而受益于背景知识领域的信息并减轻数据稀疏性的影响。这是第一次将视觉特征投影并结合先验知识进行视觉关系检测。此外，通过避免从图像中学习冗余特征，降低了网络的复杂性。最后，我们通过评估两个数据集来展示我们模型的优越性。从而受益于背景知识领域的信息并减轻数据稀疏性的影响。这是第一次将视觉特征投影并结合先验知识进行视觉关系检测。此外，通过避免从图像中学习冗余特征，降低了网络的复杂性。最后，我们通过评估两个数据集来展示我们模型的优越性。