Due to the decentralisation and complexity of knowledge in the architecture, engineering and construction (AEC) industry, the research on knowledge graphs (KGs) is still insufficient, and most of the research focuses on text-based KG structuring or updating. Entity extraction, a sub-task of knowledge extraction, is critical in general KG update approaches. While the mainstream approach for this task generally uses textual data, visual data is more readily available, more accurate and has a shorter update cycle than textual data. Therefore, this paper integrates zero-shot learning techniques with general KGs to present a novel framework called “video2entities” that can extract entities from videos to update the AEC KG. The framework combines the perceptual capabilities of computer vision with the cognitive capabilities of KG to improve the accuracy and timeliness of KG updates. Experimental results demonstrate that the framework can extract “new entities” from architectural decoration videos for AEC KG updates.

由于建筑，工程和建筑（AEC）行业中知识的分散性和复杂性，对知识图（KGs）的研究仍然不足，并且大多数研究集中在基于文本的KG的结构或更新上。实体提取是知识提取的子任务，在常规的KG更新方法中至关重要。尽管用于此任务的主流方法通常使用文本数据，但是视觉数据比文本数据更容易获得，更准确并且更新周期更短。因此，本文将零镜头学习技术与一般的KG集成在一起，提出了一种称为“ video2entities”的新颖框架，该框架可以从视频中提取实体以更新AEC KG。该框架将计算机视觉的感知能力与KG的认知能力相结合，以提高KG更新的准确性和及时性。实验结果表明，该框架可以从建筑装饰视频中提取“新实体”以进行AEC KG更新。