Abstract Knowledge Graph (KG) is identified as a major area in artificial intelligence, which is used for many real-world applications. The task of knowledge graph reasoning has been widely used and proven to be effective, which aims to find these reasonable paths for various relations to solve the issue of incompleteness in KGs. However, many previous works on KG reasoning, such as path-based or reinforcement learning-based methods, are too reliant on the pre-training, where the paths from the head entity and the target entity must be given to pre-train the model, which would easily lead the model to overfit on the given paths seen in the pre-training. To address this issue, we propose a novel reasoning model named MemoryPath with a deep reinforcement learning framework, which incorporates Long Short Term Memory (LSTM) and graph attention mechanism to form the memory component. The well-designed memory component can get rid of the pre-training so that the model doesn’t depend on the given target entity for training. A tailored mechanism of reinforcement learning is presented in this proposed deep reinforcement framework to optimize the training procedure, where two metrics, Mean Selection Rate (MSR) and Mean Alternative Rate (MAR), are defined to quantitatively measure the complexities of the query relations. Meanwhile, three different training mechanisms, Action Dropout, Reward Shaping and Force Forward, are proposed to optimize the training process of the proposed MemoryPath. The proposed MemoryPath is validated on two datasets from FB15K-237 and NELL-995 on different tasks including fact prediction, link prediction and success rate in finding paths. The experimental results demonstrate that the tailored mechanism of reinforcement learning make the MemoryPath achieves state-of-the-art performance comparing with the other models. Also, the qualitative analysis indicates that the MemoryPath can store the learning process and automatically find the promising paths for a reasoning task during the training, and shows the effectiveness of the memory component.

中文翻译：

---

MemoryPath：将记忆组件纳入知识图推理的深度强化学习框架

摘要知识图谱 (KG) 被确定为人工智能的一个主要领域，用于许多现实世界的应用程序。知识图推理的任务已被广泛使用并被证明是有效的，其目的是为各种关系找到这些合理的路径，以解决 KG 中的不完备性问题。然而，之前许多关于 KG 推理的工作，例如基于路径或基于强化学习的方法，都过于依赖预训练，其中必须给出来自头部实体和目标实体的路径来预训练模型，这很容易导致模型在预训练中看到的给定路径上过度拟合。为了解决这个问题，我们提出了一种名为 MemoryPath 的新型推理模型，该模型具有深度强化学习框架，它结合了长短期记忆（LSTM）和图注意力机制来形成记忆组件。精心设计的内存组件可以摆脱预训练，使模型不依赖于给定的目标实体进行训练。在这个提议的深度强化框架中提出了一种定制的强化学习机制来优化训练过程，其中定义了两个指标，平均选择率 (MSR) 和平均替代率 (MAR)，以定量测量查询关系的复杂性。同时，提出了三种不同的训练机制，Action Dropout、Reward Shaping 和 Force Forward，以优化所提出的 MemoryPath 的训练过程。建议的 MemoryPath 在来自 FB15K-237 和 NELL-995 的两个数据集上在不同任务上进行了验证，包括事实预测，链接预测和寻找路径的成功率。实验结果表明，与其他模型相比，强化学习的定制机制使 MemoryPath 实现了最先进的性能。此外，定性分析表明 MemoryPath 可以存储学习过程并在训练期间自动为推理任务找到有希望的路径，并显示了记忆组件的有效性。