Reinforcement learning (RL) agents are often designed specifically for a particular problem and they generally have uninterpretable working processes. Statistical methods-based agent algorithms can be improved in terms of generalizability and interpretability using symbolic Artificial Intelligence (AI) tools such as logic programming. In this study, we present a model-free RL architecture that is supported with explicit relational representations of the environmental objects. For the first time, we use the PrediNet network architecture in a dynamic decision-making problem rather than image-based tasks, and Multi-Head Dot-Product Attention Network (MHDPA) as a baseline for performance comparisons. We tested two networks in two environments ---i.e., the baseline Box-World environment and our novel environment, Relational-Grid-World (RGW). With the procedurally generated RGW environment, which is complex in terms of visual perceptions and combinatorial selections, it is easy to measure the relational representation performance of the RL agents. The experiments were carried out using different configurations of the environment so that the presented module and the environment were compared with the baselines. We reached similar policy optimization performance results with the PrediNet architecture and MHDPA; additionally, we achieved to extract the propositional representation explicitly ---which makes the agent's statistical policy logic more interpretable and tractable. This flexibility in the agent's policy provides convenience for designing non-task-specific agent architectures. The main contributions of this study are two-fold ---an RL agent that can explicitly perform relational reasoning, and a new environment that measures the relational reasoning capabilities of RL agents.

中文翻译：

---

关系网格世界：一种新的关系推理环境和关系信息提取的代理模型

强化学习 (RL) 代理通常是专门针对特定问题设计的，它们通常具有无法解释的工作过程。可以使用符号人工智能 (AI) 工具（例如逻辑编程）在泛化性和可解释性方面改进基于统计方法的代理算法。在这项研究中，我们提出了一种无模型 RL 架构，该架构由环境对象的显式关系表示支持。我们第一次在动态决策问题而不是基于图像的任务中使用 PrediNet 网络架构，并将多头点积注意力网络 (MHDPA) 作为性能比较的基线。我们在两种环境中测试了两种网络——即基线 Box-World 环境和我们的新型环境 Relational-Grid-World (RGW)。程序生成的 RGW 环境在视觉感知和组合选择方面很复杂，因此很容易衡量 RL 代理的关系表示性能。实验是使用不同的环境配置进行的，以便将所呈现的模块和环境与基线进行比较。我们使用 PrediNet 架构和 MHDPA 达到了类似的策略优化性能结果；此外，我们实现了显式提取命题表示——这使得代理的统计策略逻辑更易于解释和处理。代理策略的这种灵活性为设计非特定任务的代理架构提供了便利。