We present a Reverse Reinforcement Learning (Reverse RL) approach for representing retrospective knowledge. General Value Functions (GVFs) have enjoyed great success in representing predictive knowledge, i.e., answering questions about possible future outcomes such as "how much fuel will be consumed in expectation if we drive from A to B?". GVFs, however, cannot answer questions like "how much fuel do we expect a car to have given it is at B at time $t$?". To answer this question, we need to know when that car had a full tank and how that car came to B. Since such questions emphasize the influence of possible past events on the present, we refer to their answers as retrospective knowledge. In this paper, we show how to represent retrospective knowledge with Reverse GVFs, which are trained via Reverse RL. We demonstrate empirically the utility of Reverse GVFs in both representation learning and anomaly detection.

中文翻译：

---

用反向强化学习学习回顾性知识

我们提出了一种用于表示回顾性知识的反向强化学习（Reverse RL）方法。一般价值函数 (GVF) 在表示预测知识方面取得了巨大成功，即回答有关未来可能结果的问题，例如“如果我们从 A 点开车到 B 点，预计会消耗多少燃料？”。然而，GVF 无法回答诸如“我们期望一辆汽车在时间 $t$ 处为它提供多少燃料？”的问题。要回答这个问题，我们需要知道那辆车什么时候装满了油箱，以及那辆车是如何来到 B 的。由于这些问题强调可能的过去事件对现在的影响，我们将他们的答案称为回顾性知识。在本文中，我们展示了如何使用通过反向 RL 训练的反向 GVF 来表示回顾性知识。