In this paper we investigate the active inference framework as a means to enable autonomous behavior in artificial agents. Active inference is a theoretical framework underpinning the way organisms act and observe in the real world. In active inference, agents act in order to minimize their so called free energy, or prediction error. Besides being biologically plausible, active inference has been shown to solve hard exploration problems in various simulated environments. However, these simulations typically require handcrafting a generative model for the agent. Therefore we propose to use recent advances in deep artificial neural networks to learn generative state space models from scratch, using only observation-action sequences. This way we are able to scale active inference to new and challenging problem domains, whilst still building on the theoretical backing of the free energy principle. We validate our approach on the mountain car problem to illustrate that our learnt models can indeed trade-off instrumental value and ambiguity. Furthermore, we show that generative models can also be learnt using high-dimensional pixel observations, both in the OpenAI Gym car racing environment and a real-world robotic navigation task. Finally we show that active inference based policies are an order of magnitude more sample efficient than Deep Q Networks on RL tasks.

中文翻译：

---

学习主动推理的生成状态空间模型


在本文中，我们研究主动推理框架作为实现人工智能体自主行为的手段。主动推理是支撑有机体在现实世界中行为和观察方式的理论框架。在主动推理中，智能体采取行动以最小化其所谓的自由能或预测误差。除了在生物学上合理之外，主动推理还被证明可以解决各种模拟环境中的困难探索问题。然而，这些模拟通常需要为代理手工制作生成模型。因此，我们建议利用深度人工神经网络的最新进展，仅使用观察动作序列从头开始学习生成状态空间模型。通过这种方式，我们能够将主动推理扩展到新的和具有挑战性的问题领域，同时仍然建立在自由能原理的理论支持之上。我们验证了我们在山地汽车问题上的方法，以说明我们学习的模型确实可以权衡工具价值和模糊性。此外，我们表明，在 OpenAI Gym 赛车环境和现实世界的机器人导航任务中，也可以使用高维像素观察来学习生成模型。最后，我们表明，在 RL 任务上，基于主动推理的策略的样本效率比深度 Q 网络高一个数量级。