Active inference is an approach to understanding behaviour that rests upon the idea that the brain uses an internal generative model to predict incoming sensory data. The fit between this model and data may be improved in two ways. The brain could optimise probabilistic beliefs about the variables in the generative model (i.e. perceptual inference). Alternatively, by acting on the world, it could change the sensory data, such that they are more consistent with the model. This implies a common objective function (variational free energy) for action and perception that scores the fit between an internal model and the world. We compare two free energy functionals for active inference in the framework of Markov decision processes. One of these is a functional of beliefs (i.e. probability distributions) about states and policies, but a function of observations, while the second is a functional of beliefs about all three. In the former (expected free energy), prior beliefs about outcomes are not part of the generative model (because they are absorbed into the prior over policies). Conversely, in the second (generalised free energy), priors over outcomes become an explicit component of the generative model. When using the free energy function, which is blind to future observations, we equip the generative model with a prior over policies that ensure preferred (i.e. priors over) outcomes are realised. In other words, if we expect to encounter a particular kind of outcome, this lends plausibility to those policies for which this outcome is a consequence. In addition, this formulation ensures that selected policies minimise uncertainty about future outcomes by minimising the free energy expected in the future. When using the free energy functional-that effectively treats future observations as hidden states-we show that policies are inferred or selected that realise prior preferences by minimising the free energy of future expectations. Interestingly, the form of posterior beliefs about policies (and associated belief updating) turns out to be identical under both formulations, but the quantities used to compute them are not.

中文翻译：

---

广义自由能和主动推理。


主动推理是一种理解行为的方法，其基础是大脑使用内部生成模型来预测传入的感官数据。该模型和数据之间的拟合可以通过两种方式来改进。大脑可以优化生成模型中变量的概率信念（即感知推理）。或者，通过作用于世界，它可以改变感官数据，使它们与模型更加一致。这意味着行动和感知有一个共同的目标函数（变分自由能），可以对内部模型和世界之间的拟合度进行评分。我们比较了马尔可夫决策过程框架中的主动推理的两个自由能泛函。其中之一是关于国家和政策的信念（即概率分布）的函数，但也是观察的函数，而第二个是关于所有三者的信念的函数。在前者（预期自由能）中，关于结果的先验信念不是生成模型的一部分（因为它们被吸收到先验政策中）。相反，在第二种（广义自由能）中，结果的先验成为生成模型的明确组成部分。当使用对未来观察视而不见的自由能函数时，我们为生成模型配备了先验策略，以确保实现首选（即先验）结果。换句话说，如果我们预期会遇到某种特定的结果，那么这种结果所导致的政策就有了合理性。此外，这种表述确保所选政策通过最小化未来预期的自由能来最小化未来结果的不确定性。 当使用自由能泛函（有效地将未来观察视为隐藏状态）时，我们表明，通过最小化未来期望的自由能来推断或选择实现先验偏好的政策。有趣的是，在这两种表述下，关于政策的后验信念（以及相关的信念更新）的形式是相同的，但用于计算它们的数量却不相同。