Midbrain dopamine signals are widely thought to report reward prediction errors that drive learning in the basal ganglia. However, dopamine has also been implicated in various probabilistic computations, such as encoding uncertainty and controlling exploration. Here, we show how these different facets of dopamine signalling can be brought together under a common reinforcement learning framework. The key idea is that multiple sources of uncertainty impinge on reinforcement learning computations: uncertainty about the state of the environment, the parameters of the value function and the optimal action policy. Each of these sources plays a distinct role in the prefrontal cortex-basal ganglia circuit for reinforcement learning and is ultimately reflected in dopamine activity. The view that dopamine plays a central role in the encoding and updating of beliefs brings the classical prediction error theory into alignment with more recent theories of Bayesian reinforcement learning.

中文翻译：

---

相信多巴胺。

人们普遍认为中脑多巴胺信号可报告奖励预测错误，这些错误会推动基底神经节的学习。但是，多巴胺还涉及各种概率计算，例如编码不确定性和控制探索。在这里，我们展示了如何在共同的强化学习框架下将多巴胺信号传导的这些不同方面整合在一起。关键思想是不确定性的多种来源会影响强化学习的计算：环境状态的不确定性，价值函数的参数以及最佳行动策略。这些来源中的每一个在前额叶皮层-基底神经节回路中为强化学习起着独特的作用，并最终反映在多巴胺的活动中。