www.thelancet.com/neurology Published online January 9, 2018 http://dx.doi.org/10.1016/S1474-4422(18)30003-6 1 Animal behaviour is complex, and the understanding of human behaviour is no exception. An individual’s previous experiences can influence everyday routine decision making processes, which in turn can affect habits and behaviours. But what happens when individuals are presented with new scenarios and choices? In these situations, the brain has to anticipate the right action to reach the desired outcome. The panel discussion entitled What our brains want was presented on Oct 26, 2017, and later broadcast on Nov 21, 2017, as part of the BBC Radio 4 series All in the Mind. Three leading experts in neuroscience, computational neuroscience, and neuropsychiatry—Wolfram Schultz, Peter Dayan, and Ray Dolan—were asked questions by the chair and audience about their research on what the brain really wants in terms of recognising and processing the socalled rewards that help individuals to learn and ultimately to survive. To fully understand the importance of their research, a brief step back into history is needed. In the early 20th century, Ivan Pavlov documented that dogs can be taught to associate a neutral stimulus such as the sound of a bell ringing with an unconditioned response such as salivating, despite the food being physically absent. But how did Pavlov’s dogs learn to associate the bell’s chime with feeding? To answer this question, a fundamental chemical in the brain must be introduced: dopamine. Neurons that produce dopamine have a part in controlling movement, as observed by studies showing direct association between dopamine depletion in the striatum and motor deficits in patients with Parkinson’s disease. Dopamine also acts as the signal in the reward system, and dopaminergic neurons play a complex role in behavioural responses to rewarding stimuli. To date, real-time imaging studies have supported the role of dopamine surges in learning and motivation through incentive rewards and the feeling of pleasure—as was the case for Pavlov’s dogs. However, the association between rewards and behavioural learning has only been recently explained by the three panel speakers, who won the 2017 Brain Prize for their multidisciplinary research. The speakers began by using examples and interactive exercises to introduce the audience to their research and a learning pathway conflated with the dopamine reward system that they call reward prediction error. Schultz explained that a prediction error exists when the perceived reward that is received is different from what is predicted. Take for example, an individual who has recently started going to the gym. After a few months, his friends and family applaud him for his efforts and physical progression—an unexpected reward that is associated with dopamine surges, making the individual feel good. Thus, a positive prediction error has occurred and this effect motivates him to continue this behaviour. However, the increased dopamine response is not directly associated with the reward, but with the difference between the predicted reward and what is actually received. Schultz further explained that the opposite would be true for a negative prediction error, for which an individual might avoid a particular scenario if the received reward is worse than predicted. In both cases, the individual has learnt from their perceived prediction and outcome, resulting in changes to his behaviour. If there is no prediction error, the individual retains their perceived prediction and outcome—ie, no learning occurs—and their behaviour remains unaltered. Elaborating this concept further, Dayan mentioned that reward prediction error is essential in ensuring that decisions are made in anticipation of a reward, as often the outcome occurs much later—a socalled Pavlovian effect. With greater understanding of the role of dopamine and reward prediction error comes important clinical implications associated with reward-seeking behaviour, such as drug addiction and compulsive gambling. Dolan suggested that individuals with increased dopamine concentrations in the brain are more likely to show impulsive behaviours, and that consistent dopamine surges might overstimulate the brain with feelings of pleasure and thus motivate some to take risks, for example, compulsive gambling. By contrast, there is an association between decreased dopamine concentrations and ageing, suggesting that this effect might contribute to risk aversion later in life and subsequently reduce an individual’s motivation to learn by reward prediction errors. In a society that is rapidly exposed to novel technology, this aversion can have negative consequences for the ageing population. So what does the brain really want? The panel concludes that the brain is always anticipating rewards for its actions, whether the prediction error is positive or negative. This insight opens a new avenue for clinical research investigating conditions associated with over or under stimulation of dopaminergic neurons. But for now, the message is clear: reward prediction error can give us greater personal understanding of our own methods of learning, sources of motivation, and subsequent behaviour.

中文翻译：

---

我们的大脑真正想要的是什么？

www.thelancet.com/neurology 2018年1月9日在线发表 http://dx.doi.org/10.1016/S1474-4422(18)30003-6 1 动物行为是复杂的，对人类行为的理解也不例外。个人以往的经历会影响日常决策过程，进而影响习惯和行为。但是当个人面临新的场景和选择时会发生什么？在这些情况下，大脑必须预测正确的动作才能达到预期的结果。题为“我们的大脑想要什么”的小组讨论于 2017 年 10 月 26 日发布，随后于 2017 年 11 月 21 日播出，作为 BBC 广播 4 系列“头脑中的一切”的一部分。神经科学、计算神经科学和神经精神病学领域的三位领先专家——沃尔夫拉姆·舒尔茨、彼得·达扬、主席和听众就他们的研究提出问题，讨论大脑在识别和处理帮助个人学习并最终生存的所谓奖励方面真正想要什么。为了充分理解他们研究的重要性，需要简要回顾一下历史。在 20 世纪初期，伊万·巴甫洛夫 (Ivan Pavlov) 记录了可以教狗将中性刺激（例如铃声）与无条件反应（例如流涎）联系起来，尽管实际上没有食物。但是巴甫洛夫的狗是如何学会将钟声与喂食联系起来的呢？要回答这个问题，必须引入大脑中的一种基本化学物质：多巴胺。产生多巴胺的神经元参与控制运动，正如研究所观察到的那样，纹状体中的多巴胺耗竭与帕金森病患者的运动缺陷之间存在直接关联。多巴胺还充当奖励系统中的信号，多巴胺能神经元在对奖励刺激的行为反应中起着复杂的作用。迄今为止，实时成像研究已经通过激励奖励和愉悦感支持多巴胺激增在学习和动机中的作用——就像巴甫洛夫的狗一样。然而，奖励与行为学习之间的关联直到最近才由三位小组发言人解释，他们的多学科研究获得了 2017 年大脑奖。演讲者首先使用示例和互动练习向观众介绍他们的研究和与多巴胺奖励系统相结合的学习途径，他们称之为奖励预测错误。舒尔茨解释说，当收到的感知奖励与预测的不同时，就会存在预测错误。以最近开始去健身房的人为例。几个月后，他的朋友和家人为他的努力和身体进步鼓掌——这是一种意想不到的奖励，与多巴胺激增有关，使个人感觉良好。因此，发生了积极的预测错误，这种影响促使他继续这种行为。然而，增加的多巴胺反应与奖励没有直接关系，但是预测的奖励与实际收到的奖励之间存在差异。舒尔茨进一步解释说，对于负预测错误，情况正好相反，如果收到的奖励比预测的差，个人可能会避免特定的场景。在这两种情况下，个人都从他们感知到的预测和结果中学习，从而改变了他的行为。如果没有预测错误，个体会保留他们感知到的预测和结果——即没有学习发生——并且他们的行为保持不变。Dayan 进一步阐述了这个概念，他提到奖励预测错误对于确保在预期奖励的情况下做出决策至关重要，因为结果通常发生得更晚——所谓的巴甫洛夫效应。随着对多巴胺的作用和奖励预测错误的更多了解，与寻求奖励行为（例如毒瘾和强迫性赌博）相关的重要临床意义随之而来。多兰建议，大脑中多巴胺浓度升高的个体更容易表现出冲动行为，而持续的多巴胺激增可能会以愉悦感过度刺激大脑，从而促使一些人冒险，例如，强迫性赌博。相比之下，多巴胺浓度降低与衰老之间存在关联，这表明这种影响可能会导致晚年的风险规避，并随后通过奖励预测错误降低个人的学习动机。在快速接触新技术的社会中，这种厌恶会对人口老龄化产生负面影响。那么大脑真正想要的是什么？该小组得出的结论是，无论预测误差是正的还是负的，大脑总是在期待对其行为的奖励。这一见解为临床研究开辟了一条新途径，研究与多巴胺能神经元刺激过度或不足相关的病症。但就目前而言，信息很明确：奖励预测错误可以让我们对自己的学习方法、动机来源和后续行为有更深入的了解。这一见解为临床研究开辟了一条新途径，研究与多巴胺能神经元刺激过度或不足相关的病症。但就目前而言，信息很明确：奖励预测错误可以让我们对自己的学习方法、动机来源和后续行为有更深入的了解。这一见解为临床研究开辟了一条新途径，研究与多巴胺能神经元刺激过度或不足相关的病症。但就目前而言，信息很明确：奖励预测错误可以让我们对自己的学习方法、动机来源和后续行为有更深入的了解。