Reinforcement learning causes an action that yields a positive outcome more likely to be taken in the future. Here, we investigate how the time elapsed from an action affects subsequent decisions. Groups of C57BL6/J mice were housed in IntelliCages with access to water and chow ad libitum; they also had access to bottles with a reward: saccharin solution, alcohol or a mixture of the two. The probability of receiving a reward in two of the cage corners changed between 0.9 and 0.3 every 48 h over a period of ~33 days. As expected, in most animals, the odds of repeating a corner choice were increased if that choice was previously rewarded. Interestingly, the time elapsed from the previous choice also influenced the probability of repeating the choice, and this effect was independent of previous outcome. Behavioral data were fitted to a series of reinforcement learning models. Best fits were achieved when the reward prediction update was coupled with separate learning rates from positive and negative outcomes and additionally a "fictitious" update of the expected value of the nonselected choice. Additional inclusion of a time-dependent decay of the expected values improved the fit marginally in some cases.

中文翻译：

---

概率任务中选择之间经过的时间与重复相同的决策相关。

强化学习会导致一项行动，该行动会产生积极的结果，将来更有可能采取这种行动。在这里，我们调查行动所花费的时间如何影响后续决策。将C57BL6 / J组小鼠饲养在IntelliCages中，随意饮水和食物。他们还获得了奖励的瓶子：糖精溶液，酒精或两者的混合物。在约33天的时间内，每48小时内，在两个笼形角落中获得奖励的概率在0.9到0.3之间变化。不出所料，在大多数动物中，如果以前选择过一次，选择重复死角的可能性就会增加。有趣的是，前一个选择所花费的时间也影响了重复选择的可能性，并且这种效果与前一个结果无关。行为数据适合一系列强化学习模型。当奖励预测更新与来自积极和消极结果的单独学习率以及未选择的选择的期望值的“虚拟”更新相结合时，可获得最佳拟合。在某些情况下，附加包含预期值随时间变化的衰减会稍微改善拟合度。