W.K. Estes often championed an approach to model development whereby an existing model was augmented by the addition of one or more free parameters, and a comparison between the simple and more complex, augmented model determined whether the additions were justified. Following this same approach we utilized Estes' (1950) own augmented learning equations to improve the fit and plausibility of a win-stay-lose-shift (WSLS) model that we have used in much of our recent work. Estes also championed models that assumed a comparison between multiple concurrent cognitive processes. In line with this, we develop a WSLS-Reinforcement Learning (RL) model that assumes that the output of a WSLS process that provides a probability of staying or switching to a different option based on the last two decision outcomes is compared with the output of an RL process that determines a probability of selecting each option based on a comparison of the expected value of each option. Fits to data from three different decision-making experiments suggest that the augmentations to the WSLS and RL models lead to a better account of decision-making behavior. Our results also support the assertion that human participants weigh the output of WSLS and RL processes during decision-making.

中文翻译：

---

强化学习和赢-留-输-转移决策过程的比较模型：向 WK Estes 致敬

WK Estes 经常支持一种模型开发方法，即通过添加一个或多个自由参数来增强现有模型，并通过比较简单和更复杂的增强模型来确定添加是否合理。遵循同样的方法，我们利用 Estes (1950) 自己的增强学习方程来提高我们在最近的大部分工作中使用的 win-stay-lose-shift (WSLS) 模型的拟合和合理性。Estes 还支持假设在多个并发认知过程之间进行比较的模型。与此相符，我们开发了一个 WSLS 强化学习 (RL) 模型，该模型假设 WSLS 过程的输出基于最后两个决策结果提供停留或切换到不同选项的概率，与 RL 过程的输出进行比较，该过程确定根据每个选项的期望值的比较选择每个选项的概率。对来自三个不同决策实验的数据的拟合表明，对 WSLS 和 RL 模型的增强可以更好地解释决策行为。我们的结果还支持人类参与者在决策过程中权衡 WSLS 和 RL 过程的输出的断言。对来自三个不同决策实验的数据的拟合表明，对 WSLS 和 RL 模型的增强可以更好地解释决策行为。我们的结果还支持人类参与者在决策过程中权衡 WSLS 和 RL 过程的输出的断言。对来自三个不同决策实验的数据的拟合表明，对 WSLS 和 RL 模型的增强可以更好地解释决策行为。我们的结果还支持人类参与者在决策过程中权衡 WSLS 和 RL 过程的输出的断言。