In mobile health (mHealth) smart devices deliver behavioral treatments repeatedly over time to a user with the goal of helping the user adopt and maintain healthy behaviors. Reinforcement learning appears ideal for learning how to optimally make these sequential treatment decisions. However, significant challenges must be overcome before reinforcement learning can be effectively deployed in a mobile healthcare setting. In this work we are concerned with the following challenges: (1) individuals who are in the same context can exhibit differential response to treatments (2) only a limited amount of data is available for learning on any one individual, and (3) non-stationary responses to treatment. To address these challenges we generalize Thompson-Sampling bandit algorithms to develop IntelligentPooling. IntelligentPooling learns personalized treatment policies thus addressing challenge one. To address the second challenge, IntelligentPooling updates each user’s degree of personalization while making use of available data on other users to speed up learning. Lastly, IntelligentPooling allows responsivity to vary as a function of a user’s time since beginning treatment, thus addressing challenge three.

在移动健康 (mHealth) 中，智能设备会随着时间的推移向用户重复提供行为治疗，目的是帮助用户采取并保持健康的行为。强化学习似乎非常适合学习如何最佳地做出这些顺序治疗决策。然而，在将强化学习有效部署到移动医疗环境中之前，必须克服重大挑战。在这项工作中，我们关注以下挑战：（1）处于相同环境中的个体对治疗可能表现出不同的反应（2）只有有限数量的数据可用于任何一个个体的学习，（3）非-对治疗的稳定反应。为了解决这些挑战，我们推广 Thompson-Sampling bandit 算法来开发IntelligentPooling 。智能池学习个性化治疗策略，从而解决挑战一。为了解决第二个挑战， IntelligentPooling更新每个用户的个性化程度，同时利用其他用户的可用数据来加快学习速度。最后， IntelligentPooling允许响应度根据用户开始治疗后的时间而变化，从而解决挑战三。