We introduce a reinforcement learning framework for retail robo-advising. The robo-advisor does not know the investor's risk preference, but learns it over time by observing her portfolio choices in different market environments. We develop an exploration-exploitation algorithm which trades off costly solicitations of portfolio choices by the investor with autonomous trading decisions based on stale estimates of investor's risk aversion. We show that the algorithm's value function converges to the optimal value function of an omniscient robo-advisor over a number of periods that is polynomial in the state and action space. By correcting for the investor's mistakes, the robo-advisor may outperform a stand-alone investor, regardless of the investor's opportunity cost for making portfolio decisions.

中文翻译：

---

机器人咨询：通过投资组合选择来学习投资者的风险偏好*

我们为零售机器人咨询引入了强化学习框架。机器人顾问不知道投资者的风险偏好，而是通过观察其在不同市场环境中的投资组合选择来逐步了解其偏好。我们开发了一种探索开发算法，可以根据对投资者风险规避的过时估计，权衡投资者的投资组合选择与自主交易决策之间的权衡。我们证明了该算法的值函数在状态和动作空间中多项式为多个周期内收敛于无所不在的机器人顾问的最优值函数。通过纠正投资者的错误，机器人顾问可能会胜过独立投资者，而不管投资者做出投资组合决策的机会成本如何。