To emulate the interactivity of in-person math instruction, we developed MathBot, a rule-based chatbot that explains math concepts, provides practice questions, and offers tailored feedback. We evaluated MathBot through three Amazon Mechanical Turk studies in which participants learned about arithmetic sequences. In the first study, we found that more than 40% of our participants indicated a preference for learning with MathBot over videos and written tutorials from Khan Academy. The second study measured learning gains, and found that MathBot produced comparable gains to Khan Academy videos and tutorials. We solicited feedback from users in those two studies to emulate a real-world development cycle, with some users finding the lesson too slow and others finding it too fast. We addressed these concerns in the third and main study by integrating a contextual bandit algorithm into MathBot to personalize the pace of the conversation, allowing the bandit to either insert extra practice problems or skip explanations. We randomized participants between two conditions in which actions were chosen uniformly at random (i.e., a randomized A/B experiment) or by the contextual bandit. We found that the bandit learned a similarly effective pedagogical policy to that learned by the randomized A/B experiment while incurring a lower cost of experimentation. Our findings suggest that personalized conversational agents are promising tools to complement existing online resources for math education, and that data-driven approaches such as contextual bandits are valuable tools for learning effective personalization.

为了模拟面对面数学教学的互动性，我们开发了MathBot，这是一个基于规则的聊天机器人，可以解释数学概念，提供练习问题并提供量身定制的反馈。我们通过三项Amazon Mechanical Turk研究评估了MathBot，参与者在研究中了解了算术序列。在第一项研究中，我们发现超过40％的参与者表示比起可汗学院的视频和书面教程，他们更喜欢使用MathBot学习。第二项研究测量了学习收益，发现MathBot产生的收益与可汗学院的视频和教程相当。我们在这两项研究中征求了用户的反馈，以模拟现实世界的开发周期，其中一些用户发现课程太慢，而另一些用户发现课程太快。在第三项也是主要的研究中，我们通过将上下文盗贼算法集成到MathBot中来个性化对话的速度，从而让匪徒插入了额外的练习题或跳过了解释，从而解决了这些担忧。我们将参与者随机分为两个条件，在两个条件下，是随机地（即，随机A / B实验）或通过情境强盗均匀地选择动作。我们发现，土匪学会了与随机A / B实验所学方法相似的有效教学策略，同时降低了实验成本。我们的研究结果表明，个性化的对话代理是有前途的工具，可以补充现有的在线数学教育资源，而数据驱动的方法（如情景强盗）是学习有效个性化的宝贵工具。