The game of curling can be considered a good test bed for studying the interaction between artificial intelligence systems and the real world. In curling, the environmental characteristics change at every moment, and every throw has an impact on the outcome of the match. Furthermore, there is no time for relearning during a curling match due to the timing rules of the game. Here, we report a curling robot that can achieve human-level performance in the game of curling using an adaptive deep reinforcement learning framework. Our proposed adaptation framework extends standard deep reinforcement learning using temporal features, which learn to compensate for the uncertainties and nonstationarities that are an unavoidable part of curling. Our curling robot, Curly, was able to win three of four official matches against expert human teams [top-ranked women’s curling teams and Korea national wheelchair curling team (reserve team)]. These results indicate that the gap between physics-based simulators and the real world can be narrowed.

冰壶游戏可以被视为研究人工智能系统与现实世界之间相互作用的良好测试平台。在冰壶比赛中，环境特征随时都在变化，而且每次掷球都会对比赛结果产生影响。此外，由于游戏的时间规则，在冰壶比赛期间没有重新学习的时间。在这里，我们报告了一种卷发机器人，该机器人可以使用自适应深度强化学习框架在卷发游戏中达到人类水平的性能。我们提出的适应框架扩展了使用时间特征的标准深度强化学习，该学习方法可弥补卷发不可避免的不确定性和非平稳性。我们的冰壶机器人Curly 赢得了四场正式比赛中的三场，分别是与人类专家队[排名第一的女子冰壶队和韩国国家轮椅冰壶队（后备队）]。这些结果表明，基于物理的模拟器与现实世界之间的差距可以缩小。