Reinforcement learning (RL) has proven its worth in a series of artificial domains, and is beginning to show some successes in real-world scenarios. However, much of the research advances in RL are hard to leverage in real-world systems due to a series of assumptions that are rarely satisfied in practice. In this work, we identify and formalize a series of independent challenges that embody the difficulties that must be addressed for RL to be commonly deployed in real-world systems. For each challenge, we define it formally in the context of a Markov Decision Process, analyze the effects of the challenge on state-of-the-art learning algorithms, and present some existing attempts at tackling it. We believe that an approach that addresses our set of proposed challenges would be readily deployable in a large number of real world problems. Our proposed challenges are implemented in a suite of continuous control environments called realworldrl-suite which we propose an as an open-source benchmark.

强化学习（RL）已在一系列人工领域中证明了其价值，并开始在现实世界中展示出一些成功。但是，由于一系列在实践中很少能满足的假设，因此RL的许多研究进展在现实世界的系统中都难以利用。在这项工作中，我们确定并确定了一系列独立的挑战，这些挑战体现了将RL普遍部署在实际系统中所必须解决的困难。对于每个挑战，我们都会在马尔可夫决策过程的范围内对其进行正式定义，分析挑战对最新学习算法的影响，并提出一些现有的解决方案。我们认为，解决我们提出的一系列挑战的方法将很容易部署在许多现实世界中的问题中。我们建议使用realworldrl-suite作为开放源代码基准。