Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.

深度强化学习（RL）已成为一种有前途的方法，可以从低水平的传感器观察中自动获取复杂的行为。尽管大量的深度RL研究专注于视频游戏和模拟控制中的应用，这些研究与现实环境中的学习约束无关，但深度RL也显示了使物理机器人能够学习现实世界中复杂技能的潜力。同时，现实世界中的机器人技术可直接评估人类的学习方式，从而为评估此类算法提供了一个吸引人的领域：作为现实世界中的具体体现。在现实世界中学习感知和移动提出了许多挑战，其中一些挑战比其他挑战更容易解决，而某些RL研究通常只关注模拟域，因此通常不会考虑其中的一些挑战。在这篇评论文章中，我们介绍了许多涉及机器人深度RL的案例研究。在这些案例研究的基础上，我们讨论了深度RL中常见的挑战以及如何在这些工作中解决它们。我们还概述了其他尚待解决的挑战，其中许多挑战是现实机器人技术所特有的，并且通常不是主流RL研究的重点。我们的目标是为感兴趣的机器人技术人员和机器学习研究人员提供资源，以推动现实世界中深度RL的发展。其中许多是现实世界机器人技术所独有的，并且通常不是主流RL研究的重点。我们的目标是为感兴趣的机器人技术人员和机器学习研究人员提供资源，以推动现实世界中深度RL的发展。其中许多是现实世界机器人技术所独有的，并且通常不是主流RL研究的重点。我们的目标是为感兴趣的机器人技术人员和机器学习研究人员提供资源，以推动现实世界中深度RL的发展。