Reinforcement Learning (RL) has demonstrated a great potential for automatically solving decision-making problems in complex, uncertain environments. RL proposes a computational approach that allows learning through interaction in an environment with stochastic behavior, where agents take actions to maximize some cumulative short-term and long-term rewards. Some of the most impressive results have been shown in Game Theory where agents exhibited superhuman performance in games like Go or Starcraft 2, which led to its gradual adoption in many other domains, including Cloud Computing. Therefore, RL appears as a promising approach for Autoscaling in Cloud since it is possible to learn transparent (with no human intervention), dynamic (no static plans), and adaptable (constantly updated) resource management policies to execute applications. These are three important distinctive aspects to consider in comparison with other widely used autoscaling policies that are defined in an ad-hoc way or statically computed as in solutions based on meta-heuristics. Autoscaling exploits the Cloud elasticity to optimize the execution of applications according to given optimization criteria, which demands deciding when and how to scale up/down computational resources and how to assign them to the upcoming processing workload. Such actions have to be taken considering that the Cloud is a dynamic and uncertain environment. Motivated by this, many works apply RL to the autoscaling problem in the Cloud. In this work, we exhaustively survey those proposals from major venues, and uniformly compare them based on a set of proposed taxonomies. We also discuss open problems and prospective research in the area.

强化学习（RL）展示了在复杂，不确定的环境中自动解决决策问题的巨大潜力。RL提出了一种计算方法，该方法允许在具有随机行为的环境中通过交互进行学习，在这种行为中，代理采取行动以最大化一些累积的短期和长期奖励。游戏理论显示了一些最令人印象深刻的结果，在这些理论中，特工在Go或Starcraft 2等游戏中表现出超人的表现，这导致其在包括云计算在内的许多其他领域逐渐被采用。因此，由于可以学习透明（无人工干预），动态（无静态计划）和适应性（不断更新）的资源管理策略来执行应用程序，因此RL似乎是一种在云中进行自动伸缩的有前途的方法。与以临时方式定义或在基于元启发式方法的解决方案中静态计算的其他广泛使用的自动缩放策略相比，这是三个要考虑的重要方面。Autoscaling利用Cloud弹性根据给定的优化标准来优化应用程序的执行，这需要确定何时以及如何按比例扩展/缩减计算资源，以及如何将其分配给即将到来的处理工作负载。考虑到云是一个动态且不确定的环境，必须采取此类措施。因此，许多工作将RL应用于云中的自动缩放问题。在这项工作中，我们详尽地调查了主要场所的建议，并根据一组建议的分类法对它们进行了统一比较。