Some researchers have speculated that capable reinforcement learning (RL) agents pursuing misspecified objectives are often incentivized to seek resources and power in pursuit of those objectives. An agent seeking power is incentivized to behave in undesirable ways, including rationally preventing deactivation and correction. Others have voiced skepticism: humans seem idiosyncratic in their urges to power, which need not be present in the agents we design. We formalize a notion of power within the context of finite Markov decision processes (MDPs). With respect to a neutral class of reward function distributions, our results suggest that farsighted optimal policies tend to seek power over the environment.

中文翻译：

---

最佳有远见的代理人倾向于寻求权力

一些研究人员推测，追求错误指定目标的有能力的强化学习 (RL) 代理通常会被激励去寻求资源和权力来追求这些目标。寻求权力的代理人被激励以不受欢迎的方式行事，包括理性地防止停用和纠正。其他人则表示怀疑：人类对权力的渴望似乎是特殊的，而我们设计的智能体不需要这种冲动。我们在有限马尔可夫决策过程 (MDP) 的背景下将权力的概念形式化。对于一类中性的奖励函数分布，我们的结果表明，有远见的最优政策倾向于寻求对环境的影响。