In this article, we propose a passivity-based methodology for the analysis and design of reinforcement learning dynamics and algorithms in multiagent finite games. Starting from a known, first-order reinforcement learning scheme, we show that convergence to a Nash distribution can be attained in a broader class of games than previously considered in the literature - namely, in games characterized by the monotonicity property of their (negative) payoff vectors. We further exploit passivity techniques to design a class of higher order learning schemes that preserve the convergence properties of their first-order counterparts. Moreover, we show that the higher order schemes improve upon the rate of convergence and can even achieve convergence where the first-order scheme fails. We demonstrate these properties through numerical simulations for several representative games.

中文翻译：

---

多智能体有限博弈中的被动性、强化学习和高阶学习


在本文中，我们提出了一种基于被动性的方法，用于分析和设计多智能体有限游戏中的强化学习动力学和算法。从已知的一阶强化学习方案开始，我们表明可以在比之前文献中考虑的更广泛的游戏类别中实现纳什分布的收敛，即以（负）的单调性为特征的游戏收益向量。我们进一步利用被动技术来设计一类高阶学习方案，以保留其一阶对应项的收敛特性。此外，我们表明，高阶方案可以提高收敛速度，甚至可以在一阶方案失败的情况下实现收敛。我们通过几个代表性游戏的数值模拟来证明这些属性。