Stochastic optimal control usually requires an explicit dynamical model with probability distributions, which are difficult to obtain in practice. In this work, we consider the linear quadratic regulator (LQR) problem of unknown linear systems and adopt a Wasserstein penalty to address the distribution uncertainty of additive stochastic disturbances. By constructing an equivalent deterministic game of the penalized LQR problem, we propose a Q-learning method with convergence guarantees to learn an optimal minimax controller.

中文翻译：

---

使用 Wasserstein 度量的线性系统的 Minimax Q 学习控制

随机最优控制通常需要具有概率分布的显式动力学模型，这在实践中很难获得。在这项工作中，我们考虑未知线性系统的线性二次调节器 (LQR) 问题，并采用 Wasserstein 惩罚来解决加性随机扰动的分布不确定性。通过构建惩罚 LQR 问题的等效确定性博弈，我们提出了一种具有收敛保证的 Q 学习方法来学习最优极小极大控制器。