Turbulent flow models are critical for applications such as aircraft design, weather forecasting and climate prediction. Existing models are largely based on physical insight and engineering intuition. More recently, machine learning has been contributing to this endeavour with promising results. However, all efforts have focused on supervised learning, which is difficult to generalize beyond training data. Here we introduce multi-agent reinforcement learning as an automated discovery tool of turbulence models. We demonstrate the potential of this approach on large-eddy simulations of isotropic turbulence, using the recovery of statistical properties of direct numerical simulations as a reward. The closure model is a control policy enacted by cooperating agents, which detect critical spatio-temporal patterns in the flow field to estimate the unresolved subgrid-scale physics. Results obtained with multi-agent reinforcement learning algorithms based on experience replay compare favourably with established modelling approaches. Moreover, we show that the learned turbulence models generalize across grid sizes and flow conditions.

湍流模型对于飞机设计、天气预报和气候预测等应用至关重要。现有模型主要基于物理洞察力和工程直觉。最近，机器学习一直在为这一努力做出贡献，并取得了可喜的成果。然而，所有的努力都集中在监督学习上，这很难在训练数据之外推广。在这里，我们介绍多智能体强化学习作为湍流模型的自动发现工具。我们证明了这种方法在各向同性湍流的大涡模拟中的潜力，使用直接数值模拟的统计特性的恢复作为奖励。闭包模型是由合作代理制定的控制策略，检测流场中的关键时空模式以估计未解决的亚网格尺度物理。使用基于经验回放的多智能体强化学习算法获得的结果与已建立的建模方法相比具有优势。此外，我们表明学习的湍流模型可以泛化网格大小和流动条件。