In this article we report on the application of a model-free reinforcement learning method to the optimization of accelerator systems. We simplify a policy gradient algorithm to accelerator control from sophisticated algorithms that have recently been demonstrated to solve complex dynamic problems. After outlining a theoretical basis for the functioning of the algorithm, we explore the small hyperparameter space to develop intuition about said parameters using a simple number-guess environment. Finally, we demonstrate the algorithm optimizing both a free-electron laser and an accelerator-based terahertz source in-situ. The algorithm is applied to different accelerator control systems and optimizes the desired signals in a few hundred steps without any domain knowledge using up to five control parameters. In addition, the algorithm shows modest tolerance to accelerator fault conditions without any special preparation for such conditions.

中文翻译：

---

Elettra的FERMI自由电子激光器的自由电子激光器和太赫兹源优化和稳定的策略梯度方法

在本文中，我们报告了无模型强化学习方法在加速器系统优化中的应用。我们将策略梯度算法从最近已被证明可以解决复杂动态问题的复杂算法简化为加速器控制。在概述了算法功能的理论基础之后，我们探索了小的超参数空间，以使用简单的数字猜测环境来发展有关所述参数的直觉。最后，我们演示了在现场优化自由电子激光器和基于加速器的太赫兹源的算法。该算法适用于不同的加速器控制系统，并使用多达五个控制参数以数百步优化所需信号，而无需任何领域知识。此外，