Beamline experiments at central facilities are increasingly demanding of remote, high-throughput, and adaptive operation conditions. To accommodate such needs, new approaches must be developed that enable on-the-fly decision making for data intensive challenges. Reinforcement learning (RL) is a domain of AI that holds the potential to enable autonomous operations in a feedback loop between beamline experiments and trained agents. Here, we outline the advanced data acquisition and control software of the Bluesky suite, and demonstrate its functionality with a canonical RL problem: cartpole. We then extend these methods to efficient use of beamline resources by using RL to develop an optimal measurement strategy for samples with different scattering characteristics. The RL agents converge on the empirically optimal policy when under-constrained with time. When resource limited, the agents outperform a naive or sequential measurement strategy, often by a factor of 100%. We interface these methods directly with the data storage and provenance technologies at the National Synchrotron Light Source II, thus demonstrating the potential for RL to increase the scientific output of beamlines, and layout the framework for how to achieve this impact.

中心设施的光束线实验对远程、高通量和自适应操作条件的要求越来越高。为满足此类需求，必须开发新方法，以便针对数据密集型挑战做出即时决策。强化学习 (RL) 是 AI 的一个领域，它具有在光束线实验和受过训练的代理之间的反馈循环中实现自主操作的潜力。在这里，我们概述了 Bluesky 套件的高级数据采集和控制软件，并通过规范的 RL 问题展示了其功能：cartpole。然后，我们通过使用 RL 为具有不同散射特性的样品开发最佳测量策略，将这些方法扩展到有效利用光束线资源。当时间约束不足时，RL 代理会收敛于经验上的最优策略。当资源有限时，代理的性能通常比单纯的或顺序测量策略高 100%。我们将这些方法直接与 National Synchrotron Light Source II 的数据存储和来源技术相结合，从而展示了 RL 增加光束线科学输出的潜力，并为如何实现这种影响布局了框架。