用强化学习自动优化粒子加速器光束线,提升粒子传输效率。
RL-ABC: Reinforcement Learning for Accelerator Beamline Control

- 将加速器光束线建模为马尔可夫决策过程,自动生成状态与奖励函数。
- 在37个控制参数的测试光束线上实现70.3%粒子传输率,媲美传统方法。
- 开源框架支持多种强化学习算法,适合加速器物理与自动化研究者。
粒子加速器光束线优化是一个高维控制问题,传统上需要大量专家干预。本文提出RLABC(Reinforcement Learning for Accelerator Beamline Control),一个开源的Python框架,可将标准Elegant光束线配置自动转化为强化学习环境。RLABC通过基于SDDS的接口与广泛使用的Elegant束流动力学仿真代码集成,使研究者能以最少的RL开发工作应用现代强化学习算法进行光束线优化。核心贡献是将光束线调谐形式化为马尔可夫决策过程:框架自动预处理晶格文件,在每个可调元件前插入诊断观测点,从束流统计、协方差信息和孔径约束构建57维状态表示,并提供可配置的奖励函数用于传输率优化。框架通过Stable-Baselines3兼容支持多种强化学习算法,并实现分阶段学习策略以提升训练效率。在基于VEPP-5注入系统构建的测试光束线(11个四极磁铁和4个二极磁铁共37个控制参数)上验证表明,该框架成功实现基于强化学习的优化,深度确定性策略梯度(DDPG)代理达到70.3%的粒子传输率,性能与差分进化等成熟方法相当。其分阶段学习能力可将复杂优化问题分解为可管理子问题,显著提升训练效率。完整框架包含配置文件与示例笔记本,已作为开源软件发布,促进推广应用与后续研究。
原文摘要 · Abstract (English)
Particle accelerator beamline optimization is a high-dimensional control problem traditionally requiring significant expert intervention. We present RLABC (Reinforcement Learning for Accelerator Beamline Control), an open-source Python framework that automatically transforms standard Elegant beamline configurations into reinforcement learning environments. RLABC integrates with the widely-used Elegant beam dynamics simulation code via SDDS-based interfaces, enabling researchers to apply modern RL algorithms to beamline optimization with minimal RL-specific development. The main contribution is a general methodology for formulating beamline tuning as a Markov decision process: RLABC automatically preprocesses lattice files to insert diagnostic watch points before each tunable element, constructs a 57-dimensional state representation from beam statistics, covariance information, and aperture constraints, and provides a configurable reward function for transmission optimization. The framework supports multiple RL algorithms through Stable-Baselines3 compatibility and implements stage learning strategies for improved training efficiency. Validation on a test beamline derived from the VEPP-5 injection complex (37 control parameters across 11 quadrupoles and 4 dipoles) demonstrates that the framework successfully enables RL-based optimization, with a Deep Deterministic Policy Gradient agent achieving 70.3\% particle transmission -- performance matching established methods such as differential evolution. The framework's stage learning capability allows decomposition of complex optimization problems into manageable subproblems, improving training efficiency. The complete framework, including configuration files and example notebooks, is available as open-source software to facilitate adoption and further research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。