Speaker
Description
Particle accelerators and their design studies generate large amounts of historical data from archived operation logs and high-fidelity simulations, yet most learning-based control strategies still rely on online optimisation, where new data must be collected through direct machine interaction. To make better use of such pre-generated data and avoid additional online exploration, we present a workflow for offline reinforcement learning (offline RL) based on a two-stage modelling approach. First, an Xsuite-based high-fidelity beam dynamics model is used to generate and archive trajectories for steering tasks across a set of representative machine scenarios (e.g., optics variations, alignment errors, and jitter conditions), providing synthetic but realistic expert and non-expert behaviour. Second, a Koopman-inspired hybrid world model is learned from this dataset, yielding fast, stable multi-step prediction together with epistemic uncertainty estimates via ensemble variance. This learned model serves as a surrogate environment for offline RL. We benchmark offline RL policies against a PPO agent trained directly in the original Xsuite physics model, where PPO episodes are terminated once trajectories leave expert-like regions or enter high-epistemic-uncertainty domains, reflecting realistic operational safety limits. Results show that policies trained purely offline on the Koopman world model can match or exceed PPO performance under these constraints, while requiring no additional online exploration. The proposed workflow demonstrates how Xsuite-based simulation, uncertainty-aware surrogate modelling, and offline RL can be combined to turn historical scenario data into a safe and reproducible pathway for learning-based accelerator control.
| Student | Yes |
|---|
