4th collaboration workshop on Reinforcement Learning for Autonomous Accelerators (RL4AA'26)

Session

Contributed talks

30 Mar 2026, 13:30

Theatre 2, Teaching Hub 502 (University of Liverpool)

Conveners

Contributed talks: Student/Junior Talks

• Joseph Wolfenden (University of Liverpool)

Contributed talks: RL Applied to Particle Accelerators

• Simon Hirlaender (PLUS University Salzburg)

Contributed talks: RL Applied to Other Systems

• Alexander Brynes (Science & Technology Facilities Council)

Contributed talks: RL Applied to Other Systems and Others

• Amelia Pollard (ASTeC)

18. Reliable Control via Contextual MPC and Approximate Dynamic Programming: Integrating Bayesian Foresight

Sarah Trausner (University of Salzburg)

30/03/2026, 13:30

Talk

Industrial energy management presents a challenging control problem characterized by strict safety hierarchies, stochastic load fluctuations, and setting actions often has a significantly delayed effects. This work investigates a Reliable Hierarchical Control Architecture composed of probabilistic forecasting paired with a decision-maker.
The processed problem is characterized by an...

26. Dreaming of Schottky Spectra: Building World Models for LEIR robust automation

Borja Rodriguez Mateos (CERN)

30/03/2026, 13:50

Talk

Stripper foil degradation in the Low Energy Ion Ring (LEIR) causes beam distribution drift that progressively degrades performance during multi-turn stacking at flat bottom. World models have emerged as a promising approach for sample-efficient and robust agents, enabling them to improve their behavior by rolling out policies in learned environment models between real interactions, thereby...

32. Testing and Improving RL Policies via Rule Learning

Ignacio D. Lopez-Miguel

30/03/2026, 14:10

Talk

Using domain knowledge to improve deep RL policies is a current challenge. LEGIBLE mines rules from an RL policy, constituting a partially symbolic representation. These rules describe which decisions the RL policy makes and which it avoids making. It then generalizes the mined rules using domain knowledge. Finally, it evaluates generalized rules to determine which generalizations improve...

34. Resource-Conditioned Reinforcement Learning for Physics Instrument Design

Sara Zoccheddu (University of Zurich)

30/03/2026, 14:30

Talk

Designing advanced particle-physics instruments requires navigating a high-dimensional space of discrete and continuous choices while satisfying strict constraints on material, cost, and geometry. In practice, these constraints evolve throughout an experiment’s lifetime, making it insufficient to optimize a single “best” detector configuration. We present a resource-conditioned reinforcement...

21. Extending Reinforcement Learning for Beam Steering with Bayesian Optimization and Online System Identification in the CERN SPS North Area

Adrián Menor de Oñate (CERN)

31/03/2026, 14:00

Talk

Commissioning slow extracted beams from the CERN Super Proton Synchrotron (SPS) to the North Area experimental targets requires trajectory control through multiple transfer lines using corrector magnets—a process that traditionally demands significant expert intervention. Previous work demonstrated using reinforcement learning (RL) for automated trajectory correction based on secondary...

23. Real-Time Ion Beam Orbit Correction in the RAON LEBT Section Using Surrogate Modeling and Reinforcement Learning

Chong Shik Park (Korea University, Sejong)

31/03/2026, 14:20

Talk

Precision orbit correction is critical for maintaining beam stability and transmission efficiency in low-energy beam transport (LEBT) systems, particularly under the influence of nonlinear field effects and strong space-charge forces. Traditional correction techniques often struggle to cope with such complexities, especially when real-time responsiveness is required.
In this work, we present...

22. Autonomous Optimization of RF Triple Splitting in the CERN PS

Joel Wulff (CERN)

31/03/2026, 14:40

Talk

Reinforcement learning (RL) is a powerful technique for optimizing complex beam manipulations. An RL-based autonomous controller has been developed for the triple splitting RF manipulation in the CERN Proton Synchrotron (PS), essential to establish the bunch spacing for the LHC. The system combined a convolutional neural network for initial phase correction with sequential soft actor-critic...

50. Reinforcement Learning for Control of Polarized Cryogenic Targets at Jefferson Lab

Armen Kasparian (Jefferson Lab)

31/03/2026, 15:30

Talk

The operation of solid, cryogenic polarized targets in nuclear physics experiments relies on continuous tuning of the microwave frequency to compensate for radiation damage and evolving material properties, a task that is traditionally performed through manual trial-and-error by expert operators. This work presents a data driven control framework that combines surrogate modeling with...

30. Leveraging Reinforcement Learning, Genetic Algorithms and Transformers for background determination in particle physics

Guillermo Hijano Mendizabal (University of Zurich)

31/03/2026, 15:50

Talk

Experimental studies of beauty hadron decays face significant challenges due to a wide range of backgrounds arising from the numerous possible decay channels with similar final states. For a particular signal decay, the process for ascertaining the most relevant background processes necessitates a detailed analysis of final state particles, potential misidentifications, and kinematic overlaps,...

43. Multi-Agent Reinforcement Learning for Resource Allocation in Wireless Network Communication

Sabrina Pochaba (Salzburg Research)

31/03/2026, 16:10

Talk

Multi-Agent Reinforcement Learning (MARL) is an important subfield of Reinforcement Learning, in which multiple agents learn in a shared environment. The simultaneous learning of several players naturally arises in domains like robotics, network communication and traffic control, where agents affect and influence one another. Thus, MARL can simulate real-world problems in a reliable way, and...

29. Deploying Recurrent Neural Networks on AMD FPGAs: A GRU Implementation on the Versal AI Engine

Michail Sapkas (UniPD - INFN Padova)

01/04/2026, 13:00

Talk

This talk presents an overview of neural network deployment on reconfigurable hardware, with a particular focus on modern AMD FPGA platforms and the Versal Adaptive Compute Acceleration Platform (ACAP). The discussion begins with examples from physics applications where reinforcement learning and recurrent neural networks are jointly employed for real-time control and decision-making.

An...

58. Binary Trigger Signals for Deep Reinforcement Learning in Equity Trading

Juan Manuel Montoya Bayardo

01/04/2026, 13:20

Talk

This study introduces a novel binary trigger-based state representation for deep reinforcement learning (DRL) in stock trading. Unlike conventional approaches using continuous technical indicators (MACD, RSI, CCI, ADX), we encode market state via binary signals: MVX (moving-average crossover) and BOLLX (Bollinger band breakout). We also propose trigger-date filtering, which trains only on...

63. RL control for AGS Injection in silico

Eiad Hamwi (Brookhaven National Laboratory)

01/04/2026, 13:40

Talk

We present an MBRL-based approach for optimizing beam injection into the Alternating Gradient Synchrotron (AGS) using a fully differentiable in silico environment. Beam diagnostics from turn-by-turn beam position monitors throughout the AGS lattice are leveraged to characterize injection quality by quantifying beam survival and localizing beam losses along the ring. These observables are...