Skip to content
Hugging Face Daily Papers·· 2026-10-15AI score24

RPO: Routing policy optimization for online diffusion reinforcement learning

Routing Policy Optimization for Online Diffusion Reinforcement Learning

AI briefing

To address the high rollout optimization cost caused by perturbed reverse denoising trajectories in diffusion model reinforcement learning, RPO turns attention routing into stochastic exploration actions, samples key blocks according to native attention weights at several denoising levels, and uses a dual-rollout estimator to provide advantage signals. On text-to-image and text-to-video tasks, RPO improves generation quality while training efficiency is nearly 4× higher than Flow-GRPO-type methods, and at inference time it restores dense attention and retains the original deterministic sampler.

Open original arxiv.org