Hugging Face Daily Papers·· 2026-10-15AI score24
RPO: Routing policy optimization for online diffusion reinforcement learning
Routing Policy Optimization for Online Diffusion Reinforcement Learning
AI briefing
To address the high rollout optimization cost caused by perturbed reverse denoising trajectories in diffusion model reinforcement learning, RPO turns attention routing into stochastic exploration actions, samples key blocks according to native attention weights at several denoising levels, and uses a dual-rollout estimator to provide advantage signals. On text-to-image and text-to-video tasks, RPO improves generation quality while training efficiency is nearly 4× higher than Flow-GRPO-type methods, and at inference time it restores dense attention and retains the original deterministic sampler.
Open original arxiv.org