Skip to content
Hugging Face Daily Papers·· 28 days agoAI score31

FAPL: making MLLM latent variables truly drive cinematic video generation

Making MLLM Latents Matter: Factor-Aware Predictive Learning for Cinematic Video Generation

AI briefing

Researchers propose Factor-Aware Predictive Learning (FAPL), which, through grounded training of a latent variable generator with a frozen MLLM and connector, enables the Video DiT to truly utilize MLLM latent variables rather than bypassing the native conditioning pathway. The method combines factor-span replacement to achieve factor-aware control over characters, scenes, actions and camera, and outperforms baselines on all three cinematic video generation tasks: sparse input to video, factor-aware editing and multi-shot generation.

Open original arxiv.org