Hugging Face Daily Papers·· 28 days agoAI score31
FAPL: making MLLM latent variables truly drive cinematic video generation
Making MLLM Latents Matter: Factor-Aware Predictive Learning for Cinematic Video Generation
AI briefing
Researchers propose Factor-Aware Predictive Learning (FAPL), which, through grounded training of a latent variable generator with a frozen MLLM and connector, enables the Video DiT to truly utilize MLLM latent variables rather than bypassing the native conditioning pathway. The method combines factor-span replacement to achieve factor-aware control over characters, scenes, actions and camera, and outperforms baselines on all three cinematic video generation tasks: sparse input to video, factor-aware editing and multi-shot generation.
Open original arxiv.org