Skip to content
Hugging Face Blog·· 2023-05-08AI score30

A deep dive into text-to-video models: development and challenges from GANs to diffusion models

A Dive into Text-to-Video Models

AI briefing

A Hugging Face blog post traces the development of text-to-video models: early GAN- and VAE-based methods were limited by low resolution and short duration, after which Phenaki, Make-A-Video, NUWA, VideoGPT, CogVideo, and others shifted to transformer architectures, while current mainstream approaches use diffusion models. Text-to-video must ensure spatiotemporal consistency across frames and faces three major challenges: high compute costs, scarce high-quality datasets, and vague video descriptions; models can typically generate only very short clips.

Open original huggingface.co