Skip to content
Hugging Face Blog·· 2024-07-30AI score38

Memory-efficient Diffusion Transformer with Quanto and Diffusers

Memory-efficient Diffusion Transformers with Quanto and Diffusers

AI briefing

Hugging Face demonstrates how to use the Quanto quantization tool in the Diffusers library to reduce memory usage of Transformer diffusion models. Using PixArt-Sigma, Stable Diffusion 3, and Aura Flow as examples, quantizing the diffusion backbone and text encoder to FP8 reduces SD3 inference VRAM from 16.403 GB to 8.204 GB, with virtually no quality loss and a slight increase in latency.

Open original huggingface.co