Hugging Face Blog·· 2024-07-30AI score38
Memory-efficient Diffusion Transformer with Quanto and Diffusers
Memory-efficient Diffusion Transformers with Quanto and Diffusers
AI briefing
Hugging Face demonstrates how to use the Quanto quantization tool in the Diffusers library to reduce memory usage of Transformer diffusion models. Using PixArt-Sigma, Stable Diffusion 3, and Aura Flow as examples, quantizing the diffusion backbone and text encoder to FP8 reduces SD3 inference VRAM from 16.403 GB to 8.204 GB, with virtually no quality loss and a slight increase in latency.
Open original huggingface.co