Hugging Face Blog·· 2023-10-24AI score34
SDXL Inference Acceleration and Memory Optimization in Practice: fp16, SDPA, and torch.compile
Exploring simple optimizations for SDXL
AI briefing
On an A100 (40GB), SDXL's default full-precision inference requires 28GB of VRAM and takes 72.2 seconds; switching to fp16 reduces this to 21.7GB and 14.8 seconds, adding SDPA (enabled by default in PyTorch 2.0) further reduces it to 11.4 seconds, and wrapping the UNet with torch.compile shortens it to 10.2 seconds.
Open original huggingface.co