Luma· LumaLabsAI·· 2026-10-20AI score22
Luma's new method for sequence parallelism of multimodal masks
Exact sequence parallelism for packed multimodal masks.
AI briefing
Exact sequence parallelism for packed multimodal masks. It selects ring attention, sparse all-to-all, or Ulysses layer by layer, with numerics identical to a single GPU, and outperforms the FlashMask + Ulysses hybrid scheme on 47 of 48 cases on up to 128 H100s. https://lumalabs.ai/news/packed-multimodal-attention
Open original x.com