Skip to content
Luma· LumaLabsAI·· 2026-10-20AI score22

Luma's new method for sequence parallelism of multimodal masks

Exact sequence parallelism for packed multimodal masks.

AI briefing

Exact sequence parallelism for packed multimodal masks. It selects ring attention, sparse all-to-all, or Ulysses layer by layer, with numerics identical to a single GPU, and outperforms the FlashMask + Ulysses hybrid scheme on 47 of 48 cases on up to 128 H100s. https://lumalabs.ai/news/packed-multimodal-attention

Open original x.com