Figure 1 · MiniMax sparse attention
Each GQA group reads only the KV blocks its indexer picks
Toy example: 2 GQA groups of 3 query heads, 8 KV blocks, top-4. M3.1-Flash: 4 groups of 16 heads, 128-token blocks, top-16. Not to scale.
Method: MiniMax Sparse Attention (arXiv 2606.13392). Sizes: MiniMax-M3.1-Flash config.json (sparse_local_block = 1, sparse_init_block = 0).
index branch
selected blocks (top-k)
KV cache
not read
query heads, outputs