Figure 2 · Kernels per layer
GEMM
attention
fused all-reduce + conv, mix, norm
communication
memory-bound ops