Figure 9 · Rollout vs trainer
The trainer reproduces the rollout's log-probs bit for bit
Batch-invariant
Default
Rollout · vLLM
Trainer · Megatron
Sample tokens + log p
batch-invariant kernels
Forward on the same tokens
the rollout's kernels, same order
old log p
Log-prob values are illustrative.