Issue evidence · Serve · vllm-project/vllm

lightning_attention prefill applies one decay step too few to the stored state (tokens after position 256)

GitHub issue: https://github.com/vllm-project/vllm/issues/61730

Reproduced

The lightning attention prefill kernel's output diverges from the exact recurrence for tokens past position 256.

What Badgr ran

Ran the issue's fixture (vLLM 0.32.0rc1.dev190 (cu134 nightly image) on an RTX 4090 through Badgr, with the repository tests at commit 7f730b4 (main, 2026-10-15)): 8 heads, 512 tokens, head dim 64, MiniMax slopes, comparing the kernel with a float64 recurrence and a float32 recurrence.

What came back

  • Tokens 0 to 255: kernel max error / rms 2.27e-02 (float32 recurrence 2.7e-06).
  • Tokens 256 to 511: kernel error 2.609 relative to rms (float32 recurrence 3.3e-06), with per-head errors from 0.75 to 2.6.

The error is large for the later half of the sequence, as the issue says; TRITON_F32_DEFAULT was unset. Only the standalone kernel was tested, not a MiniMax model.

Checked 2026-10-15 in Badgr’s local development environment.

← All issue evidence