Issue evidence · Serve · vllm-project/vllm

bench forward verify cases exceeding draft capacity trigger CUDA illegal memory access

GitHub issue: https://github.com/vllm-project/vllm/issues/61766

Not reproduced

vllm bench forward with a verify case larger than the draft capacity hits a CUDA illegal memory access.

What Badgr ran

Ran the issue's command (vLLM 0.32.0rc1.dev190 (cu134 nightly image) on an RTX 4090 through Badgr, with the repository tests at commit 7f730b4 (main, 2026-10-15)): vllm bench forward --model Qwen/Qwen3-0.6B --load-format dummy --case 'verify=2*4@64:3' --speculative-config ngram_gpu k=3, with the issue's other flags.

What came back

The benchmark completed: "verify: 2 reqs, 8 tokens, median 1861.1 us", with no CUDA error.

The reporter used an L4 (sm_89, 24 GB) and an earlier build (0.32.0rc1.dev100); this was a 4090 on a later nightly, so a fix in between or a GPU difference is possible.

Checked 2026-10-15 in Badgr’s local development environment.

← All issue evidence