Issue evidence · Serve · vllm-project/vllm

Wide speculative decoding verify (num_speculative_tokens >= 7) samples EOS at prompt_len+8 on hybrid models

GitHub issue: https://github.com/vllm-project/vllm/issues/61738

Not reproduced

With num_speculative_tokens 8 on a hybrid Gated-DeltaNet model, generation stops early with EOS at prompt_len+8.

What Badgr ran

Served Qwen/Qwen3.5-9B on an RTX 4090 through Badgr (vLLM 0.32.0rc1.dev190 cu134 nightly; --enforce-eager --language-model-only, ngram_gpu, 0.86 GPU memory) and sent the issue's 1845-token repeated-block completion request at temperature 0 with k=8 and, for comparison, k=3.

What came back

k=8: 96 completion tokens, finish_reason length (no early EOS).
k=3: 96 completion tokens, finish_reason length, same text.

Only the ngram_gpu variant was run, not the MTP + VLLM_USE_V2_MODEL_RUNNER variant in the issue, and the memory setting differed from the reporter's 0.90 because 0.90 did not fit a 4090. The reporter's GPU was not identified in the excerpt read.

Checked 2026-10-15 in Badgr’s local development environment.

← All issue evidence