Issue evidence · Serve · vllm-project/vllm

--shutdown-timeout never drains in-flight requests when --api-server-count > 1

GitHub issue: https://github.com/vllm-project/vllm/issues/61561

Reproduced, workaround proven

With more than one API server, SIGTERM stops the API servers before the engines, so every in-flight request fails and --shutdown-timeout has no effect.

What Badgr ran

Ran the reporter's procedure on an RTX 4090 through Badgr: vllm/vllm-openai v0.31.0, Qwen/Qwen3-0.6B, --max-model-len 2048 --max-num-seqs 16 --shutdown-timeout 30, 8 concurrent completions (max_tokens 300, ignore_eos), SIGTERM once /metrics showed requests running. Once with --api-server-count 2 and once with a single API server.

What came back

  • --api-server-count 2: 0 of 8 completed; 6 returned HTTP 500 within 0.1 s and 2 connections dropped after 30.1 s; the launcher exited after 31.9 s.
  • Single API server: 8 of 8 completed (finish_reason=length, 300 tokens each) and the launcher exited after 4.5 s.

The single API server is the configuration that drained correctly here. A first attempt on a nightly image never started its server and is not reported.

Checked 2026-10-14 in Badgr’s local development environment.

← All issue evidence