Issue evidence · Serve · huggingface/text-generation-inference

Quantized Qwen3

GitHub issue: https://github.com/huggingface/text-generation-inference/issues/3226

Reproduced, workaround proven

TGI fails to deploy Qwen/Qwen3-4B-AWQ.

What Badgr ran

Ran ghcr.io/huggingface/text-generation-inference:3.3.4 with --model-id Qwen/Qwen3-4B-AWQ --quantize awq on an RTX 4090 through Badgr, then served the same model with badgr serve on vLLM 0.30.0.

What came back

TGI exited with NotImplementedError: awq quantization is not supported for AutoModel, then "Shard 0 failed to start" and "Error: ShardCannotStart".
badgr serve Qwen/Qwen3-4B-AWQ reached Badgr's verified state on an A6000.

The reporter's text differed (a dtype/quantize conflict), so only the failure to start was reproduced. The Badgr route was verified by Badgr's own check; I did not complete a separate request of my own on it.

Checked 2026-10-14 in Badgr’s local development environment.

← All issue evidence