Issue evidence · Serve · huggingface/text-generation-inference

using trtllm-build instead of optimum-nvidia for engine building, or optimum-nvidia wrong version?

GitHub issue: https://github.com/huggingface/text-generation-inference/issues/3304

Not run

TGI's TensorRT-LLM backend (0.17.0.post1) refuses an engine built by optimum-nvidia, which is built with TensorRT-LLM 0.16.0.

What Badgr ran

Nothing was run: it needs a TensorRT-LLM engine build for Llama 3.2 3B and the trtllm image on a matching GPU.

What came back

  • The reporter's log shows "Engine version 0.16.0 found in the config file" against the backend's v0.17.0.post1.

The version mismatch is stated in the reporter's own log; a fix is to build the engine with the TensorRT-LLM version TGI ships, or serve the model with a backend that has no engine step.

Checked 2026-10-14 in Badgr’s local development environment.

← All issue evidence