Integrations / TensorRT-LLM
TensorRT-LLM on Badgr: custom container guide
A setup guide for running your own TensorRT-LLM container. Badgr has no one-click path for it.
A setup guide only. Badgr has not tested this path, so expect to adjust it.
Why use Badgr for TensorRT-LLM
- Badgr interface
- Serve
- Where TensorRT-LLM fits
- Model runtime
- Why Badgr
- Optimised NVIDIA inference
- Bring your own TensorRT-LLM container and let Badgr run it on an NVIDIA GPU.
- A cost cap and a stop command bound the spend.
- Badgr has no one-click setup for it and has not tested this path, which is stated up front before you start.
What you can do
- Serve an engine you built with TensorRT-LLM
- Squeeze more throughput from a supported NVIDIA GPU
- Keep your own container and let Badgr run it
Quickstart: Run your own container
badgr serve --image <your-registry>/trtllm-server:<tag> --cmd "<your server start command>" --max-cost 5
How it works
- 1.You give Badgr your container image and start command
- 2.Badgr runs it on a GPU
- 3.You check that it answers
- 4.You stop it when done
What you get
- A service address if your container serves HTTP
- Logs and a receipt
Requirements
- The Badgr CLI installed and logged in (npm install -g badgr-cli)
- A funded Badgr balance
- A container image with TensorRT-LLM and your server start command
- An NVIDIA GPU architecture your engine was built for
- An engine built for the same GPU type and TensorRT-LLM version you run
What it costs
- Badgr has no ready-made setup for this. The command below runs your own container with the generic custom option. It has not been tested by Badgr, so treat this page as a setup guide.
- A service bills for as long as it runs, until you stop it with badgr down <id>.
- Pass --max-cost to bound spend. Badgr shows the rate for the GPU it picked before it starts. This page does not quote a fixed price.
See pricing for how billing works.
Proof
No tested run is recorded for this integration yet. A receipt and example output will be added here after a real run, not before.
Troubleshooting
Engine fails to load
Engines are tied to a GPU architecture and library version. Rebuild on the GPU type you will run on.
Long start time
Building an engine at start can take a long time. Build it ahead of time and include it in the image.
Service never becomes ready
Read the logs with badgr logs <id>. The most common causes are a wrong port, a missing model file or too little GPU memory.
Developer reference
Ready to start with TensorRT-LLM?
Every run or service has a spending limit you set before it starts.