Integrations / TensorRT-LLM

DeployExperimental

TensorRT-LLM on Badgr: custom container guide

A setup guide for running your own TensorRT-LLM container. Badgr has no one-click path for it.

A setup guide only. Badgr has not tested this path, so expect to adjust it.

Why use Badgr for TensorRT-LLM

Badgr interface
Serve
Where TensorRT-LLM fits
Model runtime
Why Badgr
Optimised NVIDIA inference
  • Bring your own TensorRT-LLM container and let Badgr run it on an NVIDIA GPU.
  • A cost cap and a stop command bound the spend.
  • Badgr has no one-click setup for it and has not tested this path, which is stated up front before you start.

What you can do

  • Serve an engine you built with TensorRT-LLM
  • Squeeze more throughput from a supported NVIDIA GPU
  • Keep your own container and let Badgr run it

Quickstart: Run your own container

badgr serve --image <your-registry>/trtllm-server:<tag> --cmd "<your server start command>" --max-cost 5

How it works

  1. 1.You give Badgr your container image and start command
  2. 2.Badgr runs it on a GPU
  3. 3.You check that it answers
  4. 4.You stop it when done

What you get

  • A service address if your container serves HTTP
  • Logs and a receipt

Requirements

  • The Badgr CLI installed and logged in (npm install -g badgr-cli)
  • A funded Badgr balance
  • A container image with TensorRT-LLM and your server start command
  • An NVIDIA GPU architecture your engine was built for
  • An engine built for the same GPU type and TensorRT-LLM version you run

What it costs

  • Badgr has no ready-made setup for this. The command below runs your own container with the generic custom option. It has not been tested by Badgr, so treat this page as a setup guide.
  • A service bills for as long as it runs, until you stop it with badgr down <id>.
  • Pass --max-cost to bound spend. Badgr shows the rate for the GPU it picked before it starts. This page does not quote a fixed price.

See pricing for how billing works.

Proof

No tested run is recorded for this integration yet. A receipt and example output will be added here after a real run, not before.

Troubleshooting

  • Engine fails to load

    Engines are tied to a GPU architecture and library version. Rebuild on the GPU type you will run on.

  • Long start time

    Building an engine at start can take a long time. Build it ahead of time and include it in the image.

  • Service never becomes ready

    Read the logs with badgr logs <id>. The most common causes are a wrong port, a missing model file or too little GPU memory.

Developer reference

Ready to start with TensorRT-LLM?

Every run or service has a spending limit you set before it starts.