Integrations / vLLM
Serve a model with vLLM
Start a vLLM service that speaks the OpenAI API and keep it running while your app needs it.
The steps follow the tool's documented settings. No tested run is recorded for this page yet.
Why use Badgr for vLLM
- Badgr interface
- Serve
- Where vLLM fits
- Model server
- Why Badgr
- Persistent inference endpoint
- One command starts vLLM on a suitable GPU and returns an OpenAI-compatible address and key.
- A cost cap and badgr down bound what the service costs.
- A fixed model version your app can depend on, instead of a shared API's changing model.
What you can do
- A private OpenAI-style API for an open model
- High-volume batch calls with continuous batching
- A fixed model version your app can rely on
Quickstart: Serve a model
badgr launch vllm --target cloud --model Qwen/Qwen2.5-7B-Instruct --max-cost 5
How it works
- 1.You run one command
- 2.Badgr picks a GPU and starts the service
- 3.You get an address and a key
- 4.Your app calls it until you stop it
What you get
- An OpenAI-compatible address and key
- Logs and a receipt when it stops
Requirements
- The Badgr CLI installed and logged in (npm install -g badgr-cli)
- A funded Badgr balance
- A model id that vLLM supports and fits in GPU memory
- Hub access for gated models
What it costs
- A service bills for as long as it runs, until you stop it with badgr down <id>.
- Pass --max-cost to bound spend. Badgr shows the rate for the GPU it picked before it starts. This page does not quote a fixed price.
See pricing for how billing works.
Proof
No tested run is recorded for this integration yet. A receipt and example output will be added here after a real run, not before.
Troubleshooting
Out of GPU memory
Choose a smaller or quantised model, or ask for a GPU with more memory.
Model not supported
Check the vLLM supported models list. Run badgr serve --list-aliases for models known to work.
Service never becomes ready
Read the logs with badgr logs <id>. The most common causes are a wrong port, a missing model file or too little GPU memory.
Bill keeps growing
Services run until stopped. Run badgr status, then badgr down <id>.
Developer reference
Ready to start with vLLM?
Every run or service has a spending limit you set before it starts.