Integrations / vLLM

DeployDocumented

Serve a model with vLLM

Start a vLLM service that speaks the OpenAI API and keep it running while your app needs it.

The steps follow the tool's documented settings. No tested run is recorded for this page yet.

Why use Badgr for vLLM

Badgr interface
Serve
Where vLLM fits
Model server
Why Badgr
Persistent inference endpoint
  • One command starts vLLM on a suitable GPU and returns an OpenAI-compatible address and key.
  • A cost cap and badgr down bound what the service costs.
  • A fixed model version your app can depend on, instead of a shared API's changing model.

What you can do

  • A private OpenAI-style API for an open model
  • High-volume batch calls with continuous batching
  • A fixed model version your app can rely on

Quickstart: Serve a model

badgr launch vllm --target cloud --model Qwen/Qwen2.5-7B-Instruct --max-cost 5

How it works

  1. 1.You run one command
  2. 2.Badgr picks a GPU and starts the service
  3. 3.You get an address and a key
  4. 4.Your app calls it until you stop it

What you get

  • An OpenAI-compatible address and key
  • Logs and a receipt when it stops

Requirements

  • The Badgr CLI installed and logged in (npm install -g badgr-cli)
  • A funded Badgr balance
  • A model id that vLLM supports and fits in GPU memory
  • Hub access for gated models

What it costs

  • A service bills for as long as it runs, until you stop it with badgr down <id>.
  • Pass --max-cost to bound spend. Badgr shows the rate for the GPU it picked before it starts. This page does not quote a fixed price.

See pricing for how billing works.

Proof

No tested run is recorded for this integration yet. A receipt and example output will be added here after a real run, not before.

Troubleshooting

  • Out of GPU memory

    Choose a smaller or quantised model, or ask for a GPU with more memory.

  • Model not supported

    Check the vLLM supported models list. Run badgr serve --list-aliases for models known to work.

  • Service never becomes ready

    Read the logs with badgr logs <id>. The most common causes are a wrong port, a missing model file or too little GPU memory.

  • Bill keeps growing

    Services run until stopped. Run badgr status, then badgr down <id>.

Developer reference

Ready to start with vLLM?

Every run or service has a spending limit you set before it starts.