GPU Compute / Recipe

Deploy a vLLM endpoint

badgr serve uses vLLM under the hood — the same inference engine used by most production LLM deployments. This recipe explains what happens when you run it.

Prerequisites: npm install -g badgr-cli then badgr login. Setup guide →

What badgr serve does

  1. 1Searches available GPU capacity for the type you requested.
  2. 2Provisions an instance with the vllm/vllm-openai:latest container image.
  3. 3Sets MODEL environment variable to your model ID.
  4. 4Waits for the vLLM health check to return 200 OK.
  5. 5Prints an OpenAI-compatible endpoint URL.

Command

badgr serve Qwen/Qwen2.5-7B-Instruct --gpu A100 --region EU --max-cost 20

Bring your own vLLM image

Use badgr run with a custom image if you need a specific vLLM version or extra config:

badgr run \
  --image vllm/vllm-openai:v0.6.3 \
  --gpu A100 \
  --env MODEL=Qwen/Qwen2.5-7B-Instruct \
  --env MAX_MODEL_LEN=8192 \
  --max-cost 10

Check the endpoint

badgr serve prints the endpoint URL when the model is ready. Export it as BADGR_ENDPOINT:

# Set BADGR_ENDPOINT to the URL printed by `badgr serve`

# List available models
curl $BADGR_ENDPOINT/v1/models \
  -H "Authorization: Bearer $BADGR_API_KEY"

# Health check
curl $BADGR_ENDPOINT/health

Stop billing

badgr down <deployment-id>

Use badgr status to see all running deployments and their live billing rates.

Options

--gpu <type>GPU type: RTX_4090, L40S, A6000, A100, A100_80GB, H100. Auto-selected from model size if omitted.
--region US|EU|AURegion preference. Omit for global best-capacity search.
--tier 1|21 = managed providers (default), 2 = marketplace budget providers
--max-price 3.00Hard hourly cap in USD/hr
--max-cost 30.00Auto-stop when total spend reaches this amount
--persistentRun indefinitely — billing continues until you run badgr down
--env KEY=VALUEExtra environment variables passed to vLLM — repeatable
--no-waitReturn immediately without waiting for the health check