Run on Badgr

vLLM Endpoint

Serve an OpenAI-compatible vLLM endpoint for open models with health checks, live logs, and automatic teardown on idle or budget limit.

Serve an endpoint Get help running this

Specific vLLM Endpoint jobs

Serve Qwen 7B

Deploy Qwen 7B as an OpenAI-compatible vLLM endpoint — a fast, low-cost general-purpose chat/completions model for most application traffic.

Serve Qwen Coder

Deploy a Qwen Coder model as an OpenAI-compatible vLLM endpoint — tuned for code completion, refactors, and coding-assistant integrations.

Serve Llama 3.1 8B

Deploy Llama 3.1 8B as an OpenAI-compatible vLLM endpoint — a widely compatible, low-cost general chat model.

Serve Llama 3 70B

Deploy Llama 3 70B as an OpenAI-compatible vLLM endpoint — higher-quality responses for production traffic that needs more reasoning depth than the 8B tier.

Serve a DeepSeek model

Deploy a DeepSeek model as an OpenAI-compatible vLLM endpoint — strong reasoning and code performance for chat/completions traffic.

Serve a Mistral model

Deploy a Mistral model (e.g. Mistral 7B or Mixtral) as an OpenAI-compatible vLLM endpoint — efficient throughput for chat/completions traffic.

Serve a Gemma model

Deploy a Gemma model as an OpenAI-compatible vLLM endpoint — a lightweight open model good for cost-sensitive chat/completions traffic.

Get an OpenAI-compatible API for any open model

Deploy any supported open-source model behind a standard OpenAI-compatible /v1/chat/completions and /v1/completions API — swap your existing OpenAI SDK base URL and keep your code unchanged.

Compatible GPU routes

H100 80GB H200 141GB A100 80GB A100 40GB L40S 48GB

Example command

badgr serve Qwen/Qwen3-32B \
  --gpu H100 \
  --max-cost 10

Common failure modes

Endpoint fails health check on cold start

Increase startup timeout — large models can take several minutes to load weights before /v1/models responds.

High first-token latency under load

Move to a Fast Serving route (H100) or reduce max concurrent sequences.

Context length errors on long prompts

Serve model Check live route

View details →