A protected, OpenAI-compatible model server in one command
Pick a model and get a server that needs a key. Badgr checks that the model fits your GPU, and only calls it ready after a real test request works.
On your computer: available
Needs an NVIDIA GPU, a recent driver and a model that fits its memory. Free.
On Badgr cloud: available
Runs on a Badgr cloud GPU chosen for the model, over HTTPS, billed by use with a spending limit.
Problems this handles
A model that does not fit
The size is estimated from the model name and checked against your GPU before anything starts. On cloud it picks a large enough GPU, or says the model is too big for one.
A server that says it is up but is not
Ready means a real one-token request succeeded, using the access key.
An open model server
A key is always required. Locally it is generated and kept in an owner-only file. On cloud the address is HTTPS and needs your Badgr key.
Try it
badgr login badgr launch vllm --target local --model Qwen/Qwen2.5-7B-Instruct badgr launch vllm --target cloud --model Qwen/Qwen2.5-7B-Instruct --max-cost 5
Local runs need Docker and no Badgr account. Cloud runs show the hourly rate and ask you to confirm before anything starts.
Good to know
- Always needs an explicit model.
- One GPU only.
- Badgr runs the pinned version v0.9.2. It does not change the app itself, so bugs inside vLLM itself are not fixed by Badgr.
- Source and licence: https://github.com/vllm-project/vllm (Apache-2.0).