vLLM Endpoint

Serve a Mistral model

Deploy a Mistral model (e.g. Mistral 7B or Mixtral) as an OpenAI-compatible vLLM endpoint — efficient throughput for chat/completions traffic.

Run this jobGet help running this

Estimated $0.40–$1.20/hr while serving, depending on model size · Long-running endpoint — starts in 2–6 min

What inputs you need

  • Model: Mistral or Mixtral variant
  • Max-cost budget
  • Optional: concurrency overrides

What Badgr returns

  • OpenAI-compatible endpoint URL + API key
  • Health-check status
  • Usage receipt on teardown

Recommended GPU routes

L40S 48GBA100 40GBA100 80GB

Estimated cost: $0.40–$1.20/hr while serving, depending on model size · Estimated runtime: Long-running endpoint — starts in 2–6 min

Example command

badgr serve mistralai/Mixtral-8x7B-Instruct-v0.1 \
  --gpu A100_40GB \
  --max-cost 12

Common setup failures Badgr avoids

Out-of-memory loading a mixture-of-experts model — Mixtral-class models route to 40GB+ GPUs by default.

High first-token latency under load — reduce max concurrent sequences or move to a Fast Serving route.

Ready to run this?

Launch from the dashboard, CLI, or Compute API. Max-cost protection included.

Run this jobBack to vLLM Endpoint