vLLM Endpoint

Serve Llama 3 70B

Deploy Llama 3 70B as an OpenAI-compatible vLLM endpoint — higher-quality responses for production traffic that needs more reasoning depth than the 8B tier.

Run this jobGet help running this

Estimated $2.00–$3.50/hr while serving · Long-running endpoint — starts in 4–8 min

What inputs you need

  • Model: meta-llama/Meta-Llama-3-70B-Instruct (or pinned variant)
  • Max-cost budget
  • Optional: tensor-parallel / context-length overrides

What Badgr returns

  • OpenAI-compatible endpoint URL + API key
  • Health-check status
  • Usage receipt on teardown

Recommended GPU routes

H100 80GBH200 141GBA100 80GB

Estimated cost: $2.00–$3.50/hr while serving · Estimated runtime: Long-running endpoint — starts in 4–8 min

Example command

badgr serve meta-llama/Meta-Llama-3-70B-Instruct \
  --gpu H100 \
  --max-cost 25

Common setup failures Badgr avoids

Out-of-memory loading 70B weights on a single GPU — Badgr routes 70B-class models to H100/H200/A100-80GB by default with tensor parallelism where supported.

Endpoint fails health check on cold start — startup timeout is sized generously for large-model weight loading.

Ready to run this?

Launch from the dashboard, CLI, or Compute API. Max-cost protection included.

Run this jobBack to vLLM Endpoint