Serve a Gemma model
Deploy a Gemma model as an OpenAI-compatible vLLM endpoint — a lightweight open model good for cost-sensitive chat/completions traffic.
Estimated $0.30–$0.55/hr while serving · Long-running endpoint — starts in 2–5 min
What inputs you need
- Model: Gemma variant (e.g. Gemma 2 9B)
- Max-cost budget
- Optional: concurrency overrides
What Badgr returns
- OpenAI-compatible endpoint URL + API key
- Health-check status
- Usage receipt on teardown
Recommended GPU routes
Estimated cost: $0.30–$0.55/hr while serving · Estimated runtime: Long-running endpoint — starts in 2–5 min
Example command
badgr serve google/gemma-2-9b-it \
--gpu L40S \
--max-cost 8Common setup failures Badgr avoids
Assuming Gemma's smaller size means unlimited concurrency — throughput still degrades under heavy load; monitor first-token latency and scale routes accordingly.
Context length errors on long prompts — confirm Gemma's max context before routing long-context traffic to it.
Ready to run this?
Launch from the dashboard, CLI, or Compute API. Max-cost protection included.