cyankiwi · Gemma 4
Gemma 4 31B Instruct AWQ 4-bit
Verified by Badgr · ServeGemma 4 31B Instruct AWQ 4-bit is a 31B chat and text-generation model available through supported Badgr execution routes.
Last validated or meaningfully updated: 2026-10-09
✓ Verified by Badgr · vLLM · Serve
Last verified 2026-10-09
badgr serve cyankiwi/gemma-4-31B-it-AWQ-4bit --gpu auto --max-cost 2 --startup-timeout 40- GPU
- NVIDIA RTX PRO 6000Reported by the provider, not confirmed inside the container
- Time to verified
- 295s
- Deployment
- dep-cf1e386781
- ✓ vLLM started
- ✓ Badgr's inference check returned a completion
- ✓ final_state = VERIFIED
- ✓ Teardown completed
Served on Badgr's vLLM route and verified with a real inference request. The elapsed time is estimated from the spend at verification and the hourly rate. The same model failed to start on SGLang v0.5.21 in sgl-project/sglang#43342 (the container exited on two GPU offers; the crash text itself was not visible).
GitHub issue this run was for: sgl-project/sglang#43342
Raw evidence
final_state=VERIFIED failure_class=None providers_tried=1 spend_usd=0.18
Run from Badgr's local development environment. Documents that this deployment path works end-to-end for this model.
Availability
Not yet Badgr AI API
Not currently validated
✓ Dedicated endpoint
Available through Badgr
✓ Custom GPU deployment
Available through Badgr
Deployment profile
- Architecture
- Gemma4ForConditionalGeneration
- Licence
- Apache-2.0
- Minimum VRAM
- ~66GB
- Context
- 128K tokens
Recommended: A100 80GB. VRAM is an estimate and increases with context, cache, concurrency, and runtime overhead.
Model identity
- Base model
- google/gemma-4-31B-it
- Updated
- 2 months ago
- HuggingFace revision
- 6f1b616c64
Popularity and trust
Downloads (last month)
120.2K
Likes
55
Spaces using this model
2
Files and formats
Weight formats
Safetensors
Repository files
13
Chat template
Available
GPU deployment scenarios
Estimated from parameter count and quantisation. This model's one verified real run was reported by the provider as NVIDIA RTX PRO 6000, which isn't one of the tiers below.
Testing
A100 80GB
Short context, low concurrency
Small production
H100 80GB
8K context, 1-4 concurrent requests
Higher throughput
H200 141GB
32K context, continuous batching
Large production
A100 80GB
Higher concurrency
Estimated — multimodal inputs, long context, and concurrency all increase real VRAM use beyond this estimate.
Also runs well on H100 80GB, H200 141GB. Available in United States, Europe, Asia Pacific.
Serving compatibility
| Runtime | Status | Notes |
|---|---|---|
| vLLM | Verified by Badgr | Confirmed by a real badgr serve run (deployment dep-cf1e386781) |
| Transformers | Declared by source | Basic fallback |
| llama.cpp | Requires GGUF conversion | No GGUF weights found in repo |
| SGLang | Unknown | Not evaluated |
| Ollama | Unknown | Not evaluated |
| Diffusers | Unknown | Not evaluated |
| PyTorch | Unknown | Not evaluated |
| TensorRT-LLM | Unknown | Conversion may be required |
| TGI | Unknown | Not evaluated |
Ready-to-run Badgr configurations
Quick test
badgr serve cyankiwi/gemma-4-31B-it-AWQ-4bit --max-cost 2Validated dedicated endpoint command
badgr serve cyankiwi/gemma-4-31B-it-AWQ-4bit --gpu A100 80GB --max-cost 10Advanced configuration (defaults to automatic)
- Runtime
- Quantisation
- GPU and GPU count
- Context length
- Maximum concurrency
- Region
- Maximum hourly spend
- Persistent or capped runtime
Estimated pricing
Provider estimate: cold start
3–8 min
Provider estimate: endpoint cost
$1.40 – $1.40/hr
Idle cost
$0 when stopped
Per-token throughput cost is not shown here: Badgr has not benchmarked this model yet, and this page does not display figures it cannot back with real data.
Recommended Badgr routes
A100 80GB · United States
Estimatedstarting from $1.40/hr
Badgr estimated starting price
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 3–8 min
Reliability: High
Last checked: Live marketplace price not available yet
A100 80GB · Europe
Estimatedstarting from $1.40/hr
Badgr estimated starting price
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 3–8 min
Reliability: Good
Last checked: Live marketplace price not available yet
A100 80GB · Europe
Estimatedstarting from $1.40/hr
Badgr estimated starting price
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 3–8 min
Reliability: High
Last checked: Live marketplace price not available yet
A100 80GB · Europe
Estimatedstarting from $1.40/hr
Badgr estimated starting price
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 3–8 min
Reliability: High
Last checked: Live marketplace price not available yet
A100 80GB · Europe
Estimatedstarting from $1.40/hr
Badgr estimated starting price
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 3–8 min
Reliability: Good
Last checked: Live marketplace price not available yet
A100 80GB · Europe
Estimatedstarting from $1.40/hr
Badgr estimated starting price
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 3–8 min
Reliability: High
Last checked: Live marketplace price not available yet
Known limitations
- Memory use increases with context length and concurrency.