Meta · Llama 3.1
Llama 3.1 70B Instruct GPTQ
Unverified · community-discovered, not yet reviewedLlama 3.1 70B Instruct is a 70B chat and text-generation model available through supported Badgr execution routes.
Last validated or meaningfully updated: 2026-07-27
Availability
Not yet Badgr AI API
Not currently validated
Not yet Dedicated endpoint
Not currently validated
Not yet Custom GPU deployment
Not currently validated
Deployment profile
- Architecture
- LlamaForCausalLM
- Licence
- Llama 3.1 Community
- Minimum VRAM
- ~53GB
- Context
- 128K tokens
Recommended: A100 80GB. VRAM is an estimate and increases with context, cache, concurrency, and runtime overhead.
GPU deployment scenarios
Estimated from parameter count and quantisation. Will switch to “Verified by Badgr” once a real run backs a tier.
Testing
A100 80GB
Short context, low concurrency
Small production
H100 80GB
8K context, 1-4 concurrent requests
Higher throughput
H200 141GB
32K context, continuous batching
Large production
A100 80GB
Higher concurrency
Estimated — multimodal inputs, long context, and concurrency all increase real VRAM use beyond this estimate.
Also runs well on H100 80GB, H200 141GB. Available in United States, Europe, Asia Pacific.
Serving compatibility
| Runtime | Status | Notes |
|---|---|---|
| vLLM | Estimated | Used for Badgr dedicated endpoints |
| Transformers | Unknown | Basic fallback |
| llama.cpp | Requires GGUF conversion | No GGUF weights found in repo |
| SGLang | Unknown | Not evaluated |
| TensorRT-LLM | Unknown | Conversion may be required |
| TGI | Unknown | Not evaluated |
Estimated pricing
Provider estimate: cold start
3–8 min
Provider estimate: endpoint cost
$1.25 – $1.35/hr
Idle cost
$0 when stopped
Per-token throughput cost is not shown here: Badgr has not benchmarked this model yet, and this page does not display figures it cannot back with real data.
Recommended Badgr routes
A100 80GB · United States
Availablefrom $1.25/hr
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 3–8 min
Reliability: High
Last checked: just now
A100 80GB · Europe
Availablefrom $1.25/hr
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 3–8 min
Reliability: Good
Last checked: just now
A100 80GB · Asia Pacific
Availablefrom $1.25/hr
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 3–8 min
Reliability: High
Last checked: just now
A100 80GB · United States
Availablefrom $1.35/hr
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 3–8 min
Reliability: High
Last checked: just now
A100 80GB · United States
Availablefrom $1.35/hr
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 3–8 min
Reliability: Good
Last checked: just now
A100 80GB · United States
Availablefrom $1.35/hr
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 3–8 min
Reliability: High
Last checked: just now
Known limitations
- GPTQ artifact existence, licence, quality, and runner compatibility have not been validated.