Hugging Quants · Meta Llama 3.1
Meta-Llama-3.1-70B-Instruct-AWQ-INT4
Unverified · community-discovered, not yet reviewedMeta-Llama-3.1-70B-Instruct-AWQ-INT4 is a discovered model from public model catalogues. Badgr builds a GPU serving profile from source metadata, estimated VRAM, context length, and available execution routes.
Last validated or meaningfully updated: Dynamic source discovery
Availability
Not yet Badgr AI API
Not currently validated
✓ Dedicated endpoint
Available through Badgr
✓ Custom GPU deployment
Available through Badgr
Deployment profile
- Architecture
- Source metadata pending
- Licence
- Check model card
- Minimum VRAM
- ~144GB
- Context
- Source dependent
Recommended: H200 141GB. VRAM is an estimate and increases with context, cache, concurrency, and runtime overhead.
Model identity
- Languages
- en, de, fr, it, pt, hi, es, th
- Updated
- 2 years ago
- HuggingFace revision
- 2123003760
Popularity and trust
Downloads (last month)
182.7K
Likes
110
Spaces using this model
4
Files and formats
Weight formats
Safetensors
Repository files
19
Chat template
Available
GPU deployment scenarios
Estimated from parameter count and quantisation. Will switch to “Verified by Badgr” once a real run backs a tier.
Testing
A100 80GB
Short context, low concurrency
Small production
H100 80GB
8K context, 1-4 concurrent requests
Higher throughput
H200 141GB
32K context, continuous batching
Large production
2× H200 141GB
Tensor parallel, higher concurrency
Estimated — multimodal inputs, long context, and concurrency all increase real VRAM use beyond this estimate.
Also runs well on H100 80GB. Available in United States, Europe.
Serving compatibility
| Runtime | Status | Notes |
|---|---|---|
| vLLM | Estimated | Used for Badgr dedicated endpoints |
| Transformers | Declared by source | Basic fallback |
| llama.cpp | Requires GGUF conversion | No GGUF weights found in repo |
| SGLang | Unknown | Not evaluated |
| TensorRT-LLM | Unknown | Conversion may be required |
| TGI | Unknown | Not evaluated |
Ready-to-run Badgr configurations
Quick test
badgr serve hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4 --max-cost 2Dedicated endpoint command
badgr serve hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4 --gpu H200 --max-cost 10Advanced configuration (defaults to automatic)
- Runtime
- Quantisation
- GPU and GPU count
- Context length
- Maximum concurrency
- Region
- Maximum hourly spend
- Persistent or capped runtime
Estimated pricing
Provider estimate: cold start
2–5 min
Provider estimate: endpoint cost
$2.49 – $2.50/hr
Idle cost
$0 when stopped
Per-token throughput cost is not shown here: Badgr has not benchmarked this model yet, and this page does not display figures it cannot back with real data.
Recommended Badgr routes
H100 80GB · United States
Availablefrom $2.49/hr
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 2–5 min
Reliability: High
Last checked: just now
H100 80GB · Europe
Availablefrom $2.49/hr
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 2–5 min
Reliability: High
Last checked: just now
H100 80GB · Asia Pacific
Availablefrom $2.49/hr
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 2–5 min
Reliability: High
Last checked: just now
H100 80GB · United States
Availablefrom $2.50/hr
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 2–5 min
Reliability: High
Last checked: just now
H100 80GB · United States
Availablefrom $2.50/hr
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 2–5 min
Reliability: High
Last checked: just now
H100 80GB · United States
Availablefrom $2.50/hr
Best for
LoRA Training, vLLM Endpoint, Qwen 2.5 32B Instruct
Startup: 2–5 min
Reliability: High
Last checked: just now
Known limitations
- Source-discovered pages use best-effort metadata until the model is reviewed.
- Licence, gated access, runner compatibility, and exact memory use should be checked before production deployment.