Integrations / llama.cpp

DeployDocumented

Serve GGUF models with llama.cpp

Start the llama.cpp server from the built-in template to serve a quantised GGUF model.

The steps follow the tool's documented settings. No tested run is recorded for this page yet.

Why use Badgr for llama.cpp

Badgr interface
Serve
Where llama.cpp fits
Model server
Why Badgr
GGUF model endpoint
  • Serve a quantised model on a smaller GPU, which costs less per hour than a large one.
  • A built-in template starts the server for you.
  • A cost cap and a stop command bound the spend.

What you can do

  • Serve a quantised model on a smaller, cheaper GPU
  • Run a GGUF file you already trust
  • Compare quantisation levels for quality and speed

Quickstart: Start from the template

badgr template info llama-cpp
badgr serve template llama-cpp --max-cost 5

How it works

  1. 1.You run one command
  2. 2.Badgr picks a GPU and starts the service
  3. 3.You get an address and a key
  4. 4.Your app calls it until you stop it

What you get

  • A service address
  • A receipt when it stops

Requirements

  • The Badgr CLI installed and logged in (npm install -g badgr-cli)
  • A funded Badgr balance
  • A GGUF model file or a Hub repository that has one

What it costs

  • A service bills for as long as it runs, until you stop it with badgr down <id>.
  • Pass --max-cost to bound spend. Badgr shows the rate for the GPU it picked before it starts. This page does not quote a fixed price.

See pricing for how billing works.

Proof

No tested run is recorded for this integration yet. A receipt and example output will be added here after a real run, not before.

Troubleshooting

  • Model too large for the GPU

    Pick a lower quantisation (for example Q4 instead of Q8) or fewer layers on the GPU.

  • Wrong file chosen

    A repository can hold many GGUF files. Name the exact file you want.

  • Service never becomes ready

    Read the logs with badgr logs <id>. The most common causes are a wrong port, a missing model file or too little GPU memory.

  • Bill keeps growing

    Services run until stopped. Run badgr status, then badgr down <id>.

Developer reference

Ready to start with llama.cpp?

Every run or service has a spending limit you set before it starts.