Integrations / LocalAI

DeployExperimental

LocalAI on Badgr: custom container guide

A guide for hosting a LocalAI container behind an OpenAI-style API on a Badgr GPU.

A setup guide only. Badgr has not tested this path, so expect to adjust it.

Why use Badgr for LocalAI

Badgr interface
Serve
Where LocalAI fits
Model server
Why Badgr
Hosted compatible API
  • Run a LocalAI container behind an OpenAI-style API on a rented GPU.
  • A cost cap and a stop command bound the spend.
  • Badgr has not tested this path. It is a guide for your own container.

What you can do

  • An OpenAI-style API for several model types in one server
  • Keep an existing LocalAI model configuration
  • Move a LocalAI setup onto a larger GPU

Quickstart: Run your own container

badgr serve --image localai/localai:<gpu-tag> --cmd "<your start command>" --max-cost 5

How it works

  1. 1.You give Badgr your container image and start command
  2. 2.Badgr runs it on a GPU
  3. 3.You check that it answers
  4. 4.You stop it when done

What you get

  • An OpenAI-style API address if the port is exposed
  • Logs and a receipt

Requirements

  • The Badgr CLI installed and logged in (npm install -g badgr-cli)
  • A funded Badgr balance
  • A LocalAI GPU container image
  • Model configuration files included in the image or fetched at start
  • Storage for model files (cloud files are removed on stop)

What it costs

  • Badgr has no ready-made setup for this. The command below runs your own container with the generic custom option. It has not been tested by Badgr, so treat this page as a setup guide.
  • A service bills for as long as it runs, until you stop it with badgr down <id>.
  • Pass --max-cost to bound spend. Badgr shows the rate for the GPU it picked before it starts. This page does not quote a fixed price.

See pricing for how billing works.

Proof

No tested run is recorded for this integration yet. A receipt and example output will be added here after a real run, not before.

Troubleshooting

  • Models downloaded again each start

    Cloud files are removed when a service stops. Bake models into the image or accept the download time.

  • No GPU used

    Use the GPU-enabled image tag that matches the CUDA version on the machine.

  • Service never becomes ready

    Read the logs with badgr logs <id>. The most common causes are a wrong port, a missing model file or too little GPU memory.

Developer reference

Ready to start with LocalAI?

Every run or service has a spending limit you set before it starts.