Integrations / LocalAI
DeployExperimental
LocalAI on Badgr: custom container guide
A guide for hosting a LocalAI container behind an OpenAI-style API on a Badgr GPU.
A setup guide only. Badgr has not tested this path, so expect to adjust it.
Why use Badgr for LocalAI
- Badgr interface
- Serve
- Where LocalAI fits
- Model server
- Why Badgr
- Hosted compatible API
- Run a LocalAI container behind an OpenAI-style API on a rented GPU.
- A cost cap and a stop command bound the spend.
- Badgr has not tested this path. It is a guide for your own container.
What you can do
- An OpenAI-style API for several model types in one server
- Keep an existing LocalAI model configuration
- Move a LocalAI setup onto a larger GPU
Quickstart: Run your own container
badgr serve --image localai/localai:<gpu-tag> --cmd "<your start command>" --max-cost 5
How it works
- 1.You give Badgr your container image and start command
- 2.Badgr runs it on a GPU
- 3.You check that it answers
- 4.You stop it when done
What you get
- An OpenAI-style API address if the port is exposed
- Logs and a receipt
Requirements
- The Badgr CLI installed and logged in (npm install -g badgr-cli)
- A funded Badgr balance
- A LocalAI GPU container image
- Model configuration files included in the image or fetched at start
- Storage for model files (cloud files are removed on stop)
What it costs
- Badgr has no ready-made setup for this. The command below runs your own container with the generic custom option. It has not been tested by Badgr, so treat this page as a setup guide.
- A service bills for as long as it runs, until you stop it with badgr down <id>.
- Pass --max-cost to bound spend. Badgr shows the rate for the GPU it picked before it starts. This page does not quote a fixed price.
See pricing for how billing works.
Proof
No tested run is recorded for this integration yet. A receipt and example output will be added here after a real run, not before.
Troubleshooting
Models downloaded again each start
Cloud files are removed when a service stops. Bake models into the image or accept the download time.
No GPU used
Use the GPU-enabled image tag that matches the CUDA version on the machine.
Service never becomes ready
Read the logs with badgr logs <id>. The most common causes are a wrong port, a missing model file or too little GPU memory.
Developer reference
Ready to start with LocalAI?
Every run or service has a spending limit you set before it starts.