Integrations / FastChat
FastChat on Badgr: custom setup guide
A guide for running FastChat's controller and model worker in one container. Badgr has no ready-made path for it.
A setup guide only. Badgr has not tested this path, so expect to adjust it.
Why use Badgr for FastChat
- Badgr interface
- Serve
- Where FastChat fits
- Serving application
- Why Badgr
- Persistent chat serving
- Run your FastChat controller and worker on a larger GPU than you own.
- A cost cap and a stop command bound the spend.
- Badgr has not tested this path. It is a guide for your own container, not a ready-made service.
What you can do
- Host a FastChat OpenAI-style API server for a model you chose
- Run the controller and one worker together
- Keep an existing FastChat setup and move it to a larger GPU
Quickstart: Run your own container
badgr serve --image <your-registry>/fastchat:<tag> --cmd "bash start_fastchat.sh" --max-cost 5
How it works
- 1.You give Badgr your container image and start command
- 2.Badgr runs it on a GPU
- 3.You check that it answers
- 4.You stop it when done
What you get
- An API address if your script exposes one
- Logs and a receipt
Requirements
- The Badgr CLI installed and logged in (npm install -g badgr-cli)
- A funded Badgr balance
- A container image with FastChat installed
- A start command that launches the controller, a model worker and the API server
What it costs
- Badgr has no ready-made setup for this. The command below runs your own container with the generic custom option. It has not been tested by Badgr, so treat this page as a setup guide.
- A service bills for as long as it runs, until you stop it with badgr down <id>.
- Pass --max-cost to bound spend. Badgr shows the rate for the GPU it picked before it starts. This page does not quote a fixed price.
See pricing for how billing works.
Proof
No tested run is recorded for this integration yet. A receipt and example output will be added here after a real run, not before.
Troubleshooting
Worker cannot reach the controller
Start the controller first and wait for it to listen before starting the worker, all in the same container.
API server answers but no models
The worker has not registered yet. Check the worker logs for model load errors.
Service never becomes ready
Read the logs with badgr logs <id>. The most common causes are a wrong port, a missing model file or too little GPU memory.
Developer reference
Ready to start with FastChat?
Every run or service has a spending limit you set before it starts.