Integrations / KTransformers

DeployExperimental

KTransformers on Badgr: compatibility guide

A guide for running KTransformers, which splits a large model between GPU and CPU memory, from a custom container.

A setup guide only. Badgr has not tested this path, so expect to adjust it.

Why use Badgr for KTransformers

Badgr interface
Serve
Where KTransformers fits
Model runtime
Why Badgr
Specialised CPU and GPU inference
  • Try a CPU and GPU memory split on rented hardware before you buy any.
  • A cost cap and a stop command bound the spend.
  • Badgr has not tested this path. It is a guide for your own container.

What you can do

  • Run a very large mixture-of-experts model with limited GPU memory
  • Test a CPU and GPU split before buying hardware
  • Serve a model that does not fit on one GPU alone

Quickstart: Run your own container

badgr serve --image <your-registry>/ktransformers:<tag> --cmd "<your start command>" --max-cost 5

How it works

  1. 1.You give Badgr your container image and start command
  2. 2.Badgr runs it on a GPU
  3. 3.You check that it answers
  4. 4.You stop it when done

What you get

  • A service address if your command exposes one
  • Logs and a receipt

Requirements

  • The Badgr CLI installed and logged in (npm install -g badgr-cli)
  • A funded Badgr balance
  • A container image with KTransformers and its start command
  • A machine with enough system memory for the CPU part of the model
  • A CPU and GPU combination KTransformers supports

What it costs

  • Badgr has no ready-made setup for this. The command below runs your own container with the generic custom option. It has not been tested by Badgr, so treat this page as a setup guide.
  • A service bills for as long as it runs, until you stop it with badgr down <id>.
  • Pass --max-cost to bound spend. Badgr shows the rate for the GPU it picked before it starts. This page does not quote a fixed price.
  • Machines with a lot of system memory can cost more per hour.

See pricing for how billing works.

Proof

No tested run is recorded for this integration yet. A receipt and example output will be added here after a real run, not before.

Troubleshooting

  • Crashes at load

    The CPU part of the model needs system memory. Ask for a machine with more of it.

  • Very slow generation

    Parts of the model on the CPU are slower. Move more to the GPU if memory allows.

  • Service never becomes ready

    Read the logs with badgr logs <id>. The most common causes are a wrong port, a missing model file or too little GPU memory.

Developer reference

Ready to start with KTransformers?

Every run or service has a spending limit you set before it starts.