Run on Badgr

Batch Inference

Run offline batch inference — scoring, evals, extraction — across large input sets with retries and a single receipt at completion.

Start batch job Get help running this

Specific Batch Inference jobs

Classify rows in a CSV

Run offline batch classification over a CSV file — label each row with a category, sentiment, or custom class using an open model, with results written back as a labeled CSV.

Extract structured JSON from unstructured text

Run offline batch extraction over a set of documents or text records, producing structured JSON output per a defined schema using an open model.

Summarize a batch of documents

Run offline batch summarization over a set of documents (reports, transcripts, articles), producing one summary per document in a single job.

Process a batch of PDFs

Run offline batch processing over a set of PDFs — text extraction, classification, summarization, or structured extraction — in a single job with one receipt.

Generate embeddings for a dataset

Run offline batch embedding generation over a set of text records — for building a vector index, search, or RAG pipeline — in a single job with one receipt.

Evaluate a prompt set against a model

Run offline batch evaluation of a prompt/test-case set against a model — scoring outputs against expected answers or a rubric — in a single job with one receipt.

Compatible GPU routes

A100 40GB A100 80GB L40S 48GB A40 48GB RTX 4090 24GB

Example command

badgr run "python batch_infer.py --input data.jsonl" \
  --gpu A100_40GB \
  --max-cost 20 \
  --max-runtime 180

Common failure modes

Job stops partway through a large input file

Chunk input into batches and checkpoint progress so a restart resumes instead of reprocessing.

Throughput lower than expected

Increase batch size to saturate GPU utilization, or move to a higher-tier GPU route.

Max-cost hit before full dataset completes

Check live route

View details →