Assert that AI model output contains what it should, excludes what it shouldn't, and matches the shape you expect — repeatable checks you can run in CI.
LLM outputs are non-deterministic, so regressions are easy to miss — a model update can silently start refusing questions or dropping required fields. badgr-eval-check lets you write simple assertions against model output and run them on every deploy, like unit tests for your prompts.
Give it the output string to check via --output, whether plain text or a JSON response body.
Check required substrings with --must-include, banned phrases with --must-not-include, and required JSON fields (dot-path supported) via the TypeScript API.
Prints a per-assertion result and exits 1 if any check failed, so it can gate a deploy or a CI job.
npx badgr-eval-check --output "The answer is 42." --must-include "42"Requires Node.js 18+.
Check that a model's output includes required text and excludes banned phrases in a single command.
npx badgr-eval-check \
--output "The answer is 42. Confidence: high." \
--must-include "42" \
--must-not-include "I cannot"badgr-eval-check
✓ must-include "42" found in output
✓ must-include "confidence" found in output
✗ must-not-include "I cannot" found in output — model refused to answer
1 check failed.npx badgr-eval-check --output "The answer is 42." --must-include "42" — no account needed to get started.