AI API receipt · BerriAI/litellm

Responses API mid-stream 429 can be recorded against the fallback deployment and cool it down

BerriAI/litellm#45645 · upstream bug reproduced by running the issue's own reproduction

✓ Badgr rejects this response

After two streaming 429s on the primary, a direct request to the healthy "backup" group answered HTTP 429 "No deployments available for selected model" although backup never returned a 429 (reproduced on a live LiteLLM 1.104.2 proxy).

Outcome contract verdict: response is empty. The attempt is recorded as failed (invalid_outcome) instead of a verified success, so it is not billed as a good answer and can fall back to another route.

What the upstream returned (replayed)

The output the bug produces, taken from the issue’s reproduction and fed to Badgr’s validators. Not a live provider call.

text: none
tool_calls: none

Badgr receipt

Written by Badgr’s receipt ledger and read back from GET /v1/receipts/replay-litellm-45645-fallback-cooled-by-primary-429.

{
  "request_id": "replay-litellm-45645-fallback-cooled-by-primary-429",
  "status": "failed",
  "endpoint": "/v1/responses",
  "stream": false,
  "failure_stage": "outcome_contract",
  "error_code": "invalid_outcome",
  "error_message": "Response did not satisfy the outcome contract: response is empty",
  "retryable": true,
  "provider_mode": "replay",
  "routing_decision": "replayed_upstream_output",
  "created_at": "1791652209.3107882"
}

Recorded 2026-10-11 in Badgr’s local development environment. Replay cases and verdicts are pinned by backend/test_ai_api_replay_cases.py.

← All AI API receipts