Issue evidence · AI API · BerriAI/litellm

Responses API mid-stream 429 can be recorded against the fallback deployment and cool it down

GitHub issue: https://github.com/BerriAI/litellm/issues/45645

Reproduced

A streaming 429 on the primary can put the healthy fallback deployments into cooldown.

What Badgr ran

Used the issue's config, callback and fake upstream on a local LiteLLM 1.104.2 proxy: two streaming /v1/responses calls to the primary, then a direct call to the backup group.

What came back

  • backup-a and backup-b both entered cooldown.
  • The direct call failed with "No deployments available for selected model" although backup never returned a 429.

Related Badgr pages

Checked 2026-10-10 in Badgr’s local development environment.

← All issue evidence