AI API receipt · vllm-project/vllm

tool_choice="required" with the hermes structural tag drops parallel tool calls

vllm-project/vllm#60769 · upstream bug reproduced by running the issue's own reproduction

Badgr cannot tell this response is wrong

The prompt needed get_weather for Paris and Tokyo; only the Paris call came back, with finish_reason "tool_calls".

one complete, schema-valid call satisfies a required tool choice; Badgr cannot know the prompt asked for two. The receipt therefore records success. This is a limit of what the contract can prove, shown here so it is not mistaken for protection.

What the upstream returned (replayed)

The output the bug produces, taken from the issue’s reproduction and fed to Badgr’s validators. Not a live provider call.

tool_calls: [{"type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"}}]

Badgr receipt

Written by Badgr’s receipt ledger and read back from GET /v1/receipts/replay-vllm-60769-required-drops-parallel-tool-calls.

{
  "request_id": "replay-vllm-60769-required-drops-parallel-tool-calls",
  "status": "success",
  "endpoint": "/v1/chat/completions",
  "stream": false,
  "failure_stage": null,
  "error_code": null,
  "error_message": null,
  "retryable": null,
  "provider_mode": "replay",
  "routing_decision": "replayed_upstream_output",
  "created_at": "1791540569.127625"
}

Recorded 2026-10-09 in Badgr’s local development environment. Replay cases and verdicts are pinned by backend/test_ai_api_replay_cases.py.

← All AI API receipts