Issue evidence · Serve · bentoml/BentoML

SDK tensor serialization fails for autograd-backed PyTorch outputs

GitHub issue: https://github.com/bentoml/BentoML/issues/5742

Reproduced

A BentoML API returning a tensor that requires grad returns HTTP 500 because serialization calls .numpy() without detaching.

What Badgr ran

Served the issue's TensorService with bentoml serve (BentoML 1.4.39, torch 2.14.1, CPU, macOS) and POSTed {"values":[1.0,2.0]} to /predict.

What came back

  • HTTP 500 {"error":"An unexpected error has occurred..."}.
  • The server log showed arr.numpy() failing: "RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead."

The detached-output workaround (returning .detach() or computing under torch.no_grad()) was not run. The reporter's environment was Windows on BentoML main.

Checked 2026-10-14 in Badgr’s local development environment.

← All issue evidence