Issue evidence · Serve · ollama/ollama

Default n_threads ignores cgroup CPU quota and cpuset: ~45x throughput collapse in CPU-limited containers

GitHub issue: https://github.com/ollama/ollama/issues/17916

Reproduced

Ollama picks its thread count from the host core count, so a CPU-limited container is throttled and throughput collapses.

What Badgr ran

Ran ollama/ollama:0.40.3 in a Docker container limited to 2 CPUs (cgroup v2, cpu.max 200000 100000) inside a 4-CPU Colima VM on an arm64 Mac, serving qwen2.5:0.5b on CPU, and generated 120 tokens with the default thread count and with num_thread=2.

What came back

  • Default threads: 28.0 and 34.1 tok/s (two runs).
  • num_thread=2: 56.8 tok/s, about 1.7 to 2 times faster.
  • cpu.stat showed 183 of 454 scheduling periods throttled (40%).

The direction matches the report, but not the 45x: this was a smaller 0.5B model on a 4-CPU arm64 VM, not an 8-vCPU EPYC with llama3.2:3b. I did not read which thread count Ollama picked in the logs.

Checked 2026-10-15 in Badgr’s local development environment.

← All issue evidence