Issue evidence · Serve · ggml-org/llama.cpp

Misc. bug: default thread count ignores cgroup v2 CPU limits in containers

GitHub issue: https://github.com/ggml-org/llama.cpp/issues/30070

Confirmed from source (not run)

The default thread count comes from host CPU topology, so a container with a CPU quota runs too many threads and gets throttled.

What Badgr ran

Read common/common.cpp on llama.cpp master (commit efe3626b1, 2026-10-12). Not run: a CPU-quota container was not available, so the throttling and tok/s figures were not measured.

What came back

  • common_cpu_get_num_math() uses sysconf(_SC_NPROCESSORS_ONLN), a hybrid-CPU affinity path, or common_cpu_get_num_physical_cores().
  • There is no read of cgroup cpu.max or CPU quota anywhere in common.cpp (no cgroup or cpu.max string).

Code reading (not run): it confirms no quota is consulted. The reporter's 3.1x and 4.8x differences are their measurements. A related measurement with Ollama (llama.cpp-based) under --cpus=2 is on the ollama-17916 page.

Checked 2026-10-15 in Badgr’s local development environment.

← All issue evidence