Issue evidence · Serve · ggml-org/llama.cpp

CUDA illegal memory access with flash attention on Windows releases (Gemma 4 12B QAT Q4_0 + mmproj)

GitHub issue: https://github.com/ggml-org/llama.cpp/issues/30436

Not reproduced

On Windows releases, llama-server with -fa on dies with a CUDA illegal memory access on requests with an image plus about 4.5k tokens of text; -fa off works.

What Badgr ran

Ran ghcr.io/ggml-org/llama.cpp:server-cuda on an RTX 4090 through Badgr (Linux) with google/gemma-4-12B-it-qat-q4_0-gguf and its mmproj, -fa on -ngl 999 -c 32768 -np 1 --no-context-shift, and sent a 1024x1024 PNG with about 3.5k and then about 4.6k tokens of text, 3 requests each.

What came back

  • All 6 requests returned HTTP 200 (3.5k and 4.6k prompt tokens), including the second and third requests on the same slot.

This matches the reporter's own note that Linux Docker works. The Windows builds were not tested. A flash-attention-off comparison pod never became healthy and is not reported.

Checked 2026-10-14 in Badgr’s local development environment.

← All issue evidence