Issue evidence · Serve · ollama/ollama

CUDA illegal memory access (MUL_MAT) on RTX 5090 with Cohere MoE architecture (Windows)

GitHub issue: https://github.com/ollama/ollama/issues/18642

Reproduced, workaround proven

Ollama 0.32.1 runs the Cohere MoE model north-mini-code-1.0 on an RTX 5090, but 0.32.3 and later crash with "CUDA error: an illegal memory access was encountered" in MUL_MAT.

What Badgr ran

Ran the official ollama/ollama images on RTX 5090 pods through Badgr (Linux, Docker), pulled north-mini-code-1.0:latest (cohere2moe, 30.5B, Q4_K_M, 18.6 GB), built the reporter's xtest Modelfile (num_ctx 8192, temperature 0.3, their system prompt), and sent three requests per pod: "test", a long Turkish prompt of about 2,900 tokens, and "test" again. Each of 0.32.1 and 0.32.3 was run twice on a 5090; 0.34.3 and 0.40.3 once. 0.32.1 and 0.32.3 were also run once each on an RTX 4090.

What came back

  • RTX 5090, 0.32.1: HTTP 200 on all three requests in both runs.
  • RTX 5090, 0.32.3: the first request returned HTTP 500 "CUDA error: an illegal memory access was encountered" in both runs; the runner then reloaded and the next two requests returned 200.
  • RTX 5090, 0.34.3: same first-request failure, then 200 on the next two.
  • RTX 5090, 0.40.3 (current): the same error; /api/ps showed the model loaded as family cohere2moe on the llamacpp runner.
  • RTX 4090, 0.32.1 and 0.32.3: HTTP 200 on all three requests, so the failure follows the 5090 and the version together.

Workaround verified on this setup: pin the image to 0.32.1 with badgr serve --image ollama/ollama:0.32.1 --gpu RTX_5090 --port 8000 --health-path /api/version --cmd "bash -c 'OLLAMA_HOST=0.0.0.0:8000 exec ollama serve'", then pull the model; the first request after load took about 85 s. The reporter runs Windows, this was Linux in Docker, and 0.32.2 was not tested, so the exact release that introduced it is between 0.32.1 and 0.32.3 but unconfirmed. Replies in these runs hit the num_predict cap while still thinking, so this checks that inference completes, not answer quality.

Checked 2026-10-13 in Badgr’s local development environment.

← All issue evidence