Qwen 3 0.6B

✓ Verified by Badgr · vLLM · Run

Last verified 2026-10-09

badgr run --image vllm/vllm-openai:v0.28.0 --gpu RTX_4090 --max-cost 1 --max-runtime 20 --cmd 'bash -c "vllm serve Qwen/Qwen3-0.6B --port 8001 --gpu-memory-utilization 0.35 --max-model-len 2048 & vllm serve BAAI/bge-small-en-v1.5 --port 8002 --gpu-memory-utilization 0.25 --max-model-len 512 & ..."'

GPU

NVIDIA GeForce RTX 4090Reported by the provider, not confirmed inside the container

Command runtime

283s

Deployment

dep-28df6b2b97

• ✓ Two vLLM servers started on one GPU
• ✓ A chat completion returned from Qwen/Qwen3-0.6B
• ✓ An embeddings request returned from BAAI/bge-small-en-v1.5
• ✓ Command exited with code 0
• ✓ Teardown completed

Two vLLM servers shared one GPU, each with its own --gpu-memory-utilization (0.35 and 0.25). Both answered, and per-process GPU memory was about 3.3 GB and 0.4 GB. The command above is abridged: the readiness waits and the curl probes are omitted. This was a single short run, not a load test.

GitHub issue this run was for: vllm-project/vllm#60599

deployment=dep-28df6b2b97 status=succeeded user_command_exit=0 command_runtime=283s

Run from Badgr's local development environment. Documents that this job works end-to-end for this model.

✓ Verified by Badgr · vLLM · Run

Last verified 2026-10-08

badgr run --image vllm/vllm-openai:v0.28.0 --gpu RTX_4090 --max-cost 1 --max-runtime 20 --cmd 'python3 -c "
from vllm import LLM, SamplingParams
llm = LLM(model=\"Qwen/Qwen3-0.6B\")
print(\"SMOKE 60349-valid\", llm.generate([\"Hello, my name is\"], SamplingParams(max_tokens=8)))
"'

GPU

NVIDIA GeForce RTX 4090Reported by the provider, not confirmed inside the container

Command runtime

174s

Deployment

dep-59b7293792

• ✓ Workload container started
• ✓ Command exited with code 0
• ✓ Job finished (status = succeeded)
• ✓ Teardown completed

GitHub issue this run was for: vllm-project/vllm#60349

deployment=dep-59b7293792 status=succeeded user_command_exit=0 command_runtime=174s

Run from Badgr's local development environment. Documents that this job works end-to-end for this model.

✓ Verified by Badgr · vLLM · Run

Last verified 2026-10-08

badgr run --image vllm/vllm-openai:v0.28.0 --gpu RTX_4090 --max-cost 1 --max-runtime 20 --cmd 'python3 -c "
from vllm import LLM, SamplingParams
llm = LLM(model=\"Qwen/Qwen3-0.6B\", kv_cache_dtype=\"fp8\")
print(\"SMOKE 60350-control\", llm.generate([\"Hello, my name is\"], SamplingParams(max_tokens=8)))
"'

GPU

NVIDIA GeForce RTX 4090Reported by the provider, not confirmed inside the container

Command runtime

90s

Deployment

dep-59c26861cf

• ✓ Workload container started
• ✓ Command exited with code 0
• ✓ Job finished (status = succeeded)
• ✓ Teardown completed

FP8 KV cache with default memory settings loaded and generated on a 24 GB card; the startup out-of-memory in vllm-project/vllm#60350 did not occur here.

GitHub issue this run was for: vllm-project/vllm#60350

deployment=dep-59c26861cf status=succeeded user_command_exit=0 command_runtime=90s

Run from Badgr's local development environment. Documents that this job works end-to-end for this model.

✓ Verified by Badgr · vLLM · Run

Last verified 2026-10-08

badgr run --image vllm/vllm-openai:nightly --gpu RTX_5090 --max-cost 1 --max-runtime 20 --cmd 'VLLM_USE_FLASHINFER_SAMPLER=0 python3 -c "
from vllm import LLM, SamplingParams
llm = LLM(\"Qwen/Qwen3-0.6B\", kv_cache_dtype=\"fp8\", max_model_len=2048, attention_backend=\"TRITON_ATTN\")
print(\"SMOKE 60262-workaround\", llm.generate([\"Hello, my name is\"], SamplingParams(max_tokens=8)))
"'

GPU

NVIDIA GeForce RTX 5090Reported by the provider, not confirmed inside the container

Command runtime

96s

Deployment

dep-bd8a1aef14

• ✓ Workload container started
• ✓ Command exited with code 0
• ✓ Job finished (status = succeeded)
• ✓ Teardown completed

Blackwell (sm_120) with the TRITON_ATTN attention backend selected explicitly.

GitHub issue this run was for: vllm-project/vllm#60262

deployment=dep-bd8a1aef14 status=succeeded user_command_exit=0 command_runtime=96s

Run from Badgr's local development environment. Documents that this job works end-to-end for this model.

✓ Verified by Badgr · SGLang · Serve

Last verified 2026-10-09

badgr serve --image lmsysorg/sglang:nightly-dev-20261005-f70e8c68 --gpu auto --port 30000 --health-path /health --max-cost 1.5 --cmd 'python3 -m sglang.launch_server --model-path Qwen/Qwen3-0.6B --host 0.0.0.0 --port 30000 --schedule-policy fcfs'

GPU

NVIDIA GeForce RTX 3090Reported by the provider, not confirmed inside the container

Time to verified

305.1s

Deployment

dep-be772733a4

• ✓ SGLang server started
• ✓ /health returned HTTP 200
• ✓ Chat completion returned a finished reply
• ✓ final_state = VERIFIED
• ✓ Teardown completed

Use fcfs (or lpm, dfs-weight, random, lof). SGLang advertises schedule_policy=priority, but the scheduler dies on it: the same server with --schedule-policy priority failed here (sgl-project/sglang#43161).

GitHub issue this run was for: sgl-project/sglang#43161

final_state=VERIFIED failure_class=None providers_tried=1 spend_usd=0.0241 elapsed=305.1s

Run from Badgr's local development environment. Documents that this deployment path works end-to-end for this model.

✓ Verified by Badgr · SGLang · Serve

Last verified 2026-10-09

badgr serve --image lmsysorg/sglang:nightly-dev-20261005-f70e8c68 --gpu auto --port 30000 --health-path /health --max-cost 1.5 --cmd 'python3 -m sglang.launch_server --model-path Qwen/Qwen3-0.6B --host 0.0.0.0 --port 30000 --chunked-prefill-size -1 --mem-fraction-static 0.8'

GPU

NVIDIA GeForce RTX 3090Reported by the provider, not confirmed inside the container

Time to verified

358.4s

Deployment

dep-d2976dbd99

• ✓ SGLang server started
• ✓ /health returned HTTP 200
• ✓ Chat completion returned a finished reply
• ✓ final_state = VERIFIED
• ✓ Teardown completed

Turning chunked prefill off (--chunked-prefill-size -1) needs an explicit --mem-fraction-static: without it SGLang derives a negative fraction on a 24 GB GPU and the server dies (sgl-project/sglang#43160). Badgr reported that run as workload_exited with the real error: "Loaded weights leave no GPU memory for the KV cache under --mem-fraction-static=-0.03".

GitHub issue this run was for: sgl-project/sglang#43160

final_state=VERIFIED failure_class=None providers_tried=1 spend_usd=0.0284 elapsed=358.4s

Run from Badgr's local development environment. Documents that this deployment path works end-to-end for this model.

✓ Verified by Badgr · vLLM · Serve

Last verified 2026-10-10

badgr serve Qwen/Qwen3-0.6B --max-cost 1 --vllm-arg --kv-cache-dtype=auto

GPU

NVIDIA GeForce RTX 4090Reported by the provider, not confirmed inside the container

Time to verified

344s

Deployment

dep-904b9463f5

• ✓ vLLM started
• ✓ 1-token /v1/completions request returned HTTP 200
• ✓ final_state = VERIFIED
• ✓ teardown completed

Supported configuration for vllm#60859. Asking for --kv-cache-dtype=float16 on this bf16 model crashes vLLM's engine; Badgr now refuses that with HTTP 422 before renting a GPU, and this auto setting is the working alternative.

GitHub issue this run was for: vllm-project/vllm#60859

final_state=VERIFIED failure_class=None providers_tried=1/10 teardown_confirmed=False spend_usd=0.0393

Run from Badgr's local development environment. Documents that this deployment path works end-to-end for this model.

✓ Verified by Badgr · vLLM · Serve

Last verified 2026-10-10

badgr serve Qwen/Qwen3-0.6B --max-cost 1 --vllm-arg --speculative-config={"method":"ngram","num_speculative_tokens":4,"prompt_lookup_max":5,"prompt_lookup_min":2,"num_speculative_tokens_per_batch_size":[[1,16,4],[17,256,0]]}

GPU

NVIDIA GeForce RTX 4090Reported by the provider, not confirmed inside the container

Time to verified

571s

Deployment

dep-63ba97909e

• ✓ CPU n-gram speculative decoding with a per-batch-size schedule started and passed Badgr's health and inference check
• ✓ 8, 32 and 64 concurrent chat requests all returned HTTP 200 (above the 16 where the same schedule on ngram_gpu crashes the engine)
• ✓ final_state = VERIFIED
• ✓ Teardown completed

Workaround for vllm#60981: method ngram_gpu with num_speculative_tokens_per_batch_size crashes the engine once the scheduler picks a different K. Badgr now refuses that combination before provisioning; method ngram with the same schedule served 64 concurrent requests without an error. The crash itself was not re-run live.

GitHub issue this run was for: vllm-project/vllm#60981

final_state=VERIFIED failure_class=None providers_tried=1/10 teardown_confirmed=False spend_usd=0.0664

Run from Badgr's local development environment. Documents that this deployment path works end-to-end for this model.