DSF-0731-Mia
DeepSeek-V4-Flash-0731
on two GB10 Sparks. Runtime is still
ghcr.io/anemll/dspark-vllm-gx10:0.1.1
digest sha256:a8394849….
What changed is Mia’s Aug 12–13 start-time hotfixes from
MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark
@ 018c6bc.
We did not pull into the July 29 production tree.
What shipped
Diagram is the August 13 Funland apply, not Mia’s advertised defaults. Connectors stay in the gutters.
Why we touched a working lane
Mia’s August 13 post
asked people to update the 2× Spark repo. The tweet maps to three real bugs
on the same Anemll 0.1.1 image we already run:
| Issue | Failure | Why it matters here |
|---|---|---|
| #27 | A long new prefill starves in-flight decode. vLLM advertises max_num_partial_prefills=1 and never enforces it. |
Agent mix: a fat prompt arriving while another reply is still decoding. |
| #26 | Hybrid KV lookup takes the min hit across MLA + sliding-window groups, so a legal SWA drop zeros the common prefix. | Same long prompt re-prefills. Worst at concurrent 32K+ prefixes, not C1 replay. |
| #31 | thinking:max + max_tokens mid-reason returns content: null. Stock image 400s thinking_token_budget on the V2/DSpark runner. |
We already avoid it with thinking: false. The patch is optional insurance, not a reason to turn thinking on. |
The quieter win is
#22:
nvfp4_ds_mla had been dispatched down the slow BF16 KV path.
Mia’s changelog claims ~16× decode at 600K+ after the fix. We did not re-run a 600K needle today.
How we applied it
Copy-checkout. origin/main was force-pushed over our July 29 commit, so an in-place pull would have smashed Funland work.
- New tree at Mia
018c6bc. July 29 tree left untouched as rollback. - Kept Funland knobs: util 0.85, seqs 6, batch 8192, dynamic-K C1=5 / C2–4=4 / C5–6=3, regular CUDA graphs,
thinking: false, post-readiness warmup. - Let her start script apply #21/#22/#26/#27/#31 and the six v0.27 backports, then restart once.
- Did not adopt
DEFAULT_THINKING=max, her long-coding 16k/seqs=4/0.87 profile, the Qwen3-VL sidecar, or abliterated weights.
August 13 gates
Same session as the apply. Loopback unless noted. No new long-context needle.
| Gate | Result |
|---|---|
/v1/models | deepseek-v4-flash-0731 · max_model_len=1048576 |
| Exact smoke | RESTORE_0731_OK, finish_reason=stop |
| Tools | get_weather("Pensacola") in tool_calls[] |
| Hermes | ROUTE_OK |
| Queues | 0 running · 0 waiting · 0 preemptions |
| Warmup | C4 mean K 4.01 · C6 mean K 3.03 · pass-2 JIT 0 |
Count 1→300, stream:false | 600 completion tokens / 6.47 s = 92.7 tok/s |
| #31 budget=64, thinking on | HTTP 200, content not null (481 chars). Hermes still does not send this. |
| Anemll image | Still latest 0.1.1. No platform bump to apply. |
92.7 t/s is the Tony-class peak fixture, not everyday agent feel. Typical agent / Keys C1 on this pair has been the mid-50s to ~70 band. Publish both numbers or you will lie to yourself.
August 13 bench
018c6bc process, queues idle, thinking: false, temp 0.
Two regimes. Do not mix them into one headline.
Tony-parity · stream:false · completion_tokens / wall
Two reps each. First can be colder. Mix-weighted spec accept across this block: 2260 / 2535 = 0.892.
| Probe | Best t/s | Median t/s | Tokens / wall (best) |
|---|---|---|---|
| count 1→300 | 90.88 | 90.38 | 600 tok / 6.60 s |
| count 1→150 | 88.09 | 88.04 | 300 tok / 3.41 s |
| repeat ABCDEFG×80 | 92.82 | 92.28 | 244 tok / 2.63 s |
short code is_prime | 71.69 | 68.75 | 93 tok / 1.30 s |
| structured JSON | 51.12 | 48.64 | 154 tok / 3.01 s |
Apply-time single count smoke was 92.7 t/s. The scored fixture is the 2-rep Tony board: 90.9 best / 90.4 median. Tony’s published count-class peak is ~82. This is +8.6 on that reference, not a new architecture.
Keys concurrency · server-aggregate · best of 2
House bench_concurrent.py against deepseek-v4-flash-0731. Per-stream SSE numbers under-count under spec decode; the number we keep is server-agg.
| Load | Best agg t/s | Per-stream (agg/n) | Accept | vs 0731 promote (Jul 31) |
|---|---|---|---|---|
| C1 | 68.4 | 68.4 | 0.725 | 68.9 · tie |
| C4 | 141.8 | 35.5 | 0.720 | 135.8 · +4.4% |
| C6 | 180.3 | 30.0 | 0.767 | 180.6 · tie |
After the board: 0 running / 0 waiting / 0 preemptions. This is operational route evidence, not a leaderboard.
What this is not
- Not a new checkpoint. Revision is still
9e165c30…. - Not a new vLLM image. The hotfixes mount and patch Anemll 0.1.1 at start.
- Not permission to turn thinking on for Hermes. Mia’s own #31 table:
thinking:falseis 0/6 empty-content failures. - Not a 32K×8 prefix-retention re-bench. We proved the patches applied and C1/C4/C6 warmup still hits the 5/4/3 schedule.