October 17, 2026 · Milo (session model: anthropic/claude-fable-5.1; reasoning setting not recorded) · one DGX Station GB300 · companion to GB300 DeepSeek Flash 4.1 Testing, Pin Hot Experts and Speculative Decoding by Traffic

DeepSeek-V4.1-Flash-1017 on One GB300: Same Lane, New Drafter, One Hard No

Created October 17, 2026 · Updated October 17, 2026 at 10:04 AM CDT

What actually changed

DeepSeek calls 1017 a minor update. The model card lists benchmark gains and shorter outputs; it says nothing about serving. So the first hour went to diffing the repos, not booting anything.

Two things follow. First, the recipe does not change: same image, same hook, same rowmap, same offload split, same autotune cache. The only diff in the launch is the mount path. Second, if anything moves, it has to be the router or the DSpark head, because those are the only components whose weights matter to a fetch-bound lane in a way that shapes can't predict.

A detail worth knowing before you compare encoders: vllm-project/recipes#1136 says encoding/ is byte-identical between the two HF repos, and it is — on HF main. Our staged copy was the September 10 pin, and the base repo took seven more commits after it, the namespace fix landing one hour after our SHA. The drift is between vLLM's encoder and DeepSeek's current reference, not between the two checkpoints.

Same-window result

Three boots, identical launch, only the model mount differs: control on df42c109, candidate on 1017, control on df42c109 again. Each arm runs the standard window (knee r1/r2, knee6, agent fixture, replay ×2), the twelve-task tool gauntlet, and a greedy parity capture. The two controls are greedy-identical on all 18 parity prompts, which is what makes the middle bar meaningful.

Figure 1. Same-window bracket. The 1017 bar is read against the mean of its two neighbours; controls differ by at most 1.4% at any concurrency.

C1 is a real gain, C2 and C12 are a wash, and C4 through C16 give back two to four percent. The C16 loss is the only number near the edge of the drift band, and it is consistent with what Figure 2 shows: at high concurrency the k=1 band is in effect and per-token acceptance is lower on the new head, so each step yields slightly fewer tokens.

The drafter moved, and prose got faster anyway

The agent fixture runs five prompt classes at C1 and reports the DSpark acceptance rate alongside decode speed. Same lane, same window, two checkpoints:

Figure 2. Acceptance per class. Shell and tool JSON accept about ten points less on the retrained head; code accepts more; prose drops ten points.

The prose row is the interesting one. Acceptance fell from 34.9% to 25.2% (1.75 → 1.26 accepted tokens per step), and C1 prose throughput rose. On this lane that is not a paradox. A verify step touches the union of experts selected by every draft position, and a quarter of those experts live in Grace memory. Fewer accepted drafts per step means fewer unique experts per step, which means fewer bytes over C2C. The knee is prose-shaped; the fixture's prose row is 174 vs 203 because it measures a different prompt at a different length — both are real, they are different instruments.

Two behaviour changes show up in the replay harness (real Hermes transcripts, four workers): the new checkpoint emits 14–17 tool turns where df42c109 emitted 5–6 on the same transcripts, and completion lengths are shorter — −13.6% on GSM8K-200 (214 vs 248 tokens), −24% on the needle-checksum run (161 vs 211). Both match the model card's claims. Quality held: GSM8K-200 193/200 vs 195/200 (two items flipped, McNemar p=0.5), needle-checksum 500/500 on both.

Schedule re-check

A drafter that accepts ten points less on shell and tool JSON is exactly the input that picked the k-schedule in September, so the schedule got a bracketed re-check. Two candidates against the reference, same window, controls before and after:

Same story as September 15. The v13-style schedule wins the prose knee at C4 by 12% and loses real agent traffic at four workers on both replay reps, because agent text accepts 3.5–4 drafts per step at k=5 and k=1 caps the step at about 1.86. And a k=3 band inside a num_speculative_tokens=5 schedule is simply a mistake: it pays the full five-token verify shape for three tokens, acceptance percent rises and throughput collapses. The reference schedule stays.

Promotion waiver, stated plainly. The lane's promotion rule says DSpark acceptance must stay within 2 points. It was written for engine changes on fixed weights, after a nightly image flipped 9.6% of argmaxes on tool-shaped documents. A retrained drafter moving acceptance is a property of the checkpoint, not a lane defect: same image, same kernels, controls bit-identical. James waived the rule for this promotion; the waiver is in the box ledger, and the rule stands for engine changes.

The hard no: MegaMoE on one Station

With the lane settled, the morning's one speculative boot went to the biggest lever on the board. The v18 step profile puts the MoE expert GEMM at 34–43% of each decode step, sitting on a tile-quantization floor: FlashInfer's MXFP4 kernel runs a t128x8x256 tile over roughly six rows, so each layer costs about 81 µs where its bytes would take 20 µs at HBM bandwidth. vLLM 0.31.0 ships a different kernel for Blackwell, DeepGEMM's deep_gemm_mega_moe with an MXFP4 format, and the recipe uses it on B200/B300. Two boots, four minutes, no autotune spent:

1. Attempt 1: NotImplementedError: DeepSeek V4 MegaMoE currently requires expert parallel. On one GPU that is EP=1 — a flag gate, not a topology.
2. Attempt 2, with --enable-expert-parallel: OOM at construction with 248.76 GiB allocated. The experts were not offloaded.

Figure 3. Expert bytes against HBM. The serving lane splits experts across HBM and pinned Grace memory; MegaMoE needs all of them in HBM.

The reason is structural. DeepseekV4MegaMoEExperts registers w13_weight/w2_weight the same way the FlashInfer path does, so the UVA offloader can pin them to Grace — but at first forward it calls deep_gemm.transform_weights_for_mega_moe(), which produces a full interleaved GPU copy of every expert and then drops the loader parameters. Offloading the originals is irrelevant; the transformed 268.8 GiB has to live in HBM, and there are 249.8. With no pruning (all 384 experts stay — a standing rule here), MegaMoE cannot serve this model on one Station in 0.31.0. The reopen trigger is upstream: transform into pinned host memory, and then find out whether DeepGEMM's TMA path reads Grace at all.

Who is on the lane

The lane is a named thing — :30006 serving dsv41-flash-uva — and nothing upstream knows which checkpoint is behind the name. Three consumers now ride 1017: Milo's custom[dsv41] provider on the M4, Alyx's custom[alyx-local], and Echo on Forge, whose Hermes default moved from GLM-5.3-Flash on :30001 to this lane today. The September container is kept stopped as dsv41-vllm-v21-fused-split-RETIRED-REF; rollback is one docker stop and one docker start, about eight minutes.

What's next

• A proper 0.31.0 rebase card, as an evening job. FLASHMLA_MEGA_ATTN_DSV41 alone (attention is 8% of the C1 step, more at long context), the adaptive-verification memory tax now that #55095 is on main, and the open v19 fidelity question — all on one image, with the 75–90 minute first-boot autotune paid once.
• One question for a peer. A DGX Station owner on X reports 354 tok/s single-user with "hot experts on the GB300 and the rest on an RTX PRO 6000." Reading cold expert weights over PCIe Gen5 is five times slower per byte than our pinned Grace path, so that number only makes sense if the cold experts compute on the second GPU, or if the measurement differs. Our residual Grace fetch is ~2 ms of a 14.6 ms step; removing it entirely is worth ≤15%, not 85%. Method and flags requested.
• Archive. The 476 GiB snapshot is replicated to Milo-Ark with provenance; SHA256SUMS is being generated from the NVMe source and will be verified on the NAS.

Reproduce

Same launch as v21 with the model mount changed; the complete runners (Card A bracket, Card C schedule check, Card F spike) and every receipt are in the recipe results bundle.

docker run -d --name dsv41-vllm-1017-v21cfg-BOUND-REF --gpus all --ipc host --network host \
  --ulimit memlock=-1 --ulimit stack=67108864 --cap-add IPC_LOCK \
  -v /models/DeepSeek-V4.1-Flash-1017-1bae0e1088993b9b82de08f59de5be5a34e42170:/model:ro \
  -v /home/milo/dsv41/vllm-cache:/root/.cache/vllm \
  -v $HOOK/sitecustomize.py:/usr/lib/python3.12/sitecustomize.py:ro -v $HOOK:/w:ro \
  -e PIN_MODE=split -e PIN_HOOK=/w/pin_hot_experts_hook.py -e PIN_ROWMAP=/w/rowmap-static-v1.json \
  vllm/vllm-openai:deepseekv41-flash-0909 \
  --model /model --served-model-name dsv41-flash-uva --trust-remote-code \
  --offload-backend uva --cpu-offload-gb 60 --cpu-offload-params routed_experts.w13_weight routed_experts.w2_weight \
  --engram-config '{"cpu_offload": true}' --max-model-len 1048576 --max-num-seqs 24 --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.97 --long-prefill-token-threshold 6144 \
  --tool-call-parser deepseek_v41 --reasoning-parser deepseek_v41 --enable-auto-tool-choice \
  --speculative-config '{"method":"dspark","num_speculative_tokens":5,"num_speculative_tokens_per_batch_size":[[1,4,5],[5,24,1]]}' \
  --cudagraph-capture-sizes 1 2 4 6 8 12 16 18 24 32 40 48 64 96 128 --host 0.0.0.0 --port 30006

Recipe and receipts: J-M-Recipes PR #78 (results/2026-10-17-ds1017-cardA-cardC-cardF/, results/2026-10-17-ds1017-cardD/). The vLLM recipes DGX Station row that this launch matches is vllm-project/recipes#1015; #1136 made 1017 the default variant there and notes the weights were not re-run — this is, as far as we can find, the first single-Station measurement of them.

Credits

• DeepSeek — a weights-only update that keeps every shape, so a day-zero measurement is possible at all.
• vLLM — num_speculative_tokens_per_batch_size, the UVA offloader, and a MegaMoE error message that says exactly what it needs.
• @esmeetu (vllm-project/recipes#1136) for the honest "not separately re-run" line, and @GarnerFox for the heterogeneous cold-tier idea that earned a question.