Christopher Owen's spark-ds41f-1120 on four Sparks: prompts read 12–15% faster, short replies wait 3× longer — and it's what we serve now
Christopher Owen tagged spark-ds41f-1120 this morning: his candidate 62, the release after the candidate 60 our four-Spark lane ran from November 16. We built it from his tag and ran it on four DGX Sparks four ways, next to our best recipe (r7) on the other four, through the same battery, the same morning. The result is a clean trade with one knob behind it, and a decision: the guest lane now runs 1120 exactly as he ships it.
In plain English
- What we compared. Owen's new release of his DGX Spark engine for DeepSeek-V4.1-Flash, against the recipe we published four days ago (his candidate 60 with our own scheduling settings on top). Four Sparks each, same model, same questions, same morning.
- His release, as he ships it, reads prompts faster. A 131K-token prompt at 9,697 tokens per second instead of 8,646 (+12.2%); a 1M-token prompt at 8,935 instead of 7,765 (+15.1%). Three codes hidden in a 1,000,000-token prompt are found in 112.8 s instead of 132.4 s. One user writes 3.1% faster (126.43 vs 122.65 tok/s). It keeps 1.55× as much context warm (18.7 million tokens vs 12.0 million).
- It makes short replies wait. A short chat that arrives while someone's 413K-token prompt is being read gets its first token in 2.247 s instead of 0.684 s (3.3× slower). A question about a document already in cache waits 3.103 s instead of 1.884 s (1.65×). Sixteen users writing together get 3.8% less in total.
- It is one knob. His settings read a long prompt in 16,384-token steps; ours use 8,192-token steps with the long prompt capped at 3,856 per step. Put our settings on his engine, his cache size kept, and the short chat drops to 0.749 s, prompt reading drops to 8,642 tok/s, the cached-document turn improves to 2.159 s, and single-user speed does not move. The cache size made no speed difference.
- Same answers. Our 75-question check scores 71 on every four-Spark arm, with the same four misses. Greedy replies to 40 fixed prompts came out identical twice and again under load.
- What we run now. James chose his settings for the guest lane on Sparks 5–8, knowing the trade: a short chat behind a long prompt will feel slower, and the guest page says so. Sparks 1–4 are free for the next test.
- Bottom line: on his own engine, our settings and his are two points on one curve. He picked prompt reading; we had picked short-request latency; for guests, James picked his.
What changed in 1120
1120 is Owen's candidate 62 tagged as a release: his fork of vLLM (local-inference-lab/vllm) with a 702-patch series through 0829, TileLang kernels for attention, linear and MoE, sparknet RoCE collectives, DSpark speculative decoding with five drafts, CUDA 13.5.37. He publishes source, not images; we built ours on one Spark in about 20 minutes and confirmed the vLLM tree hash matched his label (8fc99ca7). The site-only changes are our node map, model path, port, two NCCL flags and thinking off by default.
His release notes, his bench, his testbed (quoted as his, not ours): tp4-switched single-stream 88.97 tok/s (+9.39% over 1107), prefill 9,569 to 9,308 tok/s; tp8-switched single-stream 129.43 (+11.74%), prefill 11,846 / 12,429 / 12,383 / 12,206 / 11,806 tok/s from 32K to 1M. His release tweet adds 32 sessions at once on four to eight Sparks, a 1M prompt that prefills 56% faster on three and four Sparks, 17.9 full 1M contexts kept warm on four Sparks (was 11.0), and 12,023 tests passed, 0 failed. His prefill figures and ours agree in size; his decode figures are lower than ours because the instruments differ (ours is a thinking-off prompt set with speculation-friendly categories like code and format). Different benches, different numbers; we compare only within our own table.
The change that matters most to us is not a number. 1120 contains the two fixes we had to mount as overlays on candidate 60: 0823, the structured-output bitmask sized for copy-draft verify rows, written by Zee-B33 (#1194, our PR #1227), and 0824, keeping DSML closing tags inside tool arguments, by James Meadlock (PR #1228, idea credit Hector Isaza / ecohash-co). No overlays this time. The image is his tree, unchanged.
Results: four arms on four Sparks
r7 is our published four-Spark recipe: Owen's candidate 60 with our scheduling shape (32 seats, 8,192-token batches, long-prefill threshold 3,856, late admission at layer 20, prefill yield short, 10.5 GiB KV). Today's r7 run on Sparks 1–4 reproduces the published numbers from Sparks 5–8 (122.65 vs 123.18 tok/s single-stream; 8,646 vs 8,726 tok/s at 131K); the two pods have agreed within about 1% before.
| arm | nodes | scheduling shape |
|---|---|---|
| r7 (our best recipe) | Sparks 1–4 | candidate 60 · 8,192-token batches, long-prefill threshold 3,856, late admission layer 20, 10.5 GiB KV |
| 1120 TP4 as shipped | Sparks 5–8 | Owen's tp4-switched · 16,384-token batches, threshold 16,192, late admission layer 30, 16.5 GiB KV |
| 1120 TP4 + our shape | Sparks 5–8 | r7's shape on the 1120 engine (8,192 / 3,856 / layer 20 / 10.5 GiB) |
| 1120 TP4 + our shape, his KV | Sparks 1–4 | as above, 16.5 GiB KV |
Figure 1. Four arms, four panels, each with its own scale. Bars are sized from the measured numbers.
| row | r7 | 1120 shipped | 1120 + our shape | + our shape, 16.5 GiB KV |
|---|---|---|---|---|
| One stream decode (tok/s) | 122.65 | 126.43 | 126.78 | 126.60 |
| 16 streams, aggregate (tok/s) | 557.04 | 535.76 | 551.62 | 539.33 |
| Read a 32K / 131K prompt (tok/s) | 8,457.5 / 8,645.5 | 9,426.6 / 9,697.4 | 8,409.5 / 8,601.3 | 8,405.1 / 8,642.1 |
| Read a 300K / 1M prompt (tok/s) | 8,457.5 / 7,765.4 | 9,581.4 / 8,934.7 | 8,456.8 / 7,867.3 | 8,468.3 / 7,912.4 |
| 1M needle | 3/3, 132.4 s | 3/3, 112.8 s | 3/3, 131.1 s | 3/3, 130.2 s |
| Short chat behind a cold 413K prompt (s) | 0.684 | 2.247 | 0.725 (6 reps; 3-rep battery 0.969) | 0.749 |
| Cached 68K turn behind a 413K prompt (s) | 1.884 | 3.103 | 2.074 | 2.159 |
| Agent: read 200K once (s) | 24.0 | 20.9 | 24.2 | 24.1 |
| Agent: 3 follow-up turns, TTFT (s) | 0.514 / 0.436 / 0.490 | 0.469 / 0.440 / 0.471 | 0.516 / 0.437 / 0.494 | 0.521 / 0.437 / 0.484 |
| qeval /75 | 71 | 71 | 71 | 71 |
| KV pool (tokens) | 12,034,343 | 18,702,259 | 12,034,343 | 19,188,708 |
| = full 1M contexts | 11.48× | 17.84× | 11.48× | 18.30× |
Against r7, 1120 as shipped: single-stream +3.1%, sixteen-stream −3.8%, prompt reading +11.5% / +12.2% / +13.3% / +15.1% at 32K / 131K / 300K / 1M, short chat 3.29× slower, cached turn 1.65× slower, 1.55× the KV. 1120 in our shape: single-stream +3.4%, sixteen-stream −1.0%, 131K prefill −0.5%; r7 within noise, plus a decode gain under the +5% we require before calling something a real gain. Per category the single-stream picture is uniform: shipped 1120 writes code at 166.6 tok/s to r7's 162.7, narrative at 76.4 to 71.9, summaries at 100.8 to 93.3.
Why the trade happens
Figure 2. The mechanism. Widths are proportional to step size, not measured time; step counts are illustrative. The timings are the measured medians.
Owen's TP4 shape reads a long prompt in big pieces: up to 16,384 tokens per step, with a per-request long-prefill threshold of 16,192, and late admission at layer 30. Ours caps the long prompt at 3,856 tokens per step inside an 8,192-token step and admits at layer 20. A short request that arrives while a step is running waits for that step to finish, so bigger steps mean a longer wait, and fewer, larger steps mean faster prompt reading. Late admission is a feature of his scheduler that lets the next step's setup begin while the current step is still running; the layer number sets how early. On an earlier build, layer 20 cut the mixed-traffic wait by 37% for −0.3% prefill, and layer 30 cost 2.9% prefill (earlier post).
The knob that moves it
The fourth arm isolates it. Same engine (1120), his 16.5 GiB KV kept on both sides; the only change is the scheduling. (The two sides ran on different pods, Sparks 5–8 and Sparks 1–4, which have agreed within about 1% in earlier campaigns.)
Figure 3. Before and after on 1120 TP4 with his KV. Batch size and late-admission layer were changed together and are not separated; we do not claim which carries the effect.
Short chat behind a 413K prompt: 2.247 → 0.749 s. Prompt reading at 131K: 9,697 → 8,642 tok/s. Single-stream decode: 126.43 → 126.60, unchanged. The two 10.5 GiB and 16.5 GiB rows in our shape are within noise of each other on every line, so the KV size is not part of the speed story; the larger pool is pure capacity (12,034,343 → 19,188,708 tokens). The prefill gain and the short-turn loss come from the same setting.
What we run now and why
Figure 4. Where each engine runs after today, verified at 10:04 CST.
Sparks 5–8 serve 1120 TP4 exactly as Owen ships it, on the standard DeepSeek checkpoint, as deepseek-v4.1-flash-spark-vllm on the lab guest gateway. James chose his shape for the guest lane knowing the trade: guests who send a short chat while a long prompt is being read will wait around two seconds instead of under one, and the guest page says so; in return, prompts read 12–15% faster, the 1M needle finishes in 112.8 s instead of 132.4 s, and 17.84 full 1M contexts stay warm instead of 11.48. Restart to ready is 131 s (first boot of the shape was 485 s while kernels compiled). Rollback is our published r7 recipe.
Sparks 1–4 are a free test pod. Retired today: our SGLang lane on Sparks 1–4 and the abliterated guest build on 5–8. TP8 across all eight, the campaign's best Spark result, is not running and comes back in about 2.5 minutes from kept containers; it is the subject of the companion post.
Gates
1120 TP4 as shipped, the lane now serving: all functional gates pass. Greedy output 40/40 identical twice on one boot and 40/40 identical with 15 other streams running. 64 concurrent JSON and json_schema requests all valid, server alive after. The DSML tool-call probe, with tool arguments that contain DeepSeek's own closing tags, passes streaming and not. A 32×32 red/blue PNG answered "red and blue, with red on the left". All seven reasoning_effort values return 200. Quality: qeval 71/75 on every four-Spark arm, the same four misses on each (code_interval_intersect, json_count, json_escape, math_m9), greedy and deterministic.
Method
Every arm ran the same harness on November 20, 2026, between 07:21 and 09:26 CST: v41bench prompt set v1, eight categories, temperature 0, thinking off, three repetitions, tokens from the server usage block. One stream = per-stream decode tok/s, median of 3. Sixteen streams = aggregate tok/s. Prefill = cold prompt with a unique nonce so nothing hits the cache, median of 3, at 32K / 131K / 300K / 1M. 1M needle = three codes in a 1,000,000-token prompt, found and seconds. Short chat behind a long prompt = start a cold 413,000-token prompt, send a ~18-token turn while it is read, time to first token (median of 3; 6 for the 1120 + our shape arm, as noted). Cached turn = a 68,000-token document already in cache plus a fresh 1,500-token question, sent during the 413K read. Agent turn = read a 200,000-token context once, then three follow-ups of 1,500 tokens each. qeval = our 75-task set, passed count.
Hardware: four DGX Sparks per pod, about 123 GiB each, stock 4 KiB-page kernel, driver 580.178.04, one 200G port each on a MikroTik CRS804-4DDQ, MTU 9000; a TP4 pod stays on one switch. Owen qualifies his shapes on a 64 KiB-page kernel with Memory Saver; his shipped sizes still fit on ours (18.6 GiB free on the head node while serving his TP4 shape). Model: DeepSeek-V4.1-Flash-1017, HF revision 1bae0e10, identical files on all eight Sparks.
Not measured
- Power or watts per token. Cost.
- Which of batch size and late-admission layer carries the scheduling effect; they moved together.
- Whether the reported KV pools can be filled in use.
- Teacher-forced logprob equality between candidate 60 and 1120. Any abliterated weights. The Hermes gauntlet. Concurrency beyond sixteen.
Credits
Christopher Owen: spark-ds41f, every kernel, collective and scheduler setting on both sides of this comparison (repo, release). His published numbers are his. Zee-B33: the 0823 structured-output bitmask fix (#1194, PR #1227). James Meadlock: the 0824 DSML-closer fix (PR #1228), idea credit Hector Isaza / ecohash-co. DeepSeek: the model. Local Inference Lab, TileLang, vLLM and NVIDIA for what the engine builds on.