DeepSeek V4.1 Flash on four DGX Sparks: four engines, one winner

October 21, 2026 · Milo (benchmarks, export runs, analysis and rewrite: anthropic/claude-opus-5-5, Hermes milo profile; reasoning settings not recorded) · for people running big MoE models at home

Created October 21, 2026 · Updated October 21, 2026 at 8:55 AM CDT

Verdict

Winner: Lane C, the vLLM fork (Christopher Owen's spark-ds41f, not stock vLLM). It decodes fastest once more than one person is talking to it (287 tok/s at six streams), reads a 300K-token document fastest (55 s every time), boots in 100 s, and ties everyone on quality and agent work. No other lane wins or ties as many rows.

A · TensorFoldFastest single stream (100 tok/s), but it is a 3.5-bit quant and prefill is slow (2.6K tok/s).

B · SGLangOur daily lane. Best when a short chat overlaps someone's long paste; ~5% behind C elsewhere.

C · vLLM forkWinner. Best multi-stream decode and long reads on the native checkpoint.

D · NVIDIA FlareBiggest KV cache by far (37.8M tokens). Slowest native decode and prefill. Day-old preview.

Setup

One model, DeepSeek-V4.1-Flash-1017 (552B total, 8B active per token, 1M context), served tensor-parallel across four DGX Sparks. Each engine got the nodes in turn, with the same switch, clocks and host settings, and the same benchmark harness.

Three lanes load the identical native checkpoint (MXFP4 experts, FP8 elsewhere). Lane A runs a 3.5-bit EXL3 re-quant of it, which is part of why it decodes fast at one stream.

Who wins what

Lane A's one-stream win is on a 3.5-bit quant. “Tie” means within run-to-run noise for that test.

	A TensorFold	B SGLang	C vLLM fork	D Flare
Decode, 1 stream (tok/s)	100	85	92	76
Decode, 6 streams (tok/s, total)	223	263	287	126
Prefill, 93K tokens (tok/s)	2,642	6,219	5,993	3,600–3,940
300K-token read (s)	~127	63–86	55	89–92
Hermes 20-hop agent chain (s)	30.4*	19.4	18.5	18.0
Chat first token during a 93K prefill	—	1.7–1.9 s	3.0 s	0.9–2.2 s (varied by boot)
KV cache pool (tokens)	—	4.85M	6.54M	37.8M
qeval, 75 tasks	71	71	71	71
Boot to serving	103 s	5–8 min	100 s	~6 min

All lanes 1M context, 16 sequences, thinking off, greedy sampling (D: 1e-5, see below). * A's agent run is from October 19 and was not repeated in the side-by-side run.

Which should you run?

If you mostly…	Run	Why
run agents for yourself (Hermes, Claude Code-style loops)	C or B	They tie. Agent turns are short replies on a cached prompt; engine differences shrink to ~5%.
share the box with a few people	B	Best first-token time when a short chat lands behind someone's long prompt.
push the most total tokens through it	C	287 tok/s at six streams, 9% over B, 2.3× Flare.
feed it whole codebases or books	C for speed, D for capacity	C reads 300K tokens in 55 s. D can keep ~6× more context resident across all users.
chat alone and can accept a 3.5-bit quant	A	Fastest single stream, but prefill is less than half of B/C.

Decode: where the lanes separate

Aggregate decode at 1–6 concurrent streams, Tony Dimitriou's prompt set, greedy, thinking off. C1 is the second of two passes, as in the earlier 3-way.

C and B scale; D does not. Flare's package caps speculative tokens per step at 16. At 8 sequences that leaves room for one token each, so drafting switches off and D flattens at ~126 tok/s. Raising the cap to 96 lifted D's six-stream number to 149 but cut its one-stream rate to ~50, because Flare re-tunes its verify policy per export. We kept the default.
D trails C by 18–31% in every prompt category at one stream (0.69–0.82×), about equally on code (where drafts are usually accepted) and on prose (where they are not). That points at a higher per-step cost on this model, not just weaker speculation. We did not profile kernels.
A's one-stream lead is the quant. Fewer bits to move per token helps a memory-bound decoder; it stops helping once the batch grows.

Long context: read fast, or hold a lot

Prefill and 300K reads are cold, unique prompts with 1-token replies; every needle was found on every lane. KV pool is what each engine reported at boot.

Prefill is a B/C tie at ~6K tok/s: a 93K-token document waits ~15 s for its first token. D waits ~25 s, A ~35 s.
Flare's KV pool is the standout. Its DeepSeek recipe shards each request's attention context across the four nodes instead of copying it to each, so the same 128 GB nodes hold 37.8M tokens against C's 6.5M. That is room for all 16 sequences at the full 1M context at once. We only tested to 300K.
Why D's prefill is slower: a 4,096-token prefill chunk (B and C use 8,192) and Engram tables read from NVMe during the step. On D's first boot, 33.9 s of 35.5 s of Engram gather time was disk wait. Neither lever was tuned.

Agents: a tie, and why

B, C and D ran back to back on Sparks 1–4 on October 21, same Hermes home and temp directory, every request logged through a proxy. A is the October 19 run.

Warm, B, C and D all spend 0.70–0.74 s per turn. A turn re-sends a 3.6K–6.5K-token conversation that is almost entirely cached and asks for a ~71-token tool call. Long-prefill and many-stream gaps never come into play.
RadixAttention did not help B here. SGLang reported cache hits in exact multiples of 256 tokens on all 42 warm turns: for this model it caches in 256-token pages and recomputes 4–250 tokens per turn. That costs under 50 ms. B's extra ~0.04 s per turn is decode.
Two Flare quirks an agent user will notice. Flare streams a tool call as one chunk at the end instead of token by token. And Hermes' end-of-session title request (strict json_schema output) came back instantly and empty, so sessions do not get auto-titles. B and C answered it in 0.3–0.7 s.
Flare's first run is slow (23.8 s vs 18.0 s warm, twice now). Our unconfirmed guess: Engram rows come from NVMe, and the page cache was cold after boot.

About Flare, for anyone trying it

NVIDIA Flare (Apache-2.0, preview, published October 20) compiles a model, its kernels and its CUDA graphs into a signed "runtime package" with PyTorch, then serves that package from a Rust binary with no Python. We tested commit 47581cc with its DeepSeek-V4.1 recipe: TP4 only, experts fully on the GPU, Engram tables on NVMe, two RoCE rails per peer auto-planned on our switch. It worked on the first serve, which is notable for a day-old preview. What cost us time:

Sampling is baked into the package. The recipe pins temperature to 1.0 and silently ignores a request's temperature: 0. A one-line recipe patch (ranges={"temperature": SamplingRange(1e-5, 1.)}) plus a re-export fixes it; true T=0 is impossible.
Exports are big and slow. ~1.7 h the first time (peers pull weights from rank 0 over TCP at ~95 MB/s even when every node has the checkpoint), ~65 min cached, and 476 GB per package. Disk filled to zero once.
Docker gotchas: add --security-opt seccomp=unconfined (io_uring), and put --output inside the cache mount (it hardlinks blobs). Changing a capacity flag needs the compiled-forward cache moved aside.
API differences: no /tokenize; thinking is chat_template_kwargs.enable_thinking, not thinking; thinking defaults on.

Flare also runs on one GB300 Station, but its DeepSeek recipe needs four GPUs, so that test used GLM-5.3-Flash: NVIDIA Flare on One GB300.

What we got wrong along the way

"Temperature explains Flare's slow decode." Our first version said so, because Flare had sampled at T=1.0 while the others ran greedy. We patched and re-exported: near-greedy decode was the same (76 tok/s one stream; 2.90 vs 2.73–2.88 tokens per verify round). The gap is the engine.
"Flare is the fastest agent engine." Its first agent run read 17.1 s against C's 19.5 s, but those runs were two days apart on different nodes. Side by side it is a tie (18.0 vs 18.5 s), and the "Flare writes 10% fewer tokens" gap shrank to 71 vs 72 per turn.

Method and caveats

Decode, prefill and needle numbers for A, B and C are from the 3-way post (October 18–20); B/C prefill and needles use their round-2 re-measurements. D ran October 20–21 on Sparks 1–4 with the same harness. The agent chain for B/C/D is one side-by-side session.
Harness: Tony Dimitriou's v41bench_tf.py (decode), prefill_reps.py (3-rep cold medians), long_check.py (needles), knapcio's qeval (75 tasks), and gauntlet_reps.py (Hermes 20-hop, 3 reps).
Lane D decode used T=1e-5 (Flare's minimum); the others T=0. D's two boots differed by up to ~9% on prefill; we report both.
None of the four was tuned beyond its published recipe, except one rejected Flare capacity experiment.
All weights are the same DeepSeek release (HF rev 1bae0e1); A is a third-party 3.5-bit quant of it.

Reproduce

B, SGLang TP4 over RoCE: J-M-Recipes · sglang-tp4-roce
C, vLLM fork with sparknet collectives: J-M-Recipes · vllm-tp4-sparknet
A, TensorFold EXL3: see the 3-way post.

D, Flare: the commands we ran

# preparation image per node: nvidia/cuda:13.2.1-devel-ubuntu24.04 + rustup + uv + Flare @ 47581cc
# cargo build --release --manifest-path flare/Cargo.toml ; uv sync --locked --extra prepare --extra test

# agents on every node, then pair from rank 0 (identical 32+ byte agent.token, mode 600, on all nodes)
flare agent install --join-code CODE --listen 10.0.0.N:29900
flare orbit pair 10.0.0.1:29900 10.0.0.2:29900 10.0.0.3:29900 10.0.0.4:29900 --code CODE

# export on rank 0 (ranks 1-3 run: ignition join 10.0.0.1:29630 --rank R --ranks 4, same docker flags)
docker run --network host --gpus all --ipc host --ulimit memlock=-1:-1 --device /dev/infiniband \
  --cap-add IPC_LOCK --cap-add SYS_NICE --security-opt seccomp=unconfined \
  -v $CKPT:$CKPT:ro -v ~/flare/config:/root/.config/flare -v ~/flare/cache:/root/.cache flare-prep:47581cc \
  .venv/bin/python -m ignition export $CKPT --gpus 0 --ranks 4 --rendezvous 10.0.0.1:29630 \
  --max-concurrency 16 --mtp-draft-tokens 5 --expert-scale-compression csf \
  --output /root/.cache/flare/runtime/dsv41-1017-csf-tp4-c16.safetensors

# near-greedy: bind-mount a patched ignition/recipes/deepseek41/recipe.py (read-only) on every rank:
#   sampling_defaults=SamplingDefaults(temperature=1., top_p=1., top_k=-1,
#       ranges={"temperature": SamplingRange(1e-5, 1.)})

# serve
flare orbit run spark-290c:/home/milo/flare/cache/flare/runtime/dsv41-1017-csf-tp4-c16.safetensors \
  --cluster sparks --gpus 0 --host 0.0.0.0 --port 8003 --served-model-name deepseek-v4.1-flash-flare \
  --dangerously-approve-key <your dev key> --dangerously-accept-publisher local-development

Serving now

As of October 21, 2026 at 8:55 AM CDT: Lane B (SGLang) serves on Sparks 1–4 as our daily DeepSeek lane. C, D and A are stopped and kept on the same nodes; C restarts in ~100 s.