NVIDIA's NVFP4 DeepSeek V4.1 Flash on four DGX Sparks: same speed, lower quality
In plain English. DeepSeek ships V4.1 Flash in its own compact number format (FP8 plus MXFP4). NVIDIA re-packed it into a
different one, NVFP4. A post on X showed NVFP4 running on four DGX Sparks with Mia's TensorFold recipe, decoding 2.1× faster than vLLM.
We wanted to know two things: is NVFP4 itself faster, and does it cost quality? So we ran the same publisher's tests on four setups on the
same four Sparks: Mia's recipe on NVFP4, Mia's recipe on the original weights, and our two recipes on the original weights.
- NVFP4 is not faster. Same engine, same settings, only the weights swapped: every speed number is within 3%. The 2.1× in the post is TensorFold against one vLLM recipe, not NVFP4 against the original format.
- NVFP4 knows less. On all 12,032 MMLU-Pro questions it scores 67.1% against 71.2% for the original weights, and it is lower in every one of the 14 subjects.
- Our recipes read long prompts faster. SGLang processes an 8K–32K prompt 1.8–1.8× faster than Mia's TensorFold, and our vLLM recipe 1.5–1.5×. Typing speed with one user is about the same everywhere.
Verdict
Run the original weights. On Mia's TensorFold, NVFP4 matches the original weights on speed and loses
4.1 points on MMLU-Pro (12,032 questions, p < 0.001). Among the original-weight setups, the three are tied on quality;
SGLang wins prefill and vLLM wins decode under load.
SGLang v2.5 · our recipeFastest prefill at every size: 6,427 tok/s at 32K, first token at 128K in 20.7 s. Our daily lane.
vLLM r6x · our recipeBest with several users: 195 tok/s total at 8 at once. Best quality mean (63.1), tied within noise.
Mia TensorFold · NVFP4Same speed as on the original weights; about 4 points lower on MMLU-Pro.
Mia TensorFold · originalThe control: same commit, image, switches and sampling as the NVFP4 lane.
Setup
All four setups ran one at a time on the same four DGX Sparks (GB10, tensor parallel 4) through our MikroTik switch over 2×200G RoCE, 2200 MHz clock cap, driven from one client with one harness. The model is DeepSeek-V4.1-Flash-1017 with DSpark drafts on every lane.
| Setup | Engine and recipe | Weights | Sampling |
|---|---|---|---|
| SGLang v2.5 | knapcio's SGLang profile v2.5, our recipe deepseek-v4.1-flash-sglang-tp4-roce, served as it runs in production | original FP8/MXFP4 | T 1.0, top-p 1.0 |
| vLLM r6x | Christopher Owen's spark-ds41f r6x vLLM fork, our L20 late-admission + sparknet 0.5.0 shape, 32 seats (deepseek-v4.1-flash-vllm-tp4-sparknet) | original FP8/MXFP4 | T 1.0, top-p 1.0 |
| Mia TensorFold · NVFP4 | Mia's recipe @fcef6cf, NVFP4=1, KV pool 2,097,152, 16 streams, all eight "on, this cluster" switches, switch mode | nvidia/DeepSeek-V4.1-Flash-1017-NVFP4 @480550d3 | T 1.0, top-p 1.0, top-k 20 (her server default) |
| Mia TensorFold · original | the same commit, image and settings, NVFP4=0 | original FP8/MXFP4 | same as NVFP4 |
The tests are niklaslenz.com's Speed v2 and Quality v2, rebuilt from his published raw files (method, item IDs, answers, prompt fingerprints). His speed tool is a copy of the public rigmark; his quality kit is private, so we rebuilt it. What that does and does not let us compare is in What this is not.
Prefill: how fast each setup reads the prompt
Figure 1. Time to first token for a prompt of 2K, 8K, 32K and 128K tokens. Thinking off, no prompt cache, token-exact prompts, median of 3 (2K–32K) or 4 (128K) requests.
| SGLang v2.5 our recipe | vLLM r6x our recipe | Mia TensorFold · NVFP4 the post's recipe | Mia TensorFold · original control | niklaslenz's NVFP4 run his ring, Nov 1 | |
|---|---|---|---|---|---|
| Prefill at 2K (tok/s) | 3,903 | 3,825 | 2,481 | 2,436 | 2,724 |
| Prefill at 8K (tok/s) | 5,747 | 4,623 | 3,172 | 3,136 | 3,654 |
| Prefill at 32K (tok/s) | 6,427 | 5,394 | 3,482 | 3,397 | 3,968 |
| Prefill at 64K (tok/s) | 6,432 | 5,410 | 3,504 | 3,417 | — |
| Prefill at 128K (tok/s) | 6,348 | 5,246 | 3,386 | 3,366 | — |
| First token at 2K (s) | 0.52 | 0.54 | 0.83 | 0.84 | 0.75 |
| First token at 8K (s) | 1.43 | 1.77 | 2.58 | 2.61 | 2.24 |
| First token at 32K (s) | 5.10 | 6.08 | 9.41 | 9.65 | 8.26 |
| First token at 128K (s) | 20.7 | 25.0 | 38.7 | 38.9 | — |
Green = best of our four. The last column is niklaslenz's published Mia NVFP4 run on his direct-cable ring. It is about
15% faster at 8K than the same recipe on our switch, so the
ring helps TensorFold somewhat. It does not close a 1.8× gap.
- NVFP4 does not speed up prefill. On the same engine it is within 3% of the original weights at every size.
- The engine decides prefill. SGLang and our vLLM fork batch the long prompt through tuned kernels at 5–6K tok/s; Mia's TensorFold stays near 3.4K. At 128K that is 20.7 s against 38.7 s before the first word.
Decode: how fast each setup writes the answer
| SGLang v2.5 our recipe | vLLM r6x our recipe | Mia TensorFold · NVFP4 the post's recipe | Mia TensorFold · original control | |
|---|---|---|---|---|
| Decode, code prompt (tok/s) | 120.7 | 123.8 | 113.2 | 113.7 |
| Decode, prose prompt (tok/s) | 66.2 | 73.4 | 66.9 | 65.3 |
| 1 request at once (total tok/s) | 61.9 | 66.9 | 63.8 | 64.7 |
| 2 at once | 93.2 | 101.8 | 87.8 | 88.8 |
| 4 at once | 125.1 | 141.6 | 115.8 | 115.8 |
| 8 at once | 158.1 | 195.2 | 144.9 | 145.0 |
- One user: a wash. Code decodes at 113–124 tok/s on every setup, prose at 65–73. Code is faster because the draft model guesses code tokens more often.
- Several users: our vLLM recipe leads. At 8 requests at once it delivers 195 tok/s in total against 145 on both TensorFold lanes.
- Decode numbers compare across our four setups only. niklaslenz's speed prompts are not published, so ours are stand-ins of the same length, and decode speed depends on the text.
Quality: niklaslenz's seven tests
| SGLang v2.5 our recipe | vLLM r6x our recipe | Mia TensorFold · NVFP4 the post's recipe | Mia TensorFold · original control | |
|---|---|---|---|---|
| MMLU-Pro (400) | 70.0 | 69.5 | 63.8 | 70.0 |
| GPQA Diamond (198) | 51.5 | 46.0 | 49.5 | 50.5 |
| IFEval (200) | 87.5 | 89.0 | 87.0 | 90.0 |
| LiveCodeBench (40) | 57.5 | 60.0 | 60.0 | 55.0 |
| Long text 8K (40) | 50.0 | 52.5 | 47.5 | 47.5 |
| Long text 32K (40) | 42.5 | 45.0 | 45.0 | 42.5 |
| GPQA, thinking on (30) | 80.0 | 80.0 | 70.0 | 73.3 |
| Mean of the 7 | 62.7 | 63.1 | 60.4 | 61.3 |
One sampled run per setup (temperature 1.0, seed 42), thinking off except the last test, 16 questions at a time. Paired question by question (McNemar on the questions one setup got right and the other wrong), the three original-weight setups are tied (SGLang against Mia original: p = 1.0; SGLang against vLLM: p = 0.56). NVFP4 is the only one that moves, and its loss sits in MMLU-Pro. One run on 400 questions is thin evidence, so we ran a second, bigger round on the two Mia setups.
Quality, in depth: NVFP4 against the original weights on the same engine
MMLU-Pro, all 12,03267.1% vs 71.2%NVFP4 vs original, temperature 0
Right on one only333 vs 832p < 0.001 (McNemar)
GPQA-198, 3 seeds48.0% vs 52.2%285/594 vs 310/594 answers right
Same greedy output8 of 200each lane vs itself: 200/200 and 200/200
Figure 2. MMLU-Pro by subject, NVFP4 minus original weights, on Mia's TensorFold @fcef6cf. Temperature 0 makes each answer deterministic; no answer was cut off on either lane (0 cut-offs), so every lost point is a wrong letter, not a formatting slip.
The same numbers as a table
| Subject | Questions | NVFP4 % | Original % | Change | p |
|---|---|---|---|---|---|
| history | 381 | 69.3 | 76.9 | -7.6 | <0.001 |
| business | 789 | 60.5 | 67.0 | -6.6 | <0.001 |
| other | 924 | 70.6 | 76.0 | -5.4 | <0.001 |
| physics | 1299 | 59.6 | 65.0 | -5.4 | <0.001 |
| computer science | 410 | 71.0 | 75.9 | -4.9 | 0.001 |
| engineering | 969 | 52.7 | 57.4 | -4.6 | <0.001 |
| chemistry | 1132 | 55.8 | 60.3 | -4.5 | <0.001 |
| math | 1351 | 63.3 | 67.5 | -4.2 | <0.001 |
| economics | 844 | 79.9 | 83.4 | -3.6 | <0.001 |
| psychology | 798 | 79.7 | 82.5 | -2.8 | 0.004 |
| law | 1101 | 63.1 | 65.8 | -2.7 | 0.014 |
| health | 818 | 75.8 | 78.5 | -2.7 | 0.009 |
| biology | 717 | 87.7 | 89.7 | -2.0 | 0.026 |
| philosophy | 499 | 71.9 | 73.3 | -1.4 | 0.381 |
- The loss is broad. NVFP4 is lower in all 14 subjects, 13 of them significantly. Recall-heavy subjects lose most (history -7.6, business -6.6); philosophy loses least.
- The first round overstated it. 400 questions with sampling showed a 6.2-point gap; all 12,032 at temperature 0 show 4.1. niklaslenz's own two Mia runs point the same way (MMLU-Pro 65.0 on the original weights, 62.0 on NVFP4).
- Graduate science, thinking off, three seeds: GPQA-198 at 49.5 / 47.5 / 47.0 for NVFP4 against 50.5 / 54.0 / 52.0 for the original weights.
- With thinking on, we could not see a gap. On the 30-question GPQA set NVFP4 scored 70.0 / 76.7 / 73.3 and the original weights 73.3 / 63.3 / 66.7 (66/90 vs 61/90). With 30 questions the seed-to-seed spread is as large as the difference, so this says neither "NVFP4 is fine when it thinks" nor the opposite. A 198-question thinking run would settle it.
- NVFP4 is a different model at the token level. TensorFold does not return token probabilities, so we compared greedy (temperature 0) answers to 200 fixed prompts instead. Each lane reproduces itself exactly (200/200 and 200/200). Against each other only 8 of 200 match; half the answers part ways within the first 6–8 tokens and a quarter at the first token. Different is not automatically worse; the MMLU-Pro result is the evidence for worse.
What this is not
- Not his numbers. niklaslenz's speed prompts and quality instructions are private. We rebuilt them: the IFEval prompts match his fingerprints 200/200 and the prefill prompts are exact to the token. The other instruction wording is a substitute of the same length, so our quality scores differ from his site (our long-text test cuts off far more answers at 128 tokens). Compare the setups on this page with each other, not with his table.
- Not his network. He runs a direct-cable ring; we run a switch. That costs TensorFold some prefill (see the note under the prefill table).
- Not every quant. This is NVIDIA's NVFP4 release @
480550d3on Mia's engine. Another NVFP4 recipe or kernel could land differently. - Not a verdict on TensorFold. On EXL3 weights a TensorFold fork was our fastest single stream in an earlier four-engine test. Here it trails on prefill.
Reproduce
Each setup took 40–60 minutes for speed plus the seven quality tests; the deeper round took about 1 h 50 min per Mia lane.
Commands
# Mia TensorFold @fcef6cf on 4 Sparks, switch mode, node-local weights (her NFS mode clashes with our SGLang exporter)
git clone https://github.com/MiaAI-Lab/DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold && git checkout fcef6cf
# scripts/local.sh: NVFP4=1 PREPARE=0 TF_DS_RDMA_GID=3 RDMA_HCA="rocep1s0f0,roceP2p1s0f0", pool 2097152, 16 streams
PREPARE=0 ./start.sh # ranks 3,2,1 then 0; "is now LIVE! on port 8001" in ~3 min
# Speed v2 (rigmark-derived), per setup
speed2.py --base-url http://HEAD:PORT/v1 --model MODEL --parts decode,prefill,conc,l1,ctx \
--prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-depths 2048,8192,32768 --prefill-runs 3 \
--conc-levels 1,2,4,8 --conc-rounds 2 --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 \
--sampling '{"temperature": 1.0, "top_p": 1.0}' [--top-k 20 on Mia's server]
# Quality v2 (reconstructed), per setup
quality2.py --base-url ... --parts mmlupro,gpqa,ifeval,lcb,ruler --parallel 16 --seed 42
quality2.py --base-url ... --parts gpqa_think --parallel 16 --seed 42
# Round 2 (Mia NVFP4 and Mia original)
quality2.py --parts mmlupro --items-dir items_full --sampling '{"temperature": 0.0}' # all 12,032 MMLU-Pro
quality2.py --parts gpqa | gpqa_think --seed 43 / 44 # two more seeds
greedy.py collect ... (x2 per lane) ; greedy.py compare A.json B.json