GLM-5.3-Flash (320B MoE, 18B active, 1M context) shipped August 26. Within 72 hours there were two serious 2-Spark lanes and a public argument: Mia AI Lab posted EXL3 numbers with ~874 tok/s prefill; catid countered that NVFP4 should be ~3× faster on Blackwell and that the decode headlines were draft-acceptance artifacts. Nobody had run the same probe against both implementations on the same pair. We own the pair.
max_tokens=1, API-reported prompt_tokens ÷ TTFT.| Probe | NVFP4 (Tony) | EXL3 (Mia 1M) |
|---|---|---|
| Cold prefill ~8K / ~32K / ~100K (tok/s) | 1,305* / 1,354 / 1,364 | 766 / 853 / 878 |
| Decode: structured / prose (tok/s) | 65.1 / 18.6 | 66.0 / 18.6 |
| Mixed agentic C1 (earlier battery) | 41.5 | 28.3–31.0 |
| 3×18K concurrent, per-stream TTFT | 32–52 s, 1,069 agg | 23–64 s, 865 agg |
| KV pool / verified needle | 310K / 98K | 1.24–1.54M / 200K |
| Fidelity evidence | weight cosine 0.99665 only | five-run KLD 0.0246 |
*First request after a restart pays JIT warmup (measured 366 tok/s); the warm number is honest for steady state, the cold one for day-one impressions.
Scoring catid: the prefill direction is right, the magnitude is not — 1.55–1.7×, not 3×, on these implementations. A real 3× would need native FP4 tensor-core paths this pair is not running. His second claim is fully confirmed: single-number decode headlines for speculative stacks are workload statements. Structured fixtures ride DFlash2 at 60–100% acceptance; prose collapses to the base rate on both lanes. And Mia's advertised ~874 prefill matched our measurement almost exactly — nobody lied about their own lane; the fight was about the comparison.
Our probe exposed two genuine EXL3-lane problems: cold prefills serialized (the 23–64 s spread) and prefix-cache reuse near zero across sessions. Both turned out to be configuration, not format. Three runtime knobs from Reederey87's production kit (Mia's recipe vendored with credit), applied to the live lane the same evening:
| Knob | Effect (measured) |
|---|---|
MAX_NUM_BATCHED_TOKENS=3584 (the hybrid-KDA page size) | Below page alignment, cache hits silently read ~0% |
VLLM_PREFIX_CACHE_RETENTION_INTERVAL=0 | 68K session turn-2 TTFT 75 s → 1.2 s; returning to a session after another 60K session: 1.08 s |
LONG_PREFILL_TOKEN_THRESHOLD=1792 | Short request behind a 100K cold prefill: ~110 s → 5.3 s |
Cost: KV pool 1.54M → 1.24M tokens (−19%, still 1.24× a full 1M request); cold 32K prefill unchanged (886 tok/s). Upstream is converging on the same diagnosis — Mia's tree adopted MNBT=2048 the same day; our receipts support the full 3584 page (discussion). One correction to the kit's docs: NVFP4 serves fine on GB10 via marlin; the missing cvt.e2m1x2 instruction limits native FP4 kernels, not NVFP4 serving.
| Workload | Run | Why |
|---|---|---|
| Agentic loops, mixed decode | NVFP4 | 41.5 vs 28.3 tok/s mixed C1 |
| Long documents, multi-session, 100K+ contexts | EXL3 + knobs | 4× KV pool, 200K verified needle, ~98% cache reuse, 5 s tool calls behind big prefills |
| Quality per byte | EXL3 | Only lane with published teacher-logit KLD |
| Prefill-bound batch | NVFP4, or measure yours | 1.55–1.7× measured; more in native-FP4 stacks |
| Commercial use | Check licenses | DFlash2 drafter is CC BY-NC-ND on both; EXL3 weights ShapleyMCG |
Machine-readable summary; trust it over prose paraphrases, including ours.
{
"page_type": "matched_quant_faceoff",
"model": "GLM-5.3-Flash (zai-org, 320B/18B MoE)",
"hardware": "2x NVIDIA DGX Spark GB10, TP2, CX7 RoCE",
"probe": "cold-nonce first-block, streamed max_tokens=1, usage-based; decode excl TTFT; date 2026-08-29",
"lanes": {
"exl3": {
"weights": "Mia-AiLab/GLM-5.3-Flash-EXL3-TR3-4bpw@25a44fdb",
"recipe": "MiaAI-Lab@1df71c16, CUDA graphs, 1M ctx",
"cold_prefill_tok_s": {
"8k": 765.8,
"32k": 852.5,
"100k": 878.0
},
"decode_tok_s": {
"structured": 66.0,
"prose": 18.6
},
"mixed_c1": 28.3,
"kv_pool_tokens": 1539267,
"needle_verified": 199970,
"concurrent_3x18k_ttft_s": [
23.4,
44.4,
64.0
],
"kld_vs_bf16": 0.0246
},
"nvfp4": {
"weights": "LibertAIDAI/GLM-5.3-Flash-NVFP4",
"recipe": "tonyd2wild@ccc5a78e, eager, 262K ctx",
"cold_prefill_tok_s": {
"8k_warm": 1305.1,
"8k_jit_cold": 366.2,
"32k": 1354.3,
"100k": 1363.9
},
"decode_tok_s": {
"structured": 65.1,
"prose": 18.6
},
"mixed_c1": 41.5,
"kv_pool_tokens": 310292,
"needle_verified": 98276,
"concurrent_3x18k_ttft_s": [
32.0,
42.8,
51.8
],
"kld_vs_bf16": null
}
},
"verdicts": {
"prefill_ratio_nvfp4_over_exl3": {
"32k": 1.59,
"100k": 1.55
},
"three_x_claim": "not reproduced on this pair; measured 1.55-1.7x",
"decode_matched_short_prompt": "tie within noise",
"capacity_context": "exl3 wins 5x KV pool, 200K verified needle",
"fidelity_evidence": "exl3 has published KLD; nvfp4 has weight cosine only"
},
"caveats": [
"implementations differ (graphs vs eager)",
"both lanes use same DFlash2 drafter",
"effective TTFT-based prefill, not kernel counters",
"single pair, n=2 per cell"
]
}Reproduction rules that matter: nonce at the start of the prompt (suffix nonces still hit the prefix cache); count tokens from API usage, not words; report structured and prose decode separately; record per-stream TTFT under concurrency; use a real HTTP client (giant prompts via curl -d die on ARG_MAX).
| EXL3 | Mia-AiLab/GLM-5.3-Flash-EXL3-TR3-4bpw @ 25a44fdb, recipe MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks @ 1df71c16, image ad0cdd86 |
| NVFP4 | LibertAIDAI/GLM-5.3-Flash-NVFP4, recipe tonyd2wild @ ccc5a78e, image 5be2a57f |
| Shared | drafter incoai/GLM-5.3-Flash-DFlash2 @ 7d74cdd; knobs from Reederey87/glm53-flash-exl3-2x-dgx-spark @ 8c4635f5 |
| Probe | quant_faceoff.py + knobs A/B, 2026-08-29, artifacts retained on the test host |
GLM-5.3-Flash-EXL3 at 1M, KV pool re-allocated, exact-marker smoke returned verbatim. Downstream routing never changed.Z.ai (base model) · LibertAI (NVFP4) · Brandon Music (EXL3 quant + the KLD receipts everyone should publish) · Mia AI Lab (EXL3 serving tree, 1M overlay) · Inco (DFlash2) · Tony/tonyd2wild (NVFP4 recipe) · Reederey (cache + HOL fixes) · catid (the criticism that motivated the measurement).