keys suggested DFlash2 for our GLM-5.3 recipe
We tested DFlash2 against the one-GB300 selective-UVA slot-cache path. Geometry passed; K4 accepted length failed the frozen gate, so MTP stays.
Context: GB300 GLM 5.3 Testing · public recipe GLM-5.3-NVFP4-One-GB300 · daily profile live-tested at 512K context / 48 GiB bf16 KV as of September 7, 2026.
Measured verdict: stop. keys (drowzeys) suggested using incoai's DFlash2 draft model to speed up our full-GLM-5.3-on-one-GB300 recipe. The geometry matched and the candidate served successfully in explicit eager mode, but K4 produced a weighted accepted length of 1.5718 against our frozen 3.0 gate. We stopped there, skipped the expensive benchmark and quality stages by design, and returned the box to the preserved 512K/MTP lane.
keys is a community recipe author (drowzeys on GitHub) whose DFlash2 experience is with GLM-5.3-Flash on DGX Spark and Macs. His suggestion — use dflash2 to speed the recipe — was pattern-matching from a different regime: HBM-resident Flash-class models, no offload. That made it worth testing rather than safe to assume. The result below closes that uncertainty for this specific K4/UVA path.
What the evidence says
Three independent sources, one consistent direction:
- incoai's model card (the draft author): DFlash2 beats GLM-5.3's native MTP on every evaluated task — acceptance length 5.94 vs 5.12 on GSM8K, 3.2× vs 2.6× speedup at C1 — but measured on 4× GB300 TP4, everything HBM-resident.
- catid's dgx_station_benchmarks: 154.9 tok/s C1 / 742 C16 with DFlash2 — but on 2× GB300 HBM-resident PP2, not our single-box offload.
- Our own recipe: the sc13g slot-cache build with MTP(1) hits 54.7 tok/s C1 vs 43.1 without. This is the bar DFlash2 would have to beat, on hardware where cold expert rows live behind NVLink-C2C in Grace memory.
Why it is not a drop-in for us
The mechanism-level objection is simple: MTP(1) verifies 2 tokens per step. DFlash2 with K7 verifies 8. On HBM-resident silicon that wider verification is cheap. On ours, every verification step routes up to 8 tokens through 256-expert layers whose cold rows sit in host memory — up to 4× the per-step offload traffic of MTP(1), and up to 8× of plain decode. Whether our slot cache amortizes that or thrashes is the whole question, and no published benchmark answers it.
Two engineering caveats stack on top:
- Our build is patched vLLM v0.28.0 with custom UVA/slot-cache patches. I probed the image live: the
dflashspec-decode path is present (vllm/v1/spec_decode/dflash.py, config accepts the method) — so the suggestion is not engine-blocked. But our patches were never exercised together with spec decode. - Quality gate flips in DFlash2's favor, but is not automatic. Our MTP blocker was an inconclusive quality audit; DFlash2 is verified-lossless by construction, which is a cleaner position than MTP(1) — but our own Flash recipe already says "by construction is not a measurement," so a teacher-forced divergence check remains part of the plan.
The verdict
Good idea; bad transfer on this path. The DFlash2 draft itself is not the problem being claimed here. The measured result is narrower: K4 did not produce enough accepted target tokens to amortize wide verification while the 744B target's cold expert rows were served through Grace memory. The existing MTP(1) daily profile remains the practical lane.
Measured experiment: DFlash2-over-UVA stopped at K4
| Step | Gate | Why this order |
|---|---|---|
| 1. Static geometry audit | PASS: target/draft dimensions and layer mapping matched | Removed a compatibility mismatch as the explanation |
| 2. Candidate startup | PASS after one amendment: attempt one failed in CUDA-graph capture; explicit --enforce-eager booted | The failed first boot is an integration result, not an acceptance result |
| 3. K4 acceptance battery | FAIL: 1.5718 weighted accepted length; four 512-token prose/code requests | Below 3.0 means stop before spending the full campaign |
| 4. C1/C4/C8 and quality | NOT RUN | Correctly blocked by step 3, rather than converting a failed gate into benchmark tourism |
| 5. Daily lane | MTP retained: DFlash2 was not promoted to 512K | The experiment did not earn a live canary |
The four accepted-length rows were 1.5375, 1.5754, 1.6516, and 1.5284. The draft proposed 5,212 tokens and 745 were accepted. Those numbers explain the stop; the 7.8775 tok/s acceptance-harness median is included for completeness, not passed off as a matched bench3.sh result. Attribution: the idea is keys' (drowzeys); the draft model is incoai's; the one-GB300 measurements and receipts are ours.
Reproducible receipts and the frozen contract: DFlash2-over-UVA K4 result.
License note: the DFlash2 draft is released under CC BY-NC-ND 4.0. Our use is research/evaluation on our own hardware, which the license permits; serving anything commercial off this recipe would need a separate license from inco.ai.