We’re on Keys’ anchored 0731 ablit.
dspark-vllm-gx10:0.1.1
and the
MiaAI-Lab 2× Spark recipe
at 1M context / 6 seqs / util 0.85 / thinking off.
The checkpoint is now
drowzeys/keys-DeepSeekV4-Flash-GA-0731-Dspark-Abliterated-Anchored-Tensors
@ a1e69379
by Keys (@u1tra_instinct / drowzeys).
Served name: deepseek-v4-flash-0731-keys-ablit.
dspark_proposer.py / VLLM_DSPARK_DRAFT_CAPTURE_SIZES.
Live Keys-anchored count 1→300 this session: 91.15 / 87.88 t/s (median 89.52),
already in his after-patch band. Queues idle. Production unchanged.
Diagram is today’s routing, not a speed claim. Official 0731 is still the base checkpoint. Keys only edits the refusal path and leaves the DSpark/MTP tensors stock.
Credit where it belongs
Keys built the weights we are running. His public write-up is
How A Properly Performed Abliteration Enhances DeepSeekV4-Flash 0731.
The Hugging Face repo is gated; the old …-Abliterated-32-32 id redirects here.
The idea is simple: if you project refusal out of every attention writer, including the DSpark layers, the verifier stops refusing but the stock drafter keeps proposing refusal-shaped tokens. Speculative decode then eats itself. Keys stops the edit at L10–35, then restores L36–42 to official SHA-256 and never touches MTP. That is the “anchored tensors” part.
We also owe him earlier DSpark concurrency work that the whole Spark community stands on. The serving image is Anemll. The 2× recipe is MiaAI-Lab @ 489af95. The base model is deepseek-ai/DeepSeek-V4-Flash-0731 @ 9e165c30.
What is actually running
| Piece | Value |
|---|---|
| Hardware | 2× NVIDIA DGX Spark (GB10), TP=2 over RoCE |
| Image | ghcr.io/anemll/dspark-vllm-gx10:0.1.1@sha256:a8394849… |
| Recipe | Mia 489af95 · 1M / seqs=6 / batch 8192 / dynamic K 5/4/3 · thinking off |
| Util | 0.85 — Funland still needs this for 1M KV. We did not copy Keys’ 0.835 serve snippet. |
| Weights | Keys anchored 0731 @ a1e69379 · λ=3.5 · L10–35 only · MTP stock |
| Served | deepseek-v4-flash-0731-keys-ablit |
| Flag | ABLITERATED=0 — we pointed the official slot at his repo. Do not flip the launcher flag blindly. |
What we measured today
Same-session Funland gates, Spark1 loopback, JSON thinking: false. We also ran the same fixtures on a public cebeuq overlay earlier this morning, then swapped.
| Fixture | cebeuq overlay | Keys anchored |
|---|---|---|
| KV tokens @ 1M | 2,195,345 · 2.09× | 2,201,242 · 2.10× |
| Exact + tools | pass | pass |
| Count 1→30 | 0.87s | 0.84s |
| Warmup C4 / C6 mean K | 4.00 / 3.07 | 4.02 / 3.08 |
| Warmup C4 worst TTFT | 5.56s | 5.34s |
stream:false, usage.completion_tokens/wall, thinking false): count 1→300 best 91.15 t/s / median 89.52; count 1→150 86.68; repeat 93.07; short code 65.19; JSON 51.95. Tony’s production after Patch A was 87.8 / 87.1 / 89.3 versus 78–81 before. Keys-anchored Anemll is already there without his B12X overlay.
Performance: prefill, decode, quant
Fresh live pass from the lab bench (August 26, 2026), same Spark1 loopback path every fixture on this page used, deepseek-v4-flash-0731-keys-ablit. Two things worth stating plainly: prefill is the honest ceiling here — decode is already in the community’s patched band — and this serve runs 4-bit, not the FP8 the checkpoint ships as.
--moe-backend flashinfer_b12x (B12X MoE kernels → W4A16: 4-bit weights, 16-bit activations; VLLM_USE_B12X_MOE=1, VLLM_B12X_W4A16_FORCE_TILE_CONFIG) and --kv-cache-dtype nvfp4_ds_mla (NVFP4 DeepSeek-MLA KV cache, GB10_HYBRID_NVFP4_M_THRESHOLD=128). Draft is dspark probabilistic at dynamic 5/4/3 ([[1,1,5],[2,4,4],[5,6,3]], MTP_NUM_TOKENS=5). Read the quality figures above as 4-bit-runtime numbers, not FP8.
Prefill (prompt processing)
Measured stream:true on unique/cold prompts (no prefix-cache reuse), chunked prefill active, thinking off. For a lone stream the first-token latency is the full verifier prefill — there is no draft before token one — so prefill tok/s = prompt_tokens ÷ TTFT.
| Prompt size | TTFT | Prefill throughput |
|---|---|---|
| 15,645 tokens | 9.65 s | 1,622 tok/s |
| 62,445 tokens | 31.63 s | 1,974 tok/s |
| 249,765 tokens | 143.8 s | 1,737 tok/s |
DSPARK_MAX_INFLIGHT_PREFILLS=2 and --long-prefill-token-threshold 1024 serialize large prefills, and --enable-prefix-caching only helps when a prefix repeats. If time-to-first-token matters more than decode rate, this geometry is the wrong one to copy.
Decode (tok/s)
Re-ran the page’s count fixture (August 26, same stream:false, usage.completion_tokens/wall, thinking off):
| Fixture | completion | wall | decode |
|---|---|---|---|
| Count 1→300 | 450 tok | 5.127 s | 87.77 tok/s |
How we installed it
22 of 48 shards are byte-identical to official 0731. We hardlinked those. The other 26 (~93 GB) are the L10–35 edit. Pull once on Spark1, rsync -aH unique blobs to Spark2 over RoCE. Copy-checkout of the August 22 recipe. Independent 55-minute deadman pointed at the previous cebeuq tree. Cold bind about 11 minutes.
weights drowzeys/keys-DeepSeekV4-Flash-GA-0731-Dspark-Abliterated-Anchored-Tensors
rev a1e69379517383be9cd78c67defb04e77ad6aa68
served deepseek-v4-flash-0731-keys-ablit
util 0.85
rollback cebeuq overlay checkout, then official aug22 / 489af95
What this is not
- Not a new runtime. Same Anemll image, same Mia pin.
- Not the old Keys Power Pack 888k @ 0.76 geometry. That failed KV on Funland. This 0731 drop is a different artifact.
- Not a claim that abliteration is free quality. It trades refusals for operator responsibility.
- Not a reopen of batch 16384 or
DEFAULT_THINKING=max. - Not Tony’s overnight Patch A. That file is for his B12X
dspark_proposer.py. Funland stays Anemll 0.1.1 @ util 0.85 / 1M. His 0.78 @ 909K is a different product.
Neighbors: Mia 489af95 pin · August 13 apply · Flash-0731 recipe · Tony 1M NVFP4 recipe.