The dual-DGX Spark DeepSeek V4 Flash lane moved again today. This time the change was worth taking seriously: MiaAI-Lab published an updated DSpark recipe using Anemll's GX10 vLLM image, and the public numbers suggested a real single-stream gain over our known-good Patch3 lane.
1M/6 canary live. It passed direct API smoke, tool-call smoke, Hermes route smoke, and a fresh short streaming throughput check on our pair.
/v1/models reported max contextDiagram reflects the July 29 live canary: Spark1 is the OpenAI-compatible API head, Spark2 is a headless TP worker, and the Anemll image carries the DSpark/NVFP4/MTP serving path.
The previous production baseline was the Patch3 350K/12 lane under /home/milo/ds4-f-wesche-keys-c12. It was boring in the best way: it served Hermes reliably, gave us tool calls, and had a rollback path I trusted.
The new MiaAI-Lab post changed the decision calculus. Their public tweet showed roughly 72 tok/s single-stream decode and up to roughly 137 tok/s aggregate at six concurrent streams. The repository README carried a separate benchmark table with a median C1 around 66.6 tok/s. Against our old Patch3 C1 proof around 60.4 tok/s, that put the plausible improvement band in the low-double-digits to high-teens. James called it “looks 15% faster.” That was a fair read.
The speed was only half the reason to try it. The more important part was the serving shape: 1M max context with concurrent agent slots, using a prebuilt runtime image instead of continuing to carry every historical local overlay forward.
| Item | Current value |
|---|---|
| Checkout | /home/milo/ds4-f-mia-anemll-1m6 |
| Container image | ghcr.io/anemll/dspark-vllm-gx10:0.1.1 |
| Model served as | deepseek-v4-flash-dspark |
| API | http://192.168.1.11:8888/v1 |
| Head / worker | Spark1 head, Spark2 headless worker over the 10.0.0.1 ↔ 10.0.0.2 fabric |
| Context | MAX_MODEL_LEN=1048576 |
| Slots | MAX_NUM_SEQS=6 |
| Batch tokens | MAX_NUM_BATCHED_TOKENS=8192 |
| Speculative decode | MTP5 / DSpark path |
| KV dtype | nvfp4_ds_mla |
| Memory setting that worked here | GPU_MEMORY_UTILIZATION=0.85 |
| Rollback lane | /home/milo/ds4-f-wesche-keys-c12, Patch3 350K/12 |
GPU_MEMORY_UTILIZATION=0.80 did not leave enough KV room for 1M on our pair. The failed boot reported about 7.16 GiB available KV versus 7.54 GiB required, with an estimated max length around 953,088. After cleanup, 0.85 was the setting that came up cleanly.
I did not start by rewriting the production route. The canary went into a separate checkout so the old lane stayed intact:
/home/milo/ds4-f-mia-anemll-1m6 # new MiaAI-Lab / Anemll lane
/home/milo/ds4-f-wesche-keys-c12 # old Patch3 rollback laneThe old DS4-F lane cannot practically run beside this canary on the same two Sparks; both want the pair. So the safe sequence was:
ghcr.io/anemll/dspark-vllm-gx10:0.1.1 on both nodes.The first attempt stalled on insufficient KV budget. The second attempt, with GPU_MEMORY_UTILIZATION=0.85, reached a healthy OpenAI-compatible API and became the live spark-ds4 backend.
These are fresh operational probes from July 29, 2026. They are not leaderboard claims; they are “is the route alive and fast enough for James/Milo agent work?” checks.
| Gate | Result |
|---|---|
/v1/models | id=deepseek-v4-flash-dspark, max_model_len=1048576 |
| Exact smoke | CANARY_BLOG_OK |
| Tool-call smoke | Returned get_weather({"city":"Pensacola, FL"}) as a structured tool call |
| Hermes route smoke | ROUTE_OK through provider spark-ds4 |
| Fresh short streaming probe | Success | Decode tokens | Wall | Mean stream decode | Wall aggregate decode |
|---|---|---|---|---|---|
| C1 | 1/1 | 255 | 3.33 s | 82.8 tok/s | 76.6 tok/s |
| C4 | 4/4 | 1,020 | 4.82 s | 59.1 tok/s | 211.6 tok/s |
| C6 | 6/6 | 1,530 | 6.98 s | 42.5 tok/s | 219.2 tok/s |
Method note: the table above is a small streaming sanity harness with unique prompts, temperature 0, and max_tokens=256. Mean stream decode uses each request's content-token window; wall aggregate is total decoded tokens divided by benchmark wall time. It is intentionally smaller than the upstream README benchmark. Use it as a local receipt, not a universal speed claim.
0.80 may be right elsewhere; on our pair it was too conservative for the target 1M profile.350K/12 is slower and smaller-context, but it is a known-good fallback. Do not delete it just because this canary passed.Thank you to Mia / MiaAI-Lab for publishing the two-DGX Spark recipe, the update announcement, and the benchmark trail. This is exactly the kind of public operational work that makes small-lab local inference move faster: not just “it works,” but enough recipe detail for another operator to reproduce, adapt, and find the sharp edges.
Thank you also to Anemll for the GX10/DGX Spark vLLM image this lane now uses. The Anemll credit file also points to the deeper stack: vLLM, FlashInfer, Luke Alonso's b12x kernels, Keys / drowzeys, Rafael Caricio, Fraser Price, TonyD2Wild, and the broader DeepSeek DSpark work. The live recipe here is downstream of all of that.
Source trail:
# Health check
curl -sS http://192.168.1.11:8888/v1/models | python3 -m json.tool
# Expected shape
# id: deepseek-v4-flash-dspark
# max_model_len: 1048576
# Hermes smoke
hermes chat --provider spark-ds4 \
-m deepseek-v4-flash-dspark \
-q 'Reply exactly ROUTE_OK.' \
-Q --toolsets safeIf it fails, stop the canary compose project on both Sparks, clean shared memory/page cache, and either relaunch this checkout with the known-good 0.85 memory setting or roll back to /home/milo/ds4-f-wesche-keys-c12. The lesson from this round is simple: keep the rollback lane boring, but do not ignore public recipe improvements when the receipts are this concrete.