August DS4-F Testing: Anemll Stays Default
July closed with the Anemll 1M/6 recipe as the sticky dual-Spark route. Late July brought a fresh community package from r0b0tlab: vLLM 0.26.0+dspark.sm121.2, FlashInfer B12X, DSpark K=6, published C1 ~80.6 and C16 ~402. We ran a full A/B canary on the real pair, then put Anemll back.
Candidate is functionally good (tools + Hermes) and strong at C16, but Hermes-shaped Keys C1 is a tie and C4 drops ~13%. No cutover.
Architecture at a glance
Before → candidate → decision. Same pair, same weights, same Keys harness. Anemll is production again; the v0.26 image stays on disk.
What was tested
| Lane | Runtime | Shape | Role this session |
|---|---|---|---|
| A · Anemll | ghcr.io/anemll/dspark-vllm-gx10:0.1.1 |
1M / seqs=6 / batch=8192 / MTP5 / util=0.85 |
Baseline + restored default |
| B1 · r0b0tlab | v0.26.0+dspark.sm121.2 digest sha256:269a1c6a… |
327680 / seqs=16 / batch=16384 / K=6 / B12X |
Primary throughput canary |
| B2 · r0b0tlab long | same image | 327680 / seqs=1 / batch=8192 / K=6 |
Deep-prefill needle ladder (partial) |
Weights revision on both nodes: 913f0657a874f76844e2e91cbe706dbcaceeb6d7. Pair was down at session start (Anemll container exited ~18h); restored before baseline benches.
Functional gates
| Gate | Anemll A | r0b0tlab B1 | Anemll restore |
|---|---|---|---|
/v1/models max_model_len | 1,048,576 | 327,680 | 1,048,576 |
| Exact marker | PASS | PASS | PASS |
Tool call (get_weather) | PASS | PASS | PASS |
| Streaming | PASS | PASS | — |
| Staggered C2 | — | PASS | — |
| Staggered C16 | — | 16/16 PASS (agg 261.2 t/s) | — |
Hermes spark-ds4 | ROUTE_OK | ROUTE_OK | ROUTE_OK |
| Correctness under churn | — | byte-identical; accept 0.333 (watch) | — |
The Keys correctness harness labels low acceptance as FAIL; r0b0tlab’s own claims treat identical output as the hard gate and acceptance-under-churn as watch-only. We match that reading.
Throughput (same Keys harness)
Primary comparison uses benchmarks/keys-concurrency/bench_concurrent.py best-of-2 after warmup — the same ladder we used for the July Anemll and Patch3 reports.
| Probe | Anemll A | r0b0tlab B1 | Delta |
|---|---|---|---|
| Keys C1 | 72.4 t/s | 72.7 t/s | ~tie (+0.4%) |
| Keys C4 | 173.8 t/s | 151.1 t/s | −13% |
| Keys C6 | 211.0 t/s | 237.6 t/s | +13% |
| Keys C8 | n/a (cap 6) | 259.3 t/s | — |
| Keys C16 | n/a | 406.6 t/s | matches pub ~402 |
| Dedicated C1 ×2048 median | not run on A | 81.6 t/s | beats pub 80.58 |
| Draft accept (Keys C1) | ~0.74–0.78 | ~0.58–0.70 | lower on B1 |
After restore, Anemll Keys C1 was 69.7 t/s — normal cold variance versus the morning baseline, not a regression claim.
Long-context B2 (partial)
Separate boot: seqs=1, batch=8192, KV ~675k tokens. Needle retrieval (“secret code”) with synthetic filler:
| Target | Actual prompt tokens | Result | Wall |
|---|---|---|---|
| ~8k | 7,124 | PASS | 87.5 s |
| ~32k | 28,368 | PASS | 291 s |
| ~64k | 56,698 | PASS | 270 s |
| 128k+ | — | Not completed | stuck under seqs=1 |
This is not a Funland 300k qualification. It is proof that B2 deep-prefill works through ~57k actual tokens. Under max_num_seqs=1, one wedged long request owns the API until restart — plan a watchdog before finishing the ladder.
What changed since July
| Topic | July end-state | This canary |
|---|---|---|
| Default route | Anemll 1M/6 after MiaAI-Lab recipe | Still Anemll 1M/6 after explicit A/B |
| Community pressure | Patch3 / Wesche-Keys / Anemll ladder | r0b0tlab v0.26 SM121 package + GHCR image |
| Interactive C1 story | Anemll Keys C1 ~71 t/s | Candidate Keys C1 ~tie; dedicated 2k decode strong |
| Multi-stream story | Anemll capped at C6 ~225 t/s | Candidate C16 ~407 t/s if you need pile-up |
| Context ceiling | Server 1M, Hermes capped 200k | Candidate ~320k server; Hermes still 200k |
Interpretation
For single-user Hermes / agent work on this pair, r0b0tlab v0.26 is not a free upgrade:
- C1 interactive did not beat Anemll on the harness we actually use for route decisions.
- C4 got worse — the concurrency band Desktop/kanban often lives in.
- 1M headroom and larger KV stay with Anemll even though Hermes only advertises 200k.
Where the candidate is genuinely interesting:
- Many short concurrent streams (C8–C16), if that becomes a real Funland load.
- Long-decode claim class (~81–84 t/s at 2048 tokens) once you separate it from Keys short-gen.
- Packaging quality — immutable GHCR digest, sealed claims JSON, dual profiles (prod decode vs C1 long).
spark-ds4 on Anemll 1M/6. Keep the v0.26 image tagged on both Sparks for a later multi-agent-focused canary, not a silent default flip.
Ops notes worth keeping
- HF hub snapshots need the full model repo mount (blobs + snapshots). Snapshot-only mounts break relative symlinks and fail model load.
- Multi-node Gloo needs
VLLM_HOST_IPon each rank plusGLOO/TP/NCCL_SOCKET_IFNAME=bond0and dual IB HCAs — otherwise init dies on127.0.0.1. - First B1 cold bind ~13–15 minutes (FlashInfer autotune + graphs). Warm restarts are faster.
- B2
seqs=1means one stuck long request wedges the endpoint; restart the stack, don’t wait forever.
Provenance
| Item | Value |
|---|---|
| Canary date | July 31, 2026 (Funland CDT) |
| Endpoint | http://192.168.1.11:8888/v1, model deepseek-v4-flash-dspark |
| Weights | deepseek-ai/DeepSeek-V4-Flash-DSpark @ 913f0657… |
| Anemll image | ghcr.io/anemll/dspark-vllm-gx10:0.1.1 · checkout /home/milo/ds4-f-mia-anemll-1m6 |
| Candidate image | ghcr.io/r0b0tlab/deepseek-v4-flash-dspark-v026-sm121:v0.26.0-sm121-optimized · digest sha256:269a1c6ac581f4fb2fe13110d2468159c3ca65c07ffb09b4080549673f625841 |
| Keys harness | /home/milo/ds4-f-wesche-keys-c12/benchmarks/keys-concurrency/bench_concurrent.py |
| Dedicated C1 | scripts/benchmark_dspark.py --concurrency 1 --repeats 5 --max-tokens 2048 (median of client decode rates) |
| Hermes proof | hermes chat --provider spark-ds4 -m deepseek-v4-flash-dspark -q 'Reply exactly ROUTE_OK.' -Q --toolsets safe |
| Session artifacts | ~/.hermes/work/ds4f-canary-2026-07-31/DECISION.md and Spark1 /home/milo/ds4f-canary-2026-07-31/ |
| Public upstream package | r0b0tlab/DeepSeek-V4-Flash-DSpark-v026-SM121 |