This is the short archive entry for the June MiMo V2.5 work, now updated with the July 1 upstream reproduce. It was a failure as a default-route candidate, but a useful one: it found real serving limits, wrapper bugs, recovery discipline, and finally the key mismatch versus Tony/Karol — our Sparks exposed less than half the public KV-cache pool.
| Question | Answer | Evidence |
|---|---|---|
| Can MiMo V2.5 NVFP4 run on the dual Sparks? | Yes, narrowly. | 128K historical boot; 64K and 96K text-only/no-MTP profiles reached /v1/models and passed smoke. |
| Can it become a local Hermes default? | No. | The full tool benchmark scored 86, 96K regressed to 87 on the short benchmark, and DS4-F remained faster and simpler operationally. |
| Can we claim practical 500K or 1M? | No. | 500K/1M attempts failed around KV/cache reservation or never became a benchmarkable route. |
| Did Omni/vision justify the work? | Not in this run. | The measured routes were text/tool only with multimodal reservation disabled. |
| Was it still useful? | Yes. | It exposed the hidden MTP_SPEC_TOKENS=1 override, confirmed the GB10 page-cache cleanup trap, and clarified where bootability stops being deployability. |
| Measurement | Published value | Interpretation |
|---|---|---|
| Best measured profile | 64K, text-only, thinking off, true no-MTP | Stable enough to benchmark, but too narrow to promote. |
| Short pilot | 97, 29 / 30 points | Excellent first signal, but not enough by itself. |
| Full tool benchmark | 86, 119 / 138 points | Good tool syntax, wrong operational answer for default routing. |
| 96K retest | 87, 26 / 30 points | Context climb worked technically but degraded quality. |
| Cold prefill shape | ~1.6k prompt tok/s at 10.5K; ~823 prompt tok/s at 41.5K | Memory-tight and slower than the route it needed to replace. |
| Comparison | KV-cache pool | What it means |
|---|---|---|
| Tony/Karol 1M/C8 public profile | 2,171,757 tokens | Enough room for the advertised 1M shape. |
| Tony/Karol 65K recovery profile | 2,416,341 tokens | The smoke lane we should match before climbing. |
| Our July 1 best local upstream boot | 954,040 tokens | Only 43.9% of Tony's 1M/C8 pool; 1M cannot fit. |
The next question is no longer “which MiMo flag did we miss?” It is “why does this host/driver/runtime stack expose a much smaller usable KV envelope?” Until that is explained, DS4-F stays the practical route.
MiMo's failure was not a single crash. It was a sequence of narrowing truths: the patched runtime could load the model; video and multimodal reservations were too expensive; hidden wrapper defaults could invalidate a test; page cache on GB10 could poison startup memory; and a model that can pass smoke still might not be worth routing production agent traffic to.
That distinction matters. “It boots” is not the finish line. “It beats the incumbent route under the workload we actually use” is the finish line. MiMo did not clear that bar in June.
The July 1 lesson is that our measured fallback profile was not comparable to the public winning lane, and the upstream reproduce exposed a deeper KV-envelope mismatch. If we revisit MiMo, the plan should be narrow and evidence-first:
| Step | Gate |
|---|---|
| Fresh upstream repo and vendored mods on both Sparks | No old local fork assumptions; exact current recipe recorded. |
| 65K/C8 full-Omni KV pool | First match the public ~2.4M-token KV class. If we stay near 954K, stop and debug stack/driver/runtime. |
| 1M/C8 NVFP4-KV reproduce | Only climb after the 65K pool matches. Then verify KV pool, C1 decode, C8 aggregate, and bounded long-context request. |
| Spark-Bench v5c plus our tool-eval | Separate comparable tables; no mixing harnesses. |
Hermes mimocanary only | DS4-F stays default unless MiMo survives real canary traffic. |
This archive entry summarizes already-published June measurements plus the July 1 KV-envelope reproduce result. It is an operational update, not a fresh quality benchmark.