MiMo DFlash on Two DGX Sparks: What Worked, What Did Not

We tried to reproduce the public MiMo-V2.5 + DFlash result on James’s two DGX Spark boxes. The public target was attractive: MiMo with DFlash, NVFP4 target weights, NVFP4 KV cache, and a million-token context lane. The local result was more complicated and more useful than a simple pass/fail.

Decision: MiMo DFlash is now a real canary on our hardware, but it is not the production replacement for DS4-F. We proved stable-ish 131K NVFP4-KV + DFlash, booted 250K once, failed 500K, and restored DS4-F as the safer default route.

Stable MiMo canary131K/C1

Best MiMo boot250K/C1

Failed climb500K

DS4-F restored350K/12

The short version

Question	Answer
Did MiMo + DFlash run locally?	yes The working local path is causal MiMo target + DFlash drafter, not the Omni target class.
Did we match the public 1M post?	no We did not get a 500K or 1M DFlash+NVFP4 route to bind and pass smoke.
What is the best stable endpoint from the attempt?	`MiMo-V2.5-NVFP4-Causal-DFlash-NVFP4KV-131K-C1` passed exact marker, tool-call smoke, and DFlash counter checks.
What should be the production route?	DS4-F. The MiMo lane is a canary/research route until it passes longer context retrieval, concurrency, and restore gates.

Question

Answer

Did MiMo + DFlash run locally?

yes The working local path is causal MiMo target + DFlash drafter, not the Omni target class.

Did we match the public 1M post?

no We did not get a 500K or 1M DFlash+NVFP4 route to bind and pass smoke.

What is the best stable endpoint from the attempt?

MiMo-V2.5-NVFP4-Causal-DFlash-NVFP4KV-131K-C1 passed exact marker, tool-call smoke, and DFlash counter checks.

What should be the production route?

DS4-F. The MiMo lane is a canary/research route until it passes longer context retrieval, concurrency, and restore gates.

What actually booted

Profile	Result	Evidence	Operational read
Target-only NVFP4-KV `131K`	worked	Logged `nvfp4-kv-diffkv` and WMMA decode. Video encoder cache had to be disabled.	Proved the target-side NVFP4 KV path was real on the Sparks.
DFlash + FP8-KV `32K/C1`	worked	Six-case local bench averaged `41.1 tok/s`; draft acceptance was about `0.548` in one bring-up run.	Useful fallback, but not the public NVFP4-KV target.
DFlash + NVFP4-KV `64K/C1`	worked	Causal target class with target `triton_attn_diffkv` and drafter `triton_attn`.	First proof that DFlash and NVFP4 KV could coexist locally.
DFlash + NVFP4-KV `131K/C1`	restored/smoked	Exact marker `RESTORED_131K_OK`; tool-call smoke emitted `get_weather({"city":"Paris"})`; DFlash counters moved `77` draft / `46` accepted tokens after restore.	Best safe MiMo canary from this run.
DFlash + NVFP4-KV `250K/C1`	booted once	Passed exact-marker smoke once. A later restore after the 500K failure collided with overlapping Ray/restore work and got SIGKILL.	Promising, but not a service target yet.
DFlash + NVFP4-KV `500K/C1`	failed	Exited `137` during distributed/Ray/NCCL startup before useful KV diagnostics or weight-loading progress.	Not proven. Do not publish 500K as working.

Profile

Result

Evidence

Operational read

Target-only NVFP4-KV 131K

worked

Logged nvfp4-kv-diffkv and WMMA decode. Video encoder cache had to be disabled.

Proved the target-side NVFP4 KV path was real on the Sparks.

DFlash + FP8-KV 32K/C1

worked

Six-case local bench averaged 41.1 tok/s; draft acceptance was about 0.548 in one bring-up run.

Useful fallback, but not the public NVFP4-KV target.

DFlash + NVFP4-KV 64K/C1

worked

Causal target class with target triton_attn_diffkv and drafter triton_attn.

First proof that DFlash and NVFP4 KV could coexist locally.

DFlash + NVFP4-KV 131K/C1

restored/smoked

Exact marker RESTORED_131K_OK; tool-call smoke emitted get_weather({"city":"Paris"}); DFlash counters moved 77 draft / 46 accepted tokens after restore.

Best safe MiMo canary from this run.

DFlash + NVFP4-KV 250K/C1

booted once

Passed exact-marker smoke once. A later restore after the 500K failure collided with overlapping Ray/restore work and got SIGKILL.

Promising, but not a service target yet.

DFlash + NVFP4-KV 500K/C1

failed

Exited 137 during distributed/Ray/NCCL startup before useful KV diagnostics or weight-loading progress.

Not proven. Do not publish 500K as working.

The main technical lesson

The public-looking path was not the path that worked locally. The Omni target class was useful for proving target-only NVFP4 KV, but it failed with DFlash because the target model class did not expose the EAGLE3 auxiliary-hidden-state interface DFlash expected.

Path	Local result
`MiMoV2OmniForCausalLM` + target-only NVFP4-KV	Works at `131K` when video cache is disabled and batch tokens are reduced.
`MiMoV2OmniForCausalLM` + DFlash	Fails: `Model does not support EAGLE3 interface but aux_hidden_state_outputs was requested`.
`MiMoV2ForCausalLM` + DFlash	Works. This is the local path for DFlash + NVFP4-KV.

Path

Local result

MiMoV2OmniForCausalLM + target-only NVFP4-KV

Works at 131K when video cache is disabled and batch tokens are reduced.

MiMoV2OmniForCausalLM + DFlash

Fails: Model does not support EAGLE3 interface but aux_hidden_state_outputs was requested.

MiMoV2ForCausalLM + DFlash

Works. This is the local path for DFlash + NVFP4-KV.

target architecture: MiMoV2ForCausalLM target attention: triton_attn_diffkv target KV cache: nvfp4 drafter: XiaomiMiMo/MiMo-V2.5-DFlash dflash/* drafter attention: triton_attn block size: 64 max batched tokens: 2048 max seqs: 1 gpu memory util: 0.92

Patch trap: the container rebuild path needed both the repo mods and the local hot patches. Missing the drafter-cache/dtype guard patches made the drafter inherit the target’s NVFP4/non-causal settings and fail with TRITON_ATTN ... ['kv_cache_dtype not supported', 'non-causal attention not supported'].

Why DS4-F stays the default

This is the unromantic part. MiMo is interesting because the public result suggests stronger tool-agent quality and a larger NVFP4 KV pool. But local production routing needs boring properties: repeatable boot, repeatable restore, tool-call correctness, throughput, and a clean rollback path. DS4-F still wins that today.

Route	Known local status	Why it matters
DS4-F Patch3 `350K/12`	restored default Previous local C1 best: `56.8 tok/s`; profile persisted in `.env.dspark`.	Best proven production/default route for short and mid-context agent work.
DS4-F `1M/6`	long-context fallback Proven earlier as the million-token DS4-F lane.	Use when the context ceiling matters more than C1 speed.
MiMo DFlash `131K/C1`	canary Restored/smoked during this attempt.	Good research lane; not enough context or concurrency proof yet.
MiMo DFlash `250K/C1`	experimental Booted once, not yet safely restorable.	Next real target after serialization and cleanup hardening.

Route

Known local status

Why it matters

DS4-F Patch3 350K/12

restored default Previous local C1 best: 56.8 tok/s; profile persisted in .env.dspark.

Best proven production/default route for short and mid-context agent work.

DS4-F 1M/6

long-context fallback Proven earlier as the million-token DS4-F lane.

Use when the context ceiling matters more than C1 speed.

MiMo DFlash 131K/C1

canary Restored/smoked during this attempt.

Good research lane; not enough context or concurrency proof yet.

MiMo DFlash 250K/C1

experimental Booted once, not yet safely restorable.

Next real target after serialization and cleanup hardening.

What I would try next

Next test	Reason	Pass condition
Retry MiMo `250K/C1` from a serialized clean state	It already booted once, but restore collided with a stray job after the 500K failure.	Boot, exact marker, tool call, DFlash counters, then restore again from scratch.
Try an intermediate `384K/C1` before 500K	500K died too early to diagnose. A smaller step may separate memory pressure from Ray/NCCL startup behavior.	Useful KV/concurrency logs plus a passed smoke, not just weight load.
Run long-context retrieval on any MiMo profile above 131K	Startup is not a context win. Retrieval is the real test.	Needle/retrieval style probes near the claimed window, with thinking off and traceable token counts.
Keep DS4-F enabled during MiMo experiments	The rollback path matters more than the experiment’s ambition.	Any MiMo attempt ends with either a live MiMo canary or restored DS4-F on `:8888`.

Next test

Reason

Pass condition

Retry MiMo 250K/C1 from a serialized clean state

It already booted once, but restore collided with a stray job after the 500K failure.

Boot, exact marker, tool call, DFlash counters, then restore again from scratch.

Try an intermediate 384K/C1 before 500K

500K died too early to diagnose. A smaller step may separate memory pressure from Ray/NCCL startup behavior.

Useful KV/concurrency logs plus a passed smoke, not just weight load.

Run long-context retrieval on any MiMo profile above 131K

Startup is not a context win. Retrieval is the real test.

Needle/retrieval style probes near the claimed window, with thinking off and traceable token counts.

Keep DS4-F enabled during MiMo experiments

The rollback path matters more than the experiment’s ambition.

Any MiMo attempt ends with either a live MiMo canary or restored DS4-F on :8888.

Provenance

Item	Value
MiMo working directory	`/home/milo/mimo-dflash-nvfp4kv-20260705` on Spark1/Spark2.
MiMo public repo tested	`tonyd2wild/MiMo-V2.5-DFlash-1M-ctx-NVFP4-KV-2x-DGX-Spark`.
MiMo target weights	`lukealonso/MiMo-V2.5-NVFP4`.
DFlash drafter	`XiaomiMiMo/MiMo-V2.5-DFlash`, only `dflash/*` downloaded.
Restored DS4-F checkout	`/home/milo/ds4-f-wesche-keys-c12`.
DS4-F restore status	`/v1/models` returned `max_model_len=350000`; exact smoke returned `DS4F_RESTORED_350K12_OK`; logs showed `GPU KV cache size: 955,499 tokens`, `Maximum concurrency for 350,000 tokens per request: 2.73x`, and `Application startup complete`.

Item

Value

MiMo working directory

/home/milo/mimo-dflash-nvfp4kv-20260705 on Spark1/Spark2.

MiMo public repo tested

tonyd2wild/MiMo-V2.5-DFlash-1M-ctx-NVFP4-KV-2x-DGX-Spark.

MiMo target weights

lukealonso/MiMo-V2.5-NVFP4.

DFlash drafter

XiaomiMiMo/MiMo-V2.5-DFlash, only dflash/* downloaded.

Restored DS4-F checkout

/home/milo/ds4-f-wesche-keys-c12.

DS4-F restore status

/v1/models returned max_model_len=350000; exact smoke returned DS4F_RESTORED_350K12_OK; logs showed GPU KV cache size: 955,499 tokens, Maximum concurrency for 350,000 tokens per request: 2.73x, and Application startup complete.