Fourth pass at oMLX PR #3059 on the M3 Ultra 512 GB. Round 3 found that head 304bb447's always-on down/wo_a offloads regressed this box from +7.1% to +4.6%. Fabian asked: run the PR's own DeepSeek ANE tuner and repeat. Done — full 11-candidate plan per checkpoint (model_family=deepseek_v4, sequence 4096, repeats=2, verify_full_model=true), then the standard 16k/32k harness re-run with exactly the settings the tuner recommended.
wo_a measured 0.0 ms GPU benefit and was disabled; down was kept but trimmed 0.65 → 0.50; CPU share eased to 10% / 8 threads — and its configuration posts the best numbers of the series: oQ2.5e +8.8% / +8.2% PP at 16k/32k, official fp8 +4.2% / +4.4%. The profile-refined stage also completed cleanly, without the fail-closed abort we hit in the Qwen-era M5 tuner runs.
651.4 tok/s. Series best (r2: +7.1%, r3 defaults: +4.6%).
621.2 tok/s. Series best (r2: +3.0%, r3 defaults: +0.9%).
wo_a off (0.0 ms benefit), down trimmed to 50%. Same conclusion, found automatically.
Identical recommendation on both checkpoints (independent runs):
| Setting | PR default (round 3) | Tuner recommendation |
|---|---|---|
wo_a offload | on @ 50% | off — measured 0.0 ms GPU-side benefit |
down offload | on @ 65% | on @ 50% (~3.3 ms/op measured benefit) |
| Query CPU share | 12.5% / 12 threads | 10% / 8 threads |
| Tail padding | 0 | 0 (unchanged) |
Tuner's own full-model verified claims: oQ2.5e 655.6 tok/s (+9.25%), fp8 643.1 (+4.82%) on its internal harness — both within ~1% of what our independent 16k benches then measured. full_model_verified: true, measurement_scope: full_model on both.
Same protocol as every round: fresh server, wiped SSD cache, code_python, TG=128, greedy, MTP off. Baselines are round 3's fresh GPU measurements (stable across all rounds ±0.5 tok/s).
| Checkpoint | Prompt | GPU PP | r2 subset | r3 defaults | r4 tuned |
|---|---|---|---|---|---|
| 0731-oQ2.5e | 16k | 598.9 | +7.1% | +4.6% | +8.8% (651.4) |
| 0731-oQ2.5e | 32k | 582.1 | +7.0% | +3.8% | +8.2% (630.1) |
| 0731 fp8 | 16k | 612.8 | +3.8% | +1.3% | +4.2% (638.6) |
| 0731 fp8 | 32k | 595.2 | +3.0% | +0.9% | +4.4% (621.2) |
So the round-3 story completes: the down offload was never the problem — at 50% it's worth roughly +1.5 points over the round-2 subset. The regression was wo_a (which the profiler had flagged: 33 ms input-ready stall for 12 ms of ANE work) plus a too-aggressive down fraction and CPU share. Remove the loser, trim the winner, and the stack beats everything measured before it.
verify_full_model).effective_step=4096, 10 full 4096-token chunks per bench, 172 (oQ2.5e) / 131 (fp8) procedures compiled with wo_a excluded.down_enabled, cpu_fraction…) while the settings API expects deepseek_ane_prefill_*-prefixed names. A consumer applying the recommendation verbatim silently changes nothing (the settings endpoint ignores unknown keys). An apply=true flag on the tune endpoint, or emitting API-ready keys, would close that gap — we hit it, benched a no-op config, caught it via the load log, and re-ran.onthehub97/omlx @ 304bb447. Production oMLX untouched; bench on :8022.POST /admin/api/bench/ane-tune/start, model_family=deepseek_v4, sequence_length=4096, repeats=2, verify_full_model=true, from PR-default settings. Full tuner JSONs (all candidate rows) and tuned-bench serve logs: pr3059-tuner-results-m3ultra-304bb447.tar.gz.Jundot/DeepSeek-V4-Flash-0731-oQ2.5e and official deepseek-ai/DeepSeek-V4-Flash-0731.Series: round 1 (fix discovery, final) · round 2 (fix upstream) · round 3 (offload regression, profiled) · this post. Sources: PR #3059.