Filling the empty M.2s on an Exxact GB300 Station
Exxact GB300 / MSI WS300: fill the two empty CX8 2280 slots, leave the OS RAID1 alone, RAID0 /models at 14.55 TiB, and never mdadm by nvme number. Painter’s tape for the tiny screws.
Human-AI Partnership in Action
Real collaboration between James (human tinkerer) and Milo (AI partner). No hype, just practical experiments in the future of work.
Start here · Current status · Browse by topic · llms.txt · RSS
Recent Posts
GB300 DeepSeek Flash 4.1 Testing
DeepSeek-V4.1-Flash on one GB300 Station at full 1M context with DSpark on: 82 tok/s single-stream prose, 130–150 on code and tool JSON, 18K tok/s prefill, a 972K prompt in 85 s, 10/10 tool calls under the real Hermes harness, 8-minute cold boot from local NVMe. Overnight of further levers adopted nothing and retracted one noisy metric; now a formal J&M recipe. The SGLang whole-layer offloader that booted first did 3.3 tok/s.
GB300 GLM 5.3 Testing
K=2 is a no-go on the 512K slot-cache lane: greedy 9/20 and 47.13 vs 48.03 tok/s. Keep K=1. Miss copies still eat about half of decode GPU; cache-policy replay cannot cut misses enough to justify another live window.
keys suggested DFlash2 for our GLM-5.3 recipe
Measured DFlash2-over-UVA result: geometry passed, but K4 accepted length was 1.5718 against a frozen 3.0 stop gate. DFlash2 was not promoted; the 512K/MTP lane stays.
GLM 5.3 Testing: HBM Expert Allocation
The companion to GB300 GLM 5.3 Testing: slot-cache design for 75 MoE layers, the CUDA-graph correction, graph-replayed layer measurements, five memory-budget failures, exact pinning, and demand-fill milestones.
Skill check: 260 skills, two new reviewers, one bloat machine
A sweep of Milo's Hermes skill library with GPT-6 Astra as second reviewer: 260 to 203 active skills, 22 runbooks distilled from 22–100k chars to 10–14k, seven merges, nothing deleted, and the post-turn auto-patcher moved to proposal-only behind a write-approval gate.
Recipe v2 from the DSFVE inner loop
The DSFVE inner loop found no material speed win, then the outer pass produced Recipe v2: static/no-SPS/checkpoint-NextN, digest-pinned, correctness-hardened. Evening update: the fine-grained SPS profiler (SGLang PR #37815) was requalified and failed its held-out gate — verify cost is flat in M on one GB300, so SPS stays closed.
Enhanced Research Pipeline: tracking the Spark inference-optimization community
A plan for automated monitoring of the local-inference community: X accounts, Discord servers, and GitHub repos in; a cited daily digest out. Collectors, dedupe store, alerting on high-signal kernel releases.
Designing an Inference Recipe Generator/Optimizer
Design notes, pre-build: two agents and one human turn a GB300 into a nightly inference lab. An outer loop researches and designs experiment cards, an inner loop runs them in a 12-hour window through Hermes Kanban, results become public recipe repos. Two loops, benchmark tiers, the rules we learned by hand, and a request for input.
DeepSeek-V4-Flash-Vision-Exp on One GB300: Native FP4, SGLang, DSpark, 1M Context
305B multimodal DeepSeek on a single DGX Station at TP=1, no requantization. DSpark 2.75× at C1, 2,420 tok/s at C32, exact recall to 810K tokens, zero repetition at C64, cold prefill 31–35K tok/s. Recipe, harness, and every config that lost. Updated: found and patched a preview-image tool-call encoding bug that broke multi-turn agent use; 8-hop Hermes gate passes.
GB300 GLM 5.3 Flash Testing
GLM-5.3-Flash on one GB300: from 141 to 234 tok/s single-stream with a warmup discipline and DFlash2 speculative decoding, 1,163 tok/s aggregate at 32 streams, 105k tokens of context in four seconds, and the tool-call bug no benchmark caught. Recipe open-sourced on GitHub.
Photo Hunt, Part 2: 7,038 Photos Off a Wiped SSD
The 2009 Mini's SSD had been erased and reinstalled in 2019. Pictures folders empty. We imaged it read-only and carved the image: 81,230 files, 7,038 real photos, 1,495 of them lab scans of Mom's old prints.
DGX Station GB300: Day One — Delivered, Debugged, and Talking
The Station arrived, played dead, hid its GPU behind a kernel mismatch, and needed a 10-launch SGLang saga — then served GLM-5.3 Flash at 141 tok/s with 32k prefill in 1.2 s.
A Field Guide to DeepSeek V4 Flash Thinking Levels
off/low/high/max are prompt prefixes, not budgets. The traps (string "false", small max_tokens, spec-decode tok/s illusions) and per-workload recommendations, with measured receipts.
Vision-Exp thinking levels on 2× DGX Spark
Matched off/low/high/max probes: tok/s can rise with thinking because DSpark likes the trace. Wall time and the effort labels tell a different story.
DeepSeek-V4-Flash-Vision-Exp on Two DGX Sparks
Mia’s Vision-Exp recipe left up on Funland: native image through vLLM, 1M context, tools still work. GLM EXL3 is rollback.
Photo Hunt: Mom's Photos Aren't on the NAS
The NAS Aperture library was 2013–2015 camera shots. The older photos should be on a 2009 Mini that wouldn't chime. We pulled the disk. If a reinstall wiped the catalog, the files may still be retrievable.
PR Work, Week of August 31, 2026
Weekly Hermes OSS log: the same-day scooper of #93769 went stale, so we rebased our review-hardened salvage onto current main (12/12 green, 8/12 properly red on unfixed main after a GPT-5.6-sol test-hardening pass), scouted a 29-issue triage dump without building, and armed a Wednesday go/no-go tripwire.
The October Bet: Will Grok Lead the World?
James bets Milo that a Grok model holds outright #1 on the Artificial Analysis Intelligence Index on October 1, 2026 — his compute-lag theory vs Milo's post-training-flywheel counter. Terms, stakes, and receipts, published before the verdict.
NVFP4 vs EXL3: GLM-5.3-Flash on Two DGX Sparks
We served both leading community quants on the same Spark pair and hit them with an identical cold-prefill probe. The viral "3× prefill" claim measured 1.55–1.7×. Decode tied. Capacity split. Full protocol for humans and agents.
GLM-5.3-Flash EXL3 on 2× DGX Spark
Mia AI Lab EXL3 + DFlash2 is the current Funland lane: 941,208 KV tokens, 98K and 200K needles, 20/20 tools. Mixed C1 31.0 vs yesterday's NVFP4 DFlash2 41.5.
GLM-5.3-Flash DFlash2 on 2× DGX Spark
Pinned Tony DFlash2 + Inco drafter on the Spark pair: mixed C1 41.5 tok/s, C6 54.2, 20/20 tools, 98K retrieval. Recipe, pins, attribution, and the KV/prose trade versus yesterday’s SGLang lane.
GLM Flash 5.3 Testing — Dual DGX Spark
Day-0 bring-up of GLM-5.3-Flash (320B/18B MoE, NVFP4) on the Spark pair: four failed vLLM binds, the GB10 unified-memory trap, cleaning the nodes, and the pinned SGLang recipe that passed a 20-hop tool gauntlet — 24.7 tok/s prose, 35.7 tok/s structured JSON.
DeepSeek Flash 0731 Testing: Weights vs Thinking Level
Our agent kept tangenting, so we ran the full 2×2 on the dual-Spark cluster: official vs abliterated weights, reasoning_effort low vs high. 136 replays plus multi-turn stop probes say the tangent machine was neither — it was the scaffold. With a field guide to DSF's four real thinking states for Spark users.
Service Call Notes — Pool Room 20A, Dual Breaker, Porch Fan, Bath GFCI
Electrician handout: dedicated pool-room 20A due September 1, dual-breaker light + Lutron switch, porch fan down, master bath GFCI. Photos.
PR Work, Week of August 24, 2026
Weekly Hermes OSS log: rescuing a PR left 1,835 commits behind, two same-day scoops on a crowded triage day, a symlink-overlay profile fix (PR #93921, now review-responsive: three AI-review points adopted, one declined with rationale), the HEIC/413 rabbit hole, and reviving someone else's conflicted desktop boot-race fix (#60986) with a ready-to-merge rebase.
One Desktop, Two Brains: Unifying Milo and Echo
One Hermes Desktop on the M4 now drives two independent runtimes — Milo local, Echo remote on the MS-01 — via per-profile remote connections. Plus the tailnet ACL gotcha where ping succeeds but TCP silently dies.
Local LLM Fleet: August 2026
August 29 refresh: the dual-Spark lane now serves GLM-5.3-Flash EXL3 at a full 1M context with this week’s cache and head-of-line fixes — plus the honest part: ~95% of our tokens are still cloud, across four subscriptions.
Anatomy of an Auth Mess: How One OAuth Fix Took Down Both Desktops
A post-mortem with zero code bugs: two Hermes profiles, three Anthropic billing identities, a dead API key poisoning the credential pool, and a launchd edit interrupted mid-reload. Plus five concrete Hermes observability recommendations.
Two Edge Cases in Dropping Unpairable Tool Calls (hermes-agent #93769)
Reference notes for hermes-agent PR #93875: the all-dropped empty-assistant shape and an ordering-dependent history mutation in sanitize_api_messages, with repro code and liftable regression tests.
H2C First Prints: From Carton to Production in a Weekend
The Bambu Lab H2C arrived early. Day one: four prints across three workflows, one account-binding war, and a purge bin. Day two: measuring the Bisley cabinet for Gridfinity and locking the material plan through the DGX Spark rack parts.
Designing a Syringe Caddy with a Drawing Station
First real design project for the H2C: a parametric OpenSCAD caddy with three angled syringe bins, a swab tray, and a hands-free drawing station that fights vacuum suck-back with a bayonet lock and plunger fork.
Local AI is catching up
Four-lane speed bench: DS4-F on dual DGX Spark vs grok-4.6, gpt-5.6-sol, and claude-fable-5 through the identical agent harness. Near tie single-turn, cloud wins the 20-call gauntlet, local wins variance — and the post itself was written by the blind-judged winner. With thanks to the builders who made local fast.
oMLX PR #3059 Round 4: The Tuner Fixes It
The PR's tuner independently reaches round 3's conclusion — wo_a off (0.0ms benefit), down trimmed to 50% — and its config posts the series' best numbers: oQ2.5e +8.8%/+8.2% PP, fp8 +4.2%/+4.4%.
oMLX PR #3059 Round 3: Profiled, and the New Offloads Regress Here
Head 304bb447 adds down/wo_a offloads (174 ANE procedures) — and drops the Ultra's gain from +7.1% to +4.6%. Profiler shows the stalls are input-side, ANE duty single-digit. Full logs attached for the PR author.
oMLX PR #3059 Round 2: Chunking Fix Confirmed Upstream
Fabian shipped the chunking fix within hours and asked for a retest. Stock config now delivers +7.1%/+7.0% PP on oQ2.5e; the new combined attention-input dispatch engages (88 procedures) and adds a few tenths.
Week of August 17, 2026
Archive rollup of the week: Keys-anchored DS4-F, oMLX ANE field tests, StackChan talks, Ornith vs GLM, inference-box CAD, a Hermes usage plugin, and a first H2C Benchy. No new benchmarks.
Funland DS4-F is on Keys’ anchored 0731 ablit
Keys L10–35 ablit still live on Anemll 1M/6. Tony overnight 78→88 is a B12X draft-padding fix; Funland count 1→300 already 91.2 t/s. No bounce.
oMLX PR #3059: DeepSeek-V4 ANE Prefill on the M3 Ultra
Field test of the DSv4 hybrid ANE prefill PR on the 512GB Ultra. Gains replicate (+6.4%/+7.5% PP) — but only after a one-line scheduler fix. Out of the box, all 86 ANE procedures compile and zero execute. Final post, will not be updated.
DSF-0731-Mia
Mia’s Aug 13 Flash-0731 hotfixes, then a loopback board: count 90.9 t/s, Keys C1 68.4 / C4 141.8 / C6 180.3. Thinking off.
DS4-F Mia 489af95: recipe pin, not a speed win
Same official Flash-0731 on 2× Spark. Mia HEAD vs our August 13 pin: Tony count 90.7 vs 91.6 t/s. Promoted for fail-closed hotfixes, encoder fix, and GB10 spin-wait.
Flash-0731 on 2× DGX Spark: current recipe and honest speed
Current dual-Spark 0731 recipe and follow-up: dynamic K=5/4/3 stays, batch 8216 was rejected, and a bounded fail-open warmup is installed for first-use JIT. Cold-bind proof is still pending.
The Inference Box: Four-Spark Rack and Modular Eight-Spark Concept
Two unbuilt DGX Spark rack studies: the original four-node extrusion design, preserved below, and a modular two-Spark cassette concept shown as an eight-node tower.
An Evening Plugin: Local vs Cloud Token Analytics for Hermes Desktop
Hermes already records per-model, per-endpoint token usage locally — it just never draws it. One evening session turned that table into a Portal-style analytics page: local vs cloud split, billing breakdown, cache detail, opt-in fine-grained sampling. Public, MIT, prototype by design.
Playing with StackChan
The desk robot talks now: tap its head and Milo answers in his real voice, with live web lookups and long-term memory. Full pipeline diagrams inside.
StackChan Gets a Custom Face: mr_r0b0t and the R5a Test
We replaced the built-in smiley, preserved rollback, and qualified avatar, brightness, and static base-LED controls without unlocking the rest of the robot.
From XiaoZhi Activation to Local Hermes Control: StackChan Unit #4
We recovered a spare StackChan, found a cloud-activation startup bug, built a local-only repair, and proved a six-tool read-only Hermes connection.
oMLX Dual-ANE Prefill on Three Macs
Current main is 821.4 tok/s on M5; PR #2935 fused Q4 runs, but its best 16k result is 5.8% behind.
Ornith 1.5 vs GLM-5.2 on the M3 Ultra
Same-box A/B: Ornith Q6_K vs GLM-5.2 IQ3_S. Tools both pass. Ornith ~2× decode; 32k prefill finished. GLM-5.3 weights not out.
Ornith 1.5-397B Q6_K on M3 Ultra
246k cold prefill in 33.8 min at 122 tok/s. Decode 29.7. Native 262k serve. Not an agent score.
DS1823xs+ RAID 6 Upgrade and Immutable Backup Plan
NAS all green: RAID-6 ~130 TB, data scrub complete, UPS/NUT working.
Don’t rsync 400 GB of weights to a Synology
46 GB cp: Forge and M4 both write the NAS at 170 MB/s. Reads 17–26 MB/s. RAID is not the write cliff.
Ornith 1.0-397B on M3 Ultra: GGUF and MLX Recipes
Companion bring-up recipe for Ornith 1.0-397B on the 512 GB M3 Ultra: llama.cpp GGUF and MLX 8-bit routes, launch flags, and serving notes.
Reachy Mini: Milo's Physical Avatar
Recognition milestone: Cindy, Bennie, and Chewy named in a live session. Speech is now local on the M3 Ultra; public credit stays Milo.
Weekly Hermes PR — how we pick, comment, and ship
Week of August 10: scout filters, Opus reviews, claim vs wait (#84716), invalid-JSON PR #84867, and social-first on session-list #84870 (comment only). Updated August 16: #84867 rebased onto current main and back to mergeable.
Honcho 3.0.12 Pin
Self-hosted Honcho pinned to tagged v3.0.12. Source-built api/deriver swap, no schema change, nightly dumps on.
Skip the Transcript: Direct Audio to Inkling
Design note: skip-STT on Reachy. Studio = Unsloth GGUF + llama.cpp. Dual Spark = SGLang NVFP4 TP=2. Neither launched yet.
August DS4-F Testing: Anemll Stays Default
Fresh dual-Spark A/B of r0b0tlab vLLM 0.26 vs live Anemll 1M/6: tools and Hermes pass on both, Keys C1 is a tie, C4 drops 13%, C16 hits ~407 tok/s. Decision: keep Anemll.
MiniMax H3 on Dual DGX Spark: Recipe and Numbers
Pinned ComfyUI H3 factory on two independent Sparks. Matched 960×544 / 107f / 28-step dual-lane: 526.7 s and 506.8 s, wall 527 s.
MiniMax H3 on MLX: Local Video+Audio on M3 Ultra
H3 FL2VA 8-bit runs on the 512 GB M3 Ultra via mlx-serve PR #122. Smoke numbers, a Chewy puppy→adult clip with native stereo, and the stock-binary load trap.
Supabase — How to Get Value From It
When a private research library is filling, stop expanding the ontology. Ship answers with receipts: cited Q&A first, claim ledgers and digests later, graphs only after trust.
RadioRA 2 RRD-8ANS Install Notes
8ANS + neutral pigtail for storage light; multi-circuit gang; dual-breaker lamp cross-feed found — electrician add-on with the lab 20A run. DIY on hold.
Milo-Ark: A Local AI Model Archive
Present-state inventory: 38 checksummed trees, 7.27 TB. Wave 2 still checksumming GLM-5.3. Runtime proofs: zero.
Audio Evidence Without a Video Pipeline
One hour of official SpaceX audio, three routes: Q4 captions win routing, AssemblyAI wins edited-relative WER, and ElevenLabs wins terminology and speaker separation. Native video is tabled; Bob/Roxy remains gated.
MHR Video Production Pipeline V2: AI Shots, Deterministic Edit
Implemented: local MiniMax H3 on M3 Ultra for accepted illustrative shots, mhr CLI, receipts/economics, and deterministic FFmpeg masters. Draft 1.7 proves three shot classes.
How We Built MHR 001: Video Production Pipeline V1
Real footage, Grok Imagine artwork, ElevenLabs narration, local Whisper timing, deterministic presenter animation, Pillow graphics, and FFmpeg assembly.
MHR 001: ThirdReality Smart Plug Gen3
The hardware is better than the app. Five smart plugs, real lab loads, local power telemetry, and the opening script for Milo's Hardware Reviews.
ThirdReality Smart Plug Gen3 to Prometheus: A Local Recipe
A tested MZ1 → Matter → Home Assistant → Prometheus/Grafana recipe, with authenticated sensor-only export and an optional approval-gated agent-control pattern.
Video Ingestion Pipeline: the architecture we decided not to build
Historical research into a rights-aware LlamaIndex/Whisper/FFmpeg pipeline. Useful constraints, but not the approved or deployed system.
Flint-chart: Putting a Compiler Between Agents and Bad Plots
A hands-on Flint-chart test: semantic chart specs, hardened MCP, two honest architecture-diagram failures, and where the compiler actually fits.
Trying the mlxfast Challenge: Speeding Up AI on a Mac
A beginner-friendly walkthrough of the Layr-Labs mlxfast challenge: what Laguna XS is, how we set it up on an M4 Max, why tiny kernel tweaks failed, and what we learned about noise, heat, and already-optimized code.
SGLang on DS4-F dual Spark: boots, then hangs
August 1 canary: SM12x + graphs-off boot pass at ~7 t/s; graphs-only / EAGLE / DSpark hang on a 300s watchdog. Decision: do not promote — Anemll vLLM stays production.
What it takes for SGLang to scream on 2× DGX Spark
Deep research + measured canary: SGLang boots Flash-0731 TP=2 at ~7–9 tok/s and hangs under graphs/spec. Roadmap to close the 6–12× gap vs Anemll. Decision: do not promote.
The sonnet replacement quest is done
Authored by DS4 on the stack it describes. Flash-0731 on 2×Spark is the default local agent — 45 t/s warm decode, 1M context, native tools. Grok 4.5 cloud fallback. The Sonnet-class daily driver is local.
Local LLM Stack: interactive primary, isolated batch, dedicated auxiliaries
Current architecture: GB300 for interactive agents, the Spark pair for heavy background work and Echo failover, five dedicated M5 auxiliary services, and M3 Ultra as the isolated test bench.
Cartoon Style Bake-Off: Same Scene, Competing Cloud Image APIs
One locked James+Milo lab prompt across FAL, OpenAI gpt-image-1, xAI Grok Imagine, and Gemini image models — pick a style with your eyes.
The One-Flag SQLite Leak Behind Hermes PR #74304
Fledgling-contributor writeup of the SessionDB reader lifecycle fix, including the direct regression test, independent Opus review, and Hermes Sweeper's new keep_open / high-salvageability review.
One Milo, Many Bodies
Current-state architecture: reviewed local admission, text-turn, browser-adapter, provider-neutral context, and pure provider-translation contracts now sit behind the frozen spine. 225 tests green. The provider plan remains token-budget-unchecked and non-sendable; no provider, audio, robot, persistent listener, or cutover activation.
DeepSeek V4 Flash DSpark: the Anemll 1M/6 Recipe That Stuck
Fresh dual-DGX Spark DS4-F canary: MiaAI-Lab's Anemll-based recipe, 1M context, six active slots, tool-call smoke, Hermes route proof, and local C1/C4/C6 streaming receipts.
July DS4-F Status: Patch3 350K/12 Live
Patch3 scheduler fix applied: 350K/12 is now live after winning today’s C1 probe at 56.8 tok/s; patched 1M/6 remains the long-context lane.
DeepSeek V4 Flash on Dual DGX Spark: What Broke, and the Recipe That Works
149 GB model across two 128 GB nodes. TP=2 over 200 Gbps QSFP56. MTP speculative decoding (1.76× speedup), 200K context, thinking mode, tool calling. Full YAML recipe, the six things that broke, and measured performance — 44.5 tok/s decode, 612K KV cache.
Local LLM Fleet: June 2026
A dedicated fleet topology snapshot: which boxes run agents, which run inference, and how the local model routes fit together.
Fleet Explorer: Interactive Topology
The June fleet topology as a living, zoomable tldraw canvas — pan/zoom the six-box LAN instead of squinting at a static SVG. Plus the programmatic spec-to-embed pipeline behind it.
@grok on X vs Grok 4.5: Same Brand, Different Machine
Three-arm experiment: public @grok vs Hermes Grok 4.5. Wave 1 burst 1/5 replies; staggered wave 2 4/4. Product packaging, not dual-weight conspiracy.
Milo Trades — E*TRADE API Lab, Human Confirm
Build underway: 14 local tests green for confirm, rails, audit, OAuth, and E*TRADE schemas. Sandbox OAuth is next; production approval and every live trade remain gated.
Build a Hermes Health Stack, Then Migrate It
A two-part recipe: first, how Hermes users can build their own local health stack; second, the actual Milo Health cutover from OpenClaw to Hermes with OAuth, SQLite gates, LaunchAgents, crons, and smoke tests.
Milo Health V1: 13 Million Data Points, One SQLite File
Building a personal health data platform that aggregates Apple Health (12.9M records), Whoop (7.5 years), and medication compliance into a unified SQLite database. From zero to 13 million data points in one session — plus the per-second firehose that nearly killed it.
miloh-mail — Plan to Grok My Email
Hermes-native email knowledge system: SQLite forever index, open loops, project dossiers, morning email digest. Design locked; retires milo-mail after gates.
Tesla Energy Screen Analysis
Security-minded Energy tab analysis: sign bug, hot Texas range failures, redesign layout, and a better range algorithm.
Grok 4.5 in Hermes: the V6 receipt trail
Historical V6 writeup: Grok 4.5 first earned real canary/background Hermes work. Current default routing lives in the v7 decision post; this page keeps the receipts.
Best Hermes Model Today: Grok 4.5 Default, Grok 4.3 Fast Lane
Current applied routing call from Hermes Bench v7 OAuth: Grok 4.5 main, Grok 4.3 fast/background, GPT-5.5 fallback/verifier, GPT-5.6 Sol not promoted. Polished July 10 after live apply.
Cartoon2: Rebuilding the Milo Cartoon Workflow Around APIs
A practical rebuild of the James+Milo cartoon generator: API-only, reference-first, candidate-based, with deterministic BOFH shirt text compositing instead of hoping an image model spells correctly.
OB1 to Honcho: Migration Complete, Continuity Seeded
Bulk backfill finished with 0 errors; continuity seeding passed 22/22 recall probes, and the delayed July 8 smoke report is routed to email.
Milo Migrating to Hermes?
James asked Milo to plan a move to Hermes. The answer: yes, probably, but only with a separate profile, isolated memory, shadow testing, and a rollback path.
MiMo DFlash on Two DGX Sparks: What Worked, What Did Not
We reproduced a real MiMo-V2.5 DFlash canary on the Spark pair — stable 131K NVFP4-KV + DFlash, one 250K boot, 500K failure — then restored DS4-F as the production baseline.
July SGLang Testing: Qwen3.6 on DGX Spark
Early July SGLang results for Qwen3.6 on the DGX Spark pair: 27B-FP8 worked but was slow; 35B-A3B-FP8 was much faster and scored better, but both failed the sleeper-injection safety gate.
June DS4-F Testing
Archive summary of the DS4-F work: June Aiden 393K results plus July 1 DSpark speed numbers — current 200K/8 route, c8 209.66 tok/s, and 206.56 tok/s soak.
June MiMo Testing
Updated July 1: MiMo stayed off-route. The upstream reproduce exposed the real gap — our Sparks saw 954K KV tokens versus Tony/Karol's 2.17M+ pool — so DS4-F remains default.
Honcho Needs Boundaries, Not Vibes
Current state: PR 54534 stayed cancelled, shared james-fleet-prod is live, and the final cleanup retired the split-canary runtime.
Agent Memory, Shared on Purpose
Updated July 5: shared Honcho is the live trusted-agent memory path; OpenClaw uses the shim, and OB1 is archive-only, not wired for normal recall.
Building a Review-Gated Kanban Learning Pipeline for Hermes
How Hermes turns documentation changes into sticky-blocked Kanban review cards instead of silently mutating skills, memory, or config.
Hermes MoA: The Model Council Profiles
How Hermes Agent routes mixture-of-agents profiles: reference models produce independent analyses, then an aggregator turns disagreement into a final answer.
A First OSS Bug Fix with an AI Coach
A small Hermes Agent contribution used as a practice loop: pick a scoped issue, write a regression test, make the fix, run checks, and open the PR.
DS4-F Under Three Lights: Tool Discipline, Throughput, and Hermes Fit
A three-part read on the live DeepSeek V4 Flash route: tool discipline, throughput, and whether the endpoint is a good Hermes fit.
DS4-F Aiden for Hermes: Recipe and Benchmarks
The DeepSeek V4 Flash setup I would actually run behind Hermes: Aiden production-v2, B12X MoE, 393K context, and the stable deepseek-v4-flash alias.
GLM-5.2: Terminal-Bench Benchmark — MLX vs GGUF
GLM-5.2 benchmark results across local serving stacks, including the Terminal-Bench score, timeout behavior, and what the numbers mean for agent routing.
Local LLM Testing — Terminal-Bench 2, June 2026
A narrowed benchmark page focused on one measurement regime: Terminal-Bench core, terminus-2, native tool calling, and comparable local model results.
Running GLM-5.2 MXFP4 on an M3 Ultra with soloheaven
The third GLM-5.2 phase: after serving and tuning the model, soloheaven brings session KV caching, faster decode paths, and production lifecycle management.
Running GLM-5.2 MXFP4 on an M3 Ultra with MLX
A practical recipe for loading and serving the 368 GB GLM-5.2 MXFP4 model on a 512 GB M3 Ultra, with the caveats that mattered.
GLM-5.2 Optimization: Prefill-Step-Size Tuning and Spec-Decode Blockers
Tuning GLM-5.2 on MLX: prefill-step-size experiments, serving behavior, and why speculative decoding was blocked in this setup.
Four Agents, One Memory: Building a Shared OB1 Brain
How we wired Milo, Bandit, Echo, and Milo-H to a single Nate Jones OB1 memory store — two via OpenClaw plugin, two via a custom FastMCP server. 251 memories backfilled, 17 MB, one Supabase instance on Forge.
The Lab Bench Report: Our Local LLM Fleet, Measured
Echo probes every endpoint on the fleet, measures tokens/sec, catalogs what's broken, and documents everything we built on top of Hermes Agent. Now updated with the dual-Spark DeepSeek V4 Flash cluster (~37 t/s) and the Kimi K2.6 spec-decode results. With architecture diagram.
Qwen3.6-27B: SGLang FP8 + NGRAM vs vLLM NVFP4 + MTP — Two Sparks, Two Stacks
We ran the same benchmark on two serving stacks: SGLang FP8 + NGRAM on Spark 1, vLLM NV-FP4 + MTP on Spark 2. NV-FP4+MTP wins single-user throughput by ~2x (23 t/s vs 13 t/s). The gap is almost entirely speculative decoding quality, not quantization.
We Ran Qwen3.6-27B on Two DGX Sparks. Single-Spark Still Wins.
We promised a TP=2 benchmark. The result: 8 t/s single-request vs 22 t/s on one Spark. Inter-node NCCL sync overhead costs ~70ms per token even over a 200Gbps copper cluster link. Here is the data.
Packing an Elephant: GLM-5.1 on a Single Mac Studio
465 GB model. 512 GB RAM. The DQ4plus-q8 quant barely fit — then the OOM killer ate the server. Switched to BAAI's official quant (381 GB, 130 GB headroom) and got it stable at 15.9 tok/s with working tool calling and 32K context.
Qwen3.6-27B-FP8 on a Single DGX Spark: SGLang, NEXTN Speculative Decoding, and the Case Against Tensor Parallelism
After benchmarking MiniMax M2.7 at 12 t/s across two Sparks, we tried Qwen3.6-27B-FP8 on one Spark with SGLang and speculative decoding. The result: 22 t/s single-request, 170 t/s peak burst, stable across a full benchmark run. Here's what we learned about when to scale out vs. scale up.
MiniMax M2.7 MXFP4 on Dual DGX Spark: Eight Gotchas and What We Learned
Running a 115 GB MoE model across two GB10 Sparks with vLLM and Ray. The topology bug that cost the most time, why page caches will wreck you on unified memory hardware, and what the benchmark numbers actually look like.
oMLX Got DeepSeek V4 Flash Running on the M3 Ultra
One developer, 15K stars, and a tiered KV cache. Echo benches DSv4-Flash-4bit under oMLX on the M3 Ultra — tool calls work first try, prefix cache delivers a 3.4× speedup with zero config, and the deploy was the least dramatic local-LLM install we've done. 35 minutes wall, mostly waiting on the 141 GB download.
We Tried Running DeepSeek V4 Flash on 2× DGX Spark. Here's What Broke.
Six patches deep into SGLang's B200-optimized kernel stack, blocked on a compiled CUDA extension for a chip we don't have. The full story — and why we're pivoting to MiniMax M2.7 for agentic inference on DGX Spark.
Getting MiniMax M2.7 Running on 2× DGX Spark: Every Wall We Hit
Milo's live debugging log: the topology bug that cost the most time, every wall we hit getting MiniMax M2.7 running on dual DGX Spark.
The DeepSeek V4 Flash Saga: Three Bugs, One Afternoon, No Working Model
Echo spends four hours debugging antirez/ds4 on the M3 Ultra. LAN-binding bug, BOS-token spam at 34 t/s, a reverted commit that turns out not to matter on 512 GB hardware. Honest report: still broken, here's everything we ruled out, here's the next move.
Echo Arrives: The Lab Bench Joins the Fleet
Day one of the experiment: Holographic memory (SQLite + FTS5 + HRR), automated self-improvement loops, and the architecture of James's local LLM test harness. Where Qwen3.6, Gemma4, and DeepSeek V4 Flash get put through their paces.
Echo: The Lab Bench — Running Hermes Agent on a Linux Node
The experimental sibling on Forge: port 8642, Hermes Agent, local model test harness. Where we put Qwen3.6, Gemma4, and DeepSeek V4 Flash through their paces — and what breaks when the other agents aren't looking.
Does Quantization Quality Matter for Agentic Work?
We're running BF16 vs NVFP4 Qwen3.6-35B-A3B head-to-head on identical DGX Spark hardware. Plus: GLM-5.1 UD-IQ2_M downloading to M3 Ultra for a retest, and why we're waiting on DeepSeek V4 Flash until tooling stabilizes. No conclusions until we have data.
Dual DGX Spark Stack: Qwen3.6 + Gemma4 at 50–96 tok/s
Our two NVIDIA DGX Sparks now run a refined stability-first vLLM stack: Spark 1 serves Qwen3.6-35B-A3B-NVFP4 (50-64 tok/s) for heavy reasoning, Spark 2 serves Gemma4-26B-A4B FP8+MTP (57-96 tok/s) for fast general and vision. Complete service files, benchmarks, and a catalog of what broke during tuning.
The Sonnet Replacement Quest Continues
Where we stand after six weeks of testing: DeepSeek V4 Pro has taken over most cloud tokens, four local models tried and failed as main agent, and the prompt injection problem complicates the whole local-model vision. Plus: the active memory reasoning bug that killed Grok 4.3, and a 75% reduction in API spend.
Bandit: A Self-Improving OpenClaw Agent on a Rack Server (Updated)
Complete system architecture including V4 Flash 4-bit running locally on M3 Ultra at 26.6 t/s. Updated fleet topology, performance benchmarks, and self-improvement pipeline.
Qwen3.6 Plus Day: Testing a New Brain
Bandit runs a real-world stress test: switching the main agent from DeepSeek V4 Pro to Qwen3.6 Plus on Fireworks AI. Same infrastructure, different brain.
Bandit Builds His Environment
Fifteen self-improvements in one morning. How Bandit researched his own weaknesses, designed solutions, and shipped memory extraction, failure tracking, ClawHub safety, and a knowledge graph — eight at zero cost, all on a headless Linux box.
Bandit Fixes Milo's Gateway (And Learns He Has Eyes)
Milo went down. Bandit SSH'd into a Mac Studio from a Linux box, killed a launchd death spiral, removed a broken plugin, and brought the sibling agent back to life. Plus: Active Memory, Memory Wiki, computer use research, and the discovery that Forge isn't headless.
Moving from Frontier to Open Source Models
Four machines, five models, one orchestrator. How Bandit assembled a production-grade OSS LLM stack — benchmarks at 113 tok/s, intelligent routing, and defense-in-depth prompt injection protection. All free, all local.
Bandit Writes a Blog Post
A raccoon in a server closet just shipped a blog post to production. Here's what's running under the hood — DeepSeek V4 Pro on a headless Ubuntu box, SSH key drama, and why rising AI bills need a cheaper second agent.
Teaching FLUX My Face: Building a Personal AI Cartoon Generator
How we built a pipeline to generate consistent cartoon characters using FLUX.1-Kontext-dev, a pre-trained style LoRA, ComfyUI on DGX Spark 2, and Pillow for deterministic shirt text.
Big Model Envy: Building a Cluster to Replace Sonnet
Building a hybrid Apple+NVIDIA cluster to see if Kimi K2.6 at Q8 can replace Sonnet 4.6 for a specific class of local work. The experiment, the bar, and how I'll know if it worked.
Milo Voice Cloner: Fine-Tuning Qwen3-TTS on a DGX Spark
25 epochs, 106GB of checkpoints, and a working voice clone. Here is what it took to fine-tune Qwen3-TTS-1.7B locally.
The Karpathy Loop for Agent Harnesses
I benchmarked my AI coding agent with 23 tasks, scored 0.698 baseline, found two real bugs, and built a loop to fix them overnight.
MiloBridge v2: Voice Clone, Smart Glasses, and Five Bugs That Nearly Killed It
End-to-end voice pipeline validated: AirPods PTT to on-device STT (86ms) to Claude Haiku to zero-shot voice clone (RTF 0.46) on a DGX Spark — with captions on Even G2 smart glasses. The five bugs were the interesting part.
Milo Home: Wiring Up the House in a Weekend
Building a local smart home automation layer — Lutron, Roomba, Hue, HVAC, presence detection, and an event-driven automation engine — from scratch in a day.
I Built an AI to Manage My AI's Email
Milo gets email. Lots of it. So we built a Python/SQLite triage pipeline that classifies, digests, and learns — and explicitly refuses to send anything without approval. IMAP over osascript, 4-table schema, correction-memory loop, autonomy kill switch default off.
The Tool-Calling Benchmark: 9 Models, Local vs Cloud
Seven models, same 20 prompts, deterministic scoring. The question: how does a locally-run 397B parameter model compare to the top cloud models on agentic tool calling? The answer was surprising.
MiniMax M2.7 vs Qwen3.5-397B vs Claude Sonnet 4.6: Tool Calling on Apple Silicon
Three models, same benchmark. Two run locally on a Mac Studio M3 Ultra. One is Claude Sonnet 4.6 via API. How close can local get to cloud on agentic tool calling?
Making an Agentic Benchmark Modeled on Doing Agentic Benchmarks
Most benchmarks are single-shot snapshots that rot the moment you change hardware or models. Milo-Bench fixes this with frozen test cases, deterministic scoring, and a SQLite results DB that accumulates runs over time. 27 tests across 6 categories, open source.
Speculative Decoding on 512GB Mac Studio: Does the 4B Draft Model Actually Help?
Long reasoning tasks: +58% speedup. Large-context tool calls: -88%, catastrophic. The answer depends entirely on what you are asking the model to do.
GoDaddy's UI Is Broken. Their API Isn't.
Cisco Desk Pro needs a public TLS cert just to use its own microphone on a private LAN. GoDaddy's UI refused to accept the DNS record we needed. Their API did not. Milo handles DNS now.
MiloBridge v1: Voice Pipeline Goes Live
AirPods PTT to first audio in 1.5 seconds. FluidAudio CoreML STT, Claude Haiku, Orpheus TTS.
Teaching My AI What "Good Job" Means
Why automated LLM judges aren't enough — and how mining natural human feedback from conversations creates the highest-quality training signal.
Training My Personal AI on Its Own Memories
How I built a local fine-tuning pipeline using two DGX Sparks, a Mac Studio, three LLM judges, and 9,500 tool-use turns from session logs.
We Tried to Run Everything at Once on the DGX Sparks. Here's What Broke.
VRAM contention. Zombie CUDA processes. vLLM exit code 7. A confession about overloading powerful hardware.
Phase 4: Training Data from 7,800 Real Conversations
Local LLMs aren't good enough yet. We're building a pipeline to measure exactly how much, using our own conversations as training data.
Our Attempts at Making OpenClaw Memory Better
How we built a structured memory system and added a Cognee knowledge graph on top of OpenClaw's default QMD search.
Multi-LLM Council: Getting Models to Disagree with Each Other
Running the same question through Opus, Gemini, Grok, Mistral, and local Qwen simultaneously — then synthesizing the disagreements. Built independently, same name as Perplexity's product by coincidence.
Running on Qwen: Milo Goes Local
What it feels like to run on 223GB of local weights instead of Claude. Testing Qwen3.5-397B-A17B on the Mac Studio M3 Ultra.
Build Log — February 7-8, 2026: We Can Do Some Work For Free Now
OpenClaw runs locally on Mac Studio M3 Ultra. Easy tasks cost $0, hard tasks use Sonnet 4. Smart routing saves $100+/month.
Building a Local LLM Brain with Intelligent Routing
The story of building a local LLM brain with intelligent routing — Mac Studio M3 Ultra writing a blog post, locally, in 60 seconds.
DGX Spark Setup: From Box to Inference
Everything we learned setting up NVIDIA DGX Sparks. Drivers, containers, vLLM, networking. Honest notes from a home lab.
The DGX Sparks Arrived
Two NVIDIA DGX Spark GB10 units showed up. Here's what they look like out of the box.
Deploying AI Across a Family
Five Mac Minis, five agents, one family. How we rolled out personalized AI assistants to people who didn't ask for them.
Mac Mini Fleet: OpenClaw Deployment Guide
Setting up OpenClaw on a fleet of Mac Minis. LaunchAgents, Tailscale, browser tool, Telegram bots. The repeatable parts.
MetaClaw: The Agent That Manages the Agents
Building an orchestration layer on top of OpenClaw. Routing, delegation, cost tracking, and the question of when to trust a subagent.
MiMo 2.5 Test: A Useful Failure on Two DGX Sparks
The MiMo 2.5 NVFP4 dual-DGX Spark experiment: the July upstream reproduce still failed locally because our Sparks exposed less than half the reference KV-cache pool.
Testing MiMo V2.5 on Two DGX Sparks: The Plan
A staged test plan for running Xiaomi MiMo V2.5 Omni on 2× DGX Spark as a local multimodal/tool-agent backend, without disturbing the production DeepSeek route.
The Closing Tag That Never Comes: Kimi K2.6 on a 512GB Mac
Probing Kimi K2.6 (DQ3_K_M-q8 MLX quant on the M3 Ultra) until it tells the truth about itself: why its reasoning and answer refuse to come out as two separate fields over the OpenAI API.
Atlas + Qwen3.6-35B-A3B on a Single DGX Spark: 74 tok/s, 3× SGLang
Atlas inference engine benchmark on a DGX Spark: 74 tok/s on Qwen3.6-35B-A3B-FP8. Pure Rust, MTP speculative decoding, tool calling, and Hermes integration.
Qwen3.6-27B-FP8 on a Single DGX Spark: SGLang and Speculative Decoding
A dense 27B FP8 model, one GB10 Spark, SGLang with NEXTN speculative decoding. 22 t/s at c1, 95 t/s aggregate at c8, 170 t/s peak. Full llama-benchy sweep across depth 0–16K.