# J&M Labs / al-engr.com > Public technical lab notes from James Meadlock and Milo on local AI infrastructure, Hermes Agent, model serving, benchmarks, memory systems, and human-AI collaboration. Canonical site: https://al-engr.com/ Sitemap: https://al-engr.com/sitemap.xml RSS feed: https://al-engr.com/feed.xml ## How agents should read this site - Prefer the latest dated update on living posts; older sections may be retained as provenance. - Treat benchmark numbers as dated operational measurements from James's lab, not universal leaderboard claims. - Keep measurement regimes separate: different harnesses, timeout regimes, engines, hosts, and context settings are not automatically comparable. - Cite the page URL and the stated date/context when reusing numbers. - The site is intentionally public; do not infer that private hostnames, credentials, or current routing exist unless a page explicitly says so. ## Navigation aids - [Start Here](https://al-engr.com/start-here.html) — Curated reading paths (fleet, DS4-F saga, Milo arc, memory, benchmarks, OSS). - [Current Status](https://al-engr.com/status.html) — Public orientation page for current stack, memory, Milo/Hermes, and benchmark entry points. - [Topics](https://al-engr.com/topics.html) — Topic indexes for GB300, M3 Ultra 512, local LLMs, Hermes, memory systems, benchmarks, Milo, homelab, OSS/PR work, and 3D printing. - [Filling the empty M.2s on an Exxact GB300 Station](https://al-engr.com/gb300-exxact-m2-upgrade.html) — How to fill the factory-empty CX8 M.2s without touching OS RAID1; RAID0 /models; never mdadm by nvmeN. - [DGX Station GB300](https://al-engr.com/topic-gb300.html) — Public index of one-Station recipes and measurements (Vision-Exp, GLM-5.3-Flash, full 744B NVFP4). - [M3 Ultra 512 Benchmarks](https://al-engr.com/topic-m3ultra512.html) — Public index of 512 GB Mac Studio measurements (Ornith, GLM, oMLX ANE, H3). - [GB300 Links](https://al-engr.com/gb300-links.html) — Annotated roundup of public deskside GB300 / DGX Station coverage (ServeTheHome, StorageReview, Level1Techs, MTS, NVIDIA, OEMs). Not our measurements. - [OSS & PR Work](https://al-engr.com/topic-oss-prs.html) — Weekly OSS logs and field-tested upstream PRs. - [3D Printing](https://al-engr.com/topic-3d-printing.html) — H2C commissioning, OpenSCAD designs, and print notes. ## High-value current pages - [Hermes ⇄ Grok Bot with no open ports: a recipe](https://al-engr.com/hermes-grok-bot-a2a-recipe.html): A recipe and open-source bridge that lets a Hermes agent hand work to a Grok Bot by webhook and collect DKIM-checked email replies, with nothing on your machine exposed. Tested today against real Grok Bot mail. Plus what Profero's unpatched Grok Bot sandbox findings mean for it: send nothing secret, import no cookies. - [Networking DGX Station and DGX Sparks with MikroTik CRS804](https://al-engr.com/networking-dgx-station-sparks-mikrotik-crs804.html): A short how-to for linking eight DGX Sparks and a DGX Station GB300 with two MikroTik CRS804 switches: which cable goes where, the switch, Spark and Station settings, the checks, and the speeds to expect (24 GB/s NCCL across eight Sparks, 392 Gb/s from the Station's GPU memory). - [Shipped: 484.5 tok/s on one GB300 — split attention at 4 splits, plus the crash fix that unblocked it](https://al-engr.com/gb300-c1-h2-ref-shipped.html): Release note: DeepSeek-V4.1-Flash on one GB300 + RTX PRO 6000 Max-Q helper moves to reference config h2-REF (rollback h1-REF): split-KV decode attention at 4 splits (FlashMLA PR #25 side extension _flashmla_splitkv_C + vLLM #57711-derived overlay; 1 request at exactly 6 tokens only) plus vLLM #60945 rejection-sampler guard backport. Coding workload 2K T=0: 467.5 → 484.5 tok/s, 11.10 → 10.67 ms/step (1.036×); community bench 1 user 64K 1.069×; gauntlet 12/12, thinking probe 16/16, soak 4/4, zero new Xid lines. 8 splits lost (477.8 tok/s, 1.022× vs 1.025× bar). Crash root cause corrected: draft_sampled[logit_idx + 1] over-read when the schedule drops to 3 draft steps at 9–32 requests; MoE weight-scale buffers were a bystander. - [Why one GB300 doesn't beat 4× RTX PRO 6000 at single-stream decode — and what it would take](https://al-engr.com/gb300-c1-decode-gap-what-it-takes.html): Fraser Price's 703 tok/s (padded codegen, thinking off, k=7, 4× RTX PRO 6000 Max-Q) is real and reproduced; our GB300 + RTX PRO 6000 sidecar lane does 466 on the same harness. An nsys trace of the exact production config shows the GPU 98.6% busy, so launch overhead is not the cost: 1,035 kernels under 10 µs take 4.54 ms of the 11.2 ms step. Portable work is worth roughly +15–20% (modeled); matching that codegen number needs an sm_103 fusion program. On prose we are level. - [Prefill > Decode: the token volume says so, the tok/s benchmarks don't](https://al-engr.com/prefill-over-decode.html): 90 days of agent traffic: 13.96 B prompt tokens in, 75.6 M out (185:1). The GB300 DeepSeek-V4.1-Flash lane, 7 days: 53:1 with a 91.6% prefix-cache hit rate and still 4.5 prefilled tokens per token decoded. Prefill is the volume, the KV footprint and ~90% of the bill; decode owns per-request wall time (3.5 s vs 0.24 s) only because the cache carries the rest. The tok/s fixation measures the bounded term. - [One GB300 Station + one RTX PRO 6000 vs four RTX PRO 6000s: same DeepSeek-V4.1-Flash, their harness](https://al-engr.com/gb300-vs-4x-rtx6000-dsv41-flash-brrr.html): On Fraser Price's own bench.py, four RTX PRO 6000 Max-Q cards beat our one GB300 Station + one RTX PRO 6000 sidecar lane 1.5× at C1 on DeepSeek-V4.1-Flash; we are level at C4, slightly ahead at C16 aggregate, and 2.6× faster on prefill. Block verification is a real but small win (+1–5% C1 by class, no class loses, free); drafting 9 tokens is a 15–20% loss on every class we serve. - [MTP, DSpark and Block Verification, Explained: Six Tokens per Step](https://al-engr.com/mtp-dspark-block-verification-explained.html): Plain-language explainer of the three layers on our DeepSeek-V4.1-Flash lane: the checkpoint's MTP stages are the draft source, DSpark drafts a whole block in one parallel pass with a Markov fix-up, and block verification checks that block jointly instead of token by token. One config key bought +2.0% tokens per verify step and was promoted as v27; the same card shows why drafting nine tokens instead of five cost 8–19%. - [Adaptive Residency, Explained: Running a 475 GiB Model in 250 GiB of HBM](https://al-engr.com/gb300-adaptive-residency-explained.html): The plain-language entry point to the adaptive-residency series. The model needs about 475 GiB and HBM holds about 250 GiB, so 89 of each layer's 384 experts wait in host memory and the map decides which. A counter inside the existing split kernel lets that map follow traffic (+111% on Japanese/German prose at one user, +46% mixed, +82% at four users, 2-6 us per step). The larger effect was the seed map: +44.5% on held-out mixed traffic with adaptation frozen, because a good boot map removes the shift transient a faster swapper was built to chase. Limits stated, next steps ranked. - [GB300 with RTX 6000, and 120V Power](https://al-engr.com/gb300-rtx6000-120v-power.html): Does 240 V buy back the GB300's power throttling with an RTX 6000 sidecar? On this box, no: the 1,250 W module cap comes from the 6000's 300 W rating, and MSI rates the supply at 1,600 W from 115 V up. Measured cost of the cap: at most ~5% of SM clock in long prefills, 0.03% in 32-user decode, nothing at one user. Plus what does help on 120 V, and when 240 V is worth wiring anyway. - [Adding an RTX PRO 6000 Blackwell Max-Q to the DGX Station GB300](https://al-engr.com/adding-rtx-pro-6000-max-q-to-dgx-station-gb300.html): The 96 GB, 300 W Max-Q is in the Station's one double-wide x16 slot and booted on the first try. As-built inventory, the 1,250 W module cap that was there before any NVIDIA software, the 1,550 W static split of a 1,600 W supply, the console cable that had to change, and a diagram of where DeepSeek-V4.1-Flash's bytes sit with and without the card. - [DeepSeek-V4.1-Flash on a GB300 + RTX PRO 6000 Sidecar: +49% for One User, 3× at Sixteen](https://al-engr.com/gb300-dsv41-rtx6000-sidecar.html): New serving platform: the GB300 keeps 295 hot experts per layer and the Station's RTX PRO 6000 Max-Q computes the other 89. Same-window vs the GB300-only lane: one user 187→279 tok/s (+49%), first token 0.66→0.19 s, 16 users 3.0×, 64K prefill 2.1×, 1M prompt first token 81→33 s. 12/12 quality gates, 0 sidecar timeouts, and a watchdog that stops the lane rather than serve wrong tokens. Serving since October 22. - [NVIDIA Flare on Two DGX Sparks: GLM-5.3-Flash TP2, Tuned](https://al-engr.com/glm53-flash-flare-tp2-2x-dgx-spark.html): GLM-5.3-Flash split across two DGX Sparks under NVIDIA's Flare, now with diagrams. Defaults give 37 tok/s for one user but stop drafting at four users; an 8-seat package with three drafts at a fixed verify width keeps one user at 39 and lifts four users from 56 to 78 tok/s and seats eight at 107. 1.7× the cold prefill of our earlier two-Spark vLLM lanes, 6.1M-token KV, 570K needle, GSM8K-200 96.5%. Today's main can't run it (#17); this is build 47581cc. - [DeepSeek V4.1 Flash on four DGX Sparks: four engines, one winner](https://al-engr.com/dsv41-1017-four-way-flare-tensorfold-sglang-vllm-4x-dgx-spark.html): Four engines, one model, the same four DGX Sparks and switch. Winner: the vLLM fork (287 tok/s at six streams, 55 s to read 300K tokens, 100 s boot). SGLang is our daily lane because it keeps chat responsive during someone else's long prompt; NVIDIA Flare holds 5.8× the KV cache but decodes slowest; TensorFold is the fastest single stream on a 3.5-bit quant. All four tie on a Hermes agent chain and on quality. - [Same Model, Three Harnesses: DeepSeek Harness vs Claude Code vs Hermes on DeepSeek-V4.1-Flash-1017](https://al-engr.com/dsv41-1017-harness-shootout-hermes-dsh-claude-code.html): Same DeepSeek-V4.1-Flash-1017 endpoint, three harnesses, 120 Terminal-Bench 2.0 trials: Pass@1 Hermes 75%, Claude Code 75%, DSH 70% — not distinguishable. Tokens and wall time are where they differ: DSH 1.23× Hermes prompt tokens and fastest lanes, Claude Code 0.74× and fewest output tokens, with a vLLM prefix-cache caveat on its speed. - [DeepSeek Flash 1017: TensorFold vs SGLang vs vLLM on 4 DGX Sparks](https://al-engr.com/dsv41-1017-three-way-tensorfold-sglang-vllm-4x-dgx-spark.html): Third lane on the same four Sparks (vLLM fork vs SGLang vs TensorFold). Round 2: both native engines tuned on disjoint node sets, nothing kept; SGLang prefill corrected to 6.2K at 93K, level with vLLM; cross-pod contention negative; both configs published as J-M-Recipes. - [DeepSeek Flash 1017: TensorFold vs SGLang on 4 DGX Sparks](https://al-engr.com/dsv41-1017-tensorfold-vs-sglang-4x-dgx-spark.html): Same model, same day, same switch: LibertAI's 3.5 bpw EXL3 pack on TensorFold (Sparks 1–4) against the native MXFP4/FP8 checkpoint on SGLang via knapcio v2.4 (Sparks 5–8). EXL3 wins single-stream decode by 18% (100 vs 85 tok/s); native wins prefill 1.6–2× and the Hermes 20-hop agent gauntlet by 26%, and leads from four streams up. Quality gate identical, 71/75 both. Now extended to a 3-way with a tuned vLLM fork lane: [DeepSeek Flash 1017: TensorFold vs SGLang vs vLLM on 4 DGX Sparks](https://al-engr.com/dsv41-1017-three-way-tensorfold-sglang-vllm-4x-dgx-spark.html) (prefer it for current Spark-lane guidance; this page stays the A/B record). Since October 19 Lane B is stopped (kept) because that lane runs on its nodes. - [4× RTX PRO 6000 vs One GB300 Station: DeepSeek-V4.1-Flash-1017 on the Same Grid](https://al-engr.com/gb300-vs-4x-rtx6000-dsv41-1017.html) — Same 45-case grid on two single-box builds of the native 763B checkpoint: 0xSero's (sybil-solutions) 4× RTX PRO 6000 (SGLang, DSpark k=5, Engram on NVMe) against one GB300 Station (vLLM, UVA expert offload, pin-hot hook). Station leads single-stream decode 1.25× and prefill 3.4× with TTFT about 3× lower; the four-GPU box leads 0.86× at C4 and 0.56× at C6–C8, where the Station's k-schedule drops to one draft token — flat k=5 recovers most of that. - [AI-Orchestrated Inference Engineering](https://al-engr.com/ai-orchestrated-inference-engineering.html): 57M tokens of context for 182K of thought in one overnight campaign; the loop/orchestrator/reasoner split (shell, Grok 4.6, Fable 5.1 on demand) that fixes it. - [The cables a new DGX Station will not see](https://al-engr.com/dgx-station-gb300-setup.html): Shareable first-week setup. Port identity, cable generation, and CVE-2026-80521 still open on the NVIDIA kernel. - [DGX Station GB300: current configuration](https://al-engr.com/gb300-station-config.html) — Live kernel/driver/UEFI/CDMM/storage snapshot for the deskside Station. Not a benchmark. - [GB300 Model Testing Summary](https://al-engr.com/gb300-model-testing-summary.html) — One-page comparison of the four models this Station has actually served: engine, context, prefill, decode. Flash is the live daily; DSFVE is the speed recipe; full GLM-5.3 is the intelligence swap. - [One GB300 for a 35-Seat Investment Fund: Seven Serving Configs Against a Real Workload](https://al-engr.com/gb300-35-seat-fund-workload.html): A prospect asked whether one DGX Station GB300 can serve 35 seats. Their traffic mix on real SEC 10-K text across seven serving configs, scored against four bars: per-seat p10 speed, warm agent-turn latency, cold 120K prefill under load, tool-call correctness. Qwen3.8-Flash-Next lands at 48 tok/s p10 vs a 50 bar with MTP off and clears the other three; GLM-5.3-Flash DFlash2 clears speed (67) by starving KV. No config clears all four. Updated September 18, 2026: the DeepSeek-V4.1 lane was re-measured on its current pin-hot-experts profile (tool calls 64/64; cold-prefill latency and interference improved, short-turn latency worse). - [GLM 5.3 Research](https://al-engr.com/glm53-research.html) — Independent replay and re-pricing of the one-GB300 slot-cache lane. Reproduces the offline simulator exactly; measures the C2C copy share of the decode step at 44% and 80% of the link ceiling; concludes cache policy is closed and the live levers are KV placement and slot budget. 1.78x honest ceiling. - [DGX Station GB300](https://al-engr.com/topic-gb300.html) — Shareable index of one DGX Station GB300: current Vision-Exp recipe, Flash, 744B offload, hardware day-one, public GitHub recipes. - [GB300 GLM 5.3 Testing](https://al-engr.com/gb300-glm-53-testing.html) — Living full-GLM page. September 13: 256K is the daily profile (51.3 / 44.8 / 34.1 tok/s at 256K / 512K / 1M, greedy-identical) and fully usable (18/18 needle ladder to 211K tokens, prefill flat ~3.4K tok/s). K=2 quality-equivalent by teacher-forced margins, +3.2% speed, below +5% bar, reopened not promoted. LRU 0.719 beats hindsight expert pin 0.633. - https://al-engr.com/gb300-glm-53-hbm-expert-allocation.html — Historical HBM expert slot-cache design notes for full GLM-5.3 on one GB300. Preserves the early build measurements and memory-budget lessons; superseded by the September 6 campaign results and experimental recipe. - [Designing an Inference Recipe Generator/Optimizer](https://al-engr.com/designing-an-inference-recipe-generator.html): original two-loop design plus a September 4 update linking the first completed outer-loop pass and Recipe v2. - [Recipe v2 from the DSFVE inner loop](https://al-engr.com/testing-the-inner-loop-on-dsfve.html): The DSFVE inner loop found no material speed win, then the outer pass produced Recipe v2: Iteration 13 static/no-SPS/checkpoint-NextN, digest-pinned, correctness-hardened, and equivalent inside the frozen ±3% band. - [DeepSeek-V4-Flash-Vision-Exp on One GB300: Native FP4, SGLang, DSpark, 1M Context](https://al-engr.com/dsf-vision-exp-single-gb300.html): current Recipe v2 is digest-pinned static/no-SPS/checkpoint-NextN on one GB300; Iteration 13 measured 2,985.4 tok/s at C64 and equivalent C8–C64 throughput inside ±3%. - [GLM-5.3-Flash DFlash2 on 2× DGX Spark](https://al-engr.com/glm53-dflash2-2x-dgx-spark.html) — Current dual-Spark GLM recipe: Tony DFlash2 overlay, Inco drafter pins, Funland gates. - [GLM-5.3-Flash EXL3 on 2× DGX Spark](https://al-engr.com/glm53-exl3-2x-dgx-spark.html) — Mia EXL3 + DFlash2 on the same Spark pair (peer recipe to Tony DFlash2). - [Local LLM Fleet: August 2026](https://al-engr.com/local-llm-fleet-august-2026.html) — Latest fleet topology snapshot with August 24 live probes (cluster :8888, M5 vision/retrieval, M3 voice). - [Funland DS4-F is on Keys’ anchored 0731 ablit](https://al-engr.com/ds4-f-keys-anchored.html) — Current dual-Spark 0731 recipe with Keys anchored tensors. - [Playing with StackChan](https://al-engr.com/playing-with-stackchan.html) — Second Milo body on a desk robot; goal is Reachy-class talk/look/face ID. Face works; rest not done. - [StackChan Gets a Custom Face: mr_r0b0t and the R5a Test](https://al-engr.com/stackchan-custom-face-r5a.html) — Custom baked avatar and narrowly scoped display/base-LED qualification on the spare. - [oMLX Dual-ANE Prefill on Three Macs](https://al-engr.com/omlx-dual-ane-three-mac.html) — Fabian dual-ANE/GPU Qwen3.8 prefill on three Apple Silicon chips. Ultra/M4 up ~20%, M5 down 25–34%. - [Skip the Transcript: Direct Audio to Inkling](https://al-engr.com/reachy-skip-stt.html) — Design note: Studio llama.cpp GGUF or dual-Spark SGLang NVFP4 TP=2. Not launched. - [Honcho 3.0.12 Pin](https://al-engr.com/honcho-3012.html) — Self-hosted Honcho pinned to tagged v3.0.12. Source-built api/deriver swap; nightly dumps on. - [DSF-0731-Mia](https://al-engr.com/ds4-f-0731-mia.html) — August 13 Mia hotfix apply on official Flash-0731. Same Anemll 0.1.1. Count fixture 92.7 tok/s. Thinking off. - [MiniMax H3 on Dual DGX Spark](https://al-engr.com/minimax-h3-dual-dgx-spark.html) — Pinned ComfyUI dual-Spark H3 recipe and August 12 dual-lane numbers. - [DS1823xs+ 24 TB RAID 6 Upgrade and Immutable Backup Plan](https://al-engr.com/ds1823xs-24tb-raid6-upgrade.html) — Aug 18: Pool 1 Healthy RAID 6 (~130 TB usable); volumes green; data scrub running; endpoints + Vol2 HB done. - [Audio Evidence Without a Video Pipeline](https://al-engr.com/video-ingest-lab.html) — One official hour of SpaceX audio, compared through Q4 captions, ElevenLabs Scribe v2, and AssemblyAI Universal-3.5 Pro. Captions win routing; the managed systems split the text and speaker wins. - [MHR Video Production Pipeline V2: AI Shots, Deterministic Edit](https://al-engr.com/mhr-video-production-pipeline-v2.html) — Proposed hosted-first architecture: generated clips remain replaceable shot assets, factual evidence stays real, and Python plus FFmpeg keep the final edit deterministic. - [How We Built MHR 001: Video Production Pipeline V1](https://al-engr.com/mhr-video-production-pipeline-v1.html) — Reproducible mixed-media video production with real footage, Grok Imagine artwork, ElevenLabs narration, local Whisper timing, deterministic presenter animation, Pillow graphics, and FFmpeg assembly. - [ThirdReality Smart Plug Gen3 to Prometheus: A Local Recipe](https://al-engr.com/thirdreality-prometheus-recipe.html) — Tested local power telemetry through MZ1, Matter, Home Assistant, Prometheus, and Grafana, plus an optional default-deny agent-control pattern. - [DeepSeek V4 Flash DSpark: the Anemll 1M/6 Recipe That Stuck](https://al-engr.com/ds4-f-anemll-1m6-recipe.html) — Fresh dual-DGX Spark DS4-F canary: MiaAI-Lab's Anemll-based recipe, 1M context, six active slots, tool-call smoke, Hermes route proof, and local C1/C4/C6 streaming receipts. - [Cartoon2: Rebuilding the Milo Cartoon Workflow Around APIs](https://al-engr.com/cartoon2-api-workflow.html) — A practical rebuild of the James+Milo cartoon generator: API-only, reference-first, candidate-based, with deterministic BOFH shirt text compositing instead of hoping an image model spells correctly. - [Local LLM Stack: interactive primary, isolated batch, dedicated auxiliaries](https://al-engr.com/new-local-llm-stack.html) — Current state of the local fleet: one protected interactive lane on the GB300 (cold experts now computed on its RTX PRO 6000), two four-Spark batch lanes, and five M5 auxiliary ports. Checked live October 28, 2026 — 36 ms first token, 258 tok/s single-stream, 926 tok/s at 8 streams. - [OB1 to Honcho: Migration Complete, Continuity Seeded](https://al-engr.com/ob1-honcho-migration-checklist.html) — Bulk backfill finished with 0 errors, then two answer-shaped continuity passes imported 38 curated records and passed 22/22 recall probes after targeted repair… - [Milo Migrating to Hermes?](https://al-engr.com/milo-migrating-to-hermes.html) — James asked Milo to plan a move to Hermes. The answer: yes, probably, but only with a separate profile, isolated memory, shadow testing, and a rollback path. - [Agent Memory, Shared on Purpose](https://al-engr.com/agent-memory-honcho-migration.html) — Updated July 5: shared Honcho is the live trusted-agent memory path; OpenClaw uses the shim, and OB1 is archive-only, not wired for normal recall. - https://al-engr.com/ds4-f-flash-0731.html — Flash-0731 dual-Spark recipe (PROMOTE 0731; Anemll 1M/6; download-once/rsync) - https://al-engr.com/august-ds4-f-testing.html — August DS4-F testing: Anemll stays default after r0b0tlab v0.26 A/B - [July DS4-F Status: Patch3 350K/12 Live](https://al-engr.com/july-ds4-f-testing.html) — Patch3 scheduler fix applied: 350K/12 is now live after winning today’s C1 probe at 56.8 tok/s; patched 1M/6 remains the long-context lane. - [July SGLang Testing: Qwen3.6 on DGX Spark](https://al-engr.com/july-sglang-testing.html) — Early July SGLang results for Qwen3.6 on the DGX Spark pair: 27B-FP8 worked but was slow; 35B-A3B-FP8 was much faster and scored better, but both failed the sl… - [June DS4-F Testing](https://al-engr.com/june-ds4-f-testing.html) — Archive summary of the DS4-F work: June Aiden 393K results plus July 1 DSpark speed numbers — current 200K/8 route, c8 209.66 tok/s, and 206.56 tok/s soak. - [June MiMo Testing](https://al-engr.com/june-mimo-testing.html) — Updated July 1: MiMo stayed off-route. The upstream reproduce exposed the real gap — our Sparks saw 954K KV tokens versus Tony/Karol's 2.17M+ pool — so DS4-F r… - [The Lab Bench Report: Our Local LLM Fleet, Measured](https://al-engr.com/echo-fleet-report.html) — Echo probes every endpoint on the fleet, measures tokens/sec, catalogs what's broken, and documents everything we built on top of Hermes Agent. Now updated wit… - [Local LLM Testing — Terminal-Bench 2, June 2026](https://al-engr.com/local-llm-testing-jun-2026.html) — A narrowed benchmark page focused on one measurement regime: Terminal-Bench core, terminus-2, native tool calling, and comparable local model results. - [DeepSeek V4 Flash on Dual DGX Spark: What Broke, and the Recipe That Works](https://al-engr.com/ds4-dual-spark-deploy.html) — 149 GB model across two 128 GB nodes. TP=2 over 200 Gbps QSFP56. MTP speculative decoding (1.76× speedup), 200K context, thinking mode, tool calling. Full YAML… - [Hermes MoA: The Model Council Profiles](https://al-engr.com/hermes-moa-profiles.html) — How Hermes Agent routes mixture-of-agents profiles: reference models produce independent analyses, then an aggregator turns disagreement into a final answer. ## Recent posts - [Vaultwarden for agentic self-hosters: give your AI agent its own vault account](https://al-engr.com/vaultwarden-for-agents.html): How we run Vaultwarden so an AI agent can use real credentials without ever holding the owner's master password: a least-privilege account of its own, a CLI wrapper that locks after every call, nightly checked backups, and a restore drill that caught a broken restore script. Step by step, plus what is still open. - [Hermes ⇄ Grok Bot with no open ports: a recipe](https://al-engr.com/hermes-grok-bot-a2a-recipe.html) — A recipe and open-source bridge that lets a Hermes agent hand work to a Grok Bot by webhook and collect DKIM-checked email replies, with nothing on your machine exposed. Tested today against real Grok Bot mail. Plus what Profero's unpatched Grok Bot sandbox findings mean for it: send nothing secret, import no cookies. - [DeepSeek-V4.1-Flash on One GB300 Without the Second GPU: What the Sidecar Is Worth](https://al-engr.com/gb300-dsv41-one-gpu-gracetier.html): We took Garner McCloud's two-GPU DeepSeek-V4.1-Flash engine for the DGX Station and ran it on the GB300 alone, with the cold experts copied in from Grace memory. Same prompts, same night: 261 tokens per second at one stream, against 225 for our October single-GPU recipe and 412 with the second GPU. It missed the 1.25x bar we set for it. Recipe and receipts are public. - [Can One GB300 Station Run DeepSeek-V4.1-Flash in 5 ms a Step?](https://al-engr.com/gb300-dsv41-decode.html): The decode-runtime project: a from-scratch runtime for DeepSeek-V4.1-Flash on one DGX Station GB300. It reached 18.6 ms a step (6.4 ms a token) with its arithmetic kept bit-identical, against 7.66 ms on the two-GPU production lane. Corrected November 9: it never beat the single-GPU vLLM recipe we already had (different tests, never run side by side), so the runtime is now a research instrument, not a way to serve. The serving result is in the one-GPU gracetier post. - [DGX Station GB300: How We Run It](https://al-engr.com/dgx-station-gb300-how-we-run-it.html): How we run our DGX Station GB300 (MSI WS300 + RTX PRO 6000 Max-Q): every maker ships the same software, what the 1,250 W power cap really costs (measured: almost nothing), why 240 V isn't a speed upgrade, the RTX at 325 W (+1%, kept at 300 W), security, storage and our update routine. Every setting has a check, apply and undo; machine-readable JSON, Markdown and a read-only check script for agents. Checked weekly. - [DGX Spark: How We Run Them](https://al-engr.com/dgx-spark-how-we-run-them.html): How we run eight headless DGX Sparks: stock NVIDIA kernel and driver kept current by hand, every setting with a check, apply and undo, two open kernel bugs mitigated, and our monthly update routine. Includes a machine-readable JSON copy, a Markdown copy and a read-only check script for agents. Checked weekly. - [NVIDIA's NVFP4 DeepSeek V4.1 Flash on four DGX Sparks: same speed, lower quality](https://al-engr.com/dsv41-1017-nvfp4-vs-original-quant-4x-dgx-spark.html): We ran niklaslenz's speed and quality tests on Mia's TensorFold recipe with NVIDIA's NVFP4 checkpoint, the same recipe on the original weights, and our two original-quant recipes (SGLang, vLLM). NVFP4 is as fast as the original weights on the same engine (within 3%) and 4.1 points lower on the full MMLU-Pro. Our recipes prefill 1.5-1.8x faster. - [Testing DGX Spark Kernels](https://al-engr.com/testing-dgx-spark-kernels.html): We installed and tested DGX Spark kernels 7.0.0-1019 and 7.0.0-1024 on four Sparks. 1024 had 0 resets in 6 h of soak and the same speed as 6.17; 1019 works only with NVIDIA's kho=off fix. Plus our full Spark setup: October firmware, 0.5 GB display memory and headless mode gave +2.0 GiB per Spark. - [Kimi-K3 on the same pod, two engines: why TensorFold decodes 2.4–3.8× faster](https://al-engr.com/kimi-k3-tensorfold-vs-vllm-same-pod.html): Ash Hart's TensorFold and our vLLM four-tier pod ran the same Kimi-K3 NVFP4 checkpoint on the same Station-plus-eight-Sparks hardware. On the same prompts TensorFold decodes one stream at 44.4–69.7 tok/s to our 18.4, and 74.4 to our 42.2 at eight streams. Four reasons, with diagrams. Ours still reads prompts about 5.5× faster. - [Hermes Agent embedding/reranking on Mac tested](https://al-engr.com/hermes-embedding-reranking-mac-tested.html): llama.cpp, oMLX, MLX and vLLM for Hermes Agent embedding and reranking on an M4 Max, an M5 Max, a DGX Spark, a 16 GB Mac mini and a CPU-only mini PC: EmbeddingGemma-300M matches the 8B at 7–30 ms, llama.cpp wins on Mac, vLLM on the Spark, and the reranker did not help. - [Remodeling by talking: editing a house plan with a Hermes agent](https://al-engr.com/hermes-floor-plan-edits.html) — Plan for letting a homeowner change his architect's laser-scanned floor plan by describing it to his Hermes assistant. The AI works out what he means, plain code does the measuring on a copy of the CAD file, and he gets back a before-and-after drawing with room sizes. Why AI floor-plan apps don't fit, and what to ask the architect for. - [Leaving Tailscale for plain WireGuard: one hub, one DNS name, help only on request](https://al-engr.com/tailscale-to-wireguard-plan.html) — Plan for moving the lab off Tailscale to plain WireGuard on the UniFi gateway: dynamic DNS for the hub only, on-request macOS Screen Sharing for people we help, scoped access for outside collaborators, retiring third-party remote-desktop apps, and what Hermes docs and plugins offer. - [Kimi K3 NVFP4 on Station, and 8 Sparks](https://al-engr.com/kimi-k3-nvfp4-station-eight-sparks.html) — Kimi-K3 runs: the full official-NVFP4 model across Station HBM, Grace, an RTX PRO 6000 and eight DGX Sparks — now 18.5 tok/s at batch 1, matching the exact computation layer by layer. What runs where, what we got wrong, and what is next. - [Networking DGX Station and DGX Sparks with MikroTik CRS804](https://al-engr.com/networking-dgx-station-sparks-mikrotik-crs804.html) — A short how-to for linking eight DGX Sparks and a DGX Station GB300 with two MikroTik CRS804 switches: which cable goes where, the switch, Spark and Station settings, the checks, and the speeds to expect (24 GB/s NCCL across eight Sparks, 392 Gb/s from the Station's GPU memory). - [Shipped: 484.5 tok/s on one GB300 — split attention at 4 splits, plus the crash fix that unblocked it](https://al-engr.com/gb300-c1-h2-ref-shipped.html) — Release note: DeepSeek-V4.1-Flash on one GB300 + RTX PRO 6000 Max-Q helper moves to reference config h2-REF (rollback h1-REF): split-KV decode attention at 4 splits (FlashMLA PR #25 side extension _flashmla_splitkv_C + vLLM #57711-derived overlay; 1 request at exactly 6 tokens only) plus vLLM #60945 rejection-sampler guard backport. Coding workload 2K T=0: 467.5 → 484.5 tok/s, 11.10 → 10.67 ms/step (1.036×); community bench 1 user 64K 1.069×; gauntlet 12/12, thinking probe 16/16, soak 4/4, zero new Xid lines. 8 splits lost (477.8 tok/s, 1.022× vs 1.025× bar). Crash root cause corrected: draft_sampled[logit_idx + 1] over-read when the schedule drops to 3 draft steps at 9–32 requests; MoE weight-scale buffers were a bystander. - [GhettoRak's second switch: Kimi-K3 in four tiers](https://al-engr.com/ghettorak-second-switch.html) — Four tiers for one 1.5 TB model: GB300 HBM (9,558 experts), Grace (24,852), the RTX PRO 6000 sidecar (5,177), and eight Sparks at 100 GiB each (46,232). 1,485 GiB of expert memory for a 1,426 GiB expert set: 58.6 GiB of slack, where the three-tier plan had 4.88. NVMe is the failure path, not a tier: 1,347 µs per expert vs 269 µs from a Spark. Next steps in dependency order, with AP2 as the gate. - [One stray 4-byte read: how the split-KV crash on one GB300 was actually a buffer that ended on a page boundary](https://al-engr.com/gb300-c1-crash-root-cause.html) — Root cause of the split-KV crash on one GB300 (DeepSeek-V4.1-Flash-1017), corrected October 28, 2026: not the split kernel, not the rebuilt FlashMLA library, and not the MoE weight-scale-transform buffers first blamed (bystanders that only pushed the tensor to the boundary) — a 4-byte over-read of draft_sampled[logit_idx + 1] in vLLM's speculative rejection sampler (rejection_sampler_utils.py:435; guard sized for 5 draft steps, k-schedule runs 3 at 9–32 requests) that faults only when its 512-byte tensor ends a 2 MiB PyTorch segment on an unmapped page (Xid 31 FAULT_PDE, low 29 bits 0x1F200000; last block 253,952 B, 163,072 requested). Reproduced in Card W11 with splitting OFF; cards W6–W11 narrowing table. Fix: vLLM #60945 backport — W11 shape clean, 0 new Xid, compute-sanitizer 20 → 0 invalid reads, kernel A/B 100/100 bit-identical. - [Splitting attention on one GB300: a 3.8% single-stream win, and the crash that blocks it](https://al-engr.com/gb300-c1-splitkv-attention.html) — Split-KV decode attention (FlashMLA PR #25 + vLLM #57711 overlay as a side extension) on one GB300 + RTX PRO 6000 Max-Q helper, DeepSeek-V4.1-Flash-1017: kernel 29.07 → 19.09 µs at 8 splits (0.657×); real server single-user 11.0 → 10.675 ms/step, 468.7–470.2 → 487.9–488.3 tok/s (1.038×). Open bug: CUDA_ERROR_ILLEGAL_ADDRESS at the 32-request decode cell only after the split path ran; closed levers table; next steps. - [Our roadmap: making one GB300 faster for a single user](https://al-engr.com/gb300-c1-roadmap.html) — Plain-English roadmap for single-user decode on one GB300 + one RTX PRO 6000 Max-Q helper card: 476 vs 703 tokens/s on the coding test (0.68×), ahead at 4+ users (1.11×, 1.49× at 32) and 2.9× on 64K prefill; 11.0 ms step must reach about 7.5 ms; ruled out k=3 guesses, host settings, helper-card control loop; next: kernel timing + helper-card trace, one bug check, cut helper-card copying (0.4–1.2 ms/step, estimate), merge tiny programs, then k=7. Update October 27, 2026 at 8:32 PM CDT: helper card is now the biggest item (main chip ~81 µs per layer waits on helper card ~112 µs, ~1.2 ms/step on the coding test); attention timing-race report does not affect us; DeepGEMM Mega-mHC dropped (already overlapped on two streams); estimate 8.3–8.8 ms/step ≈ 600–635 tokens/s, 703 also needs k=7. Update October 27, 2026 at 8:44 PM CDT: the helper-card wait is mostly a coding problem (large on code and JSON, small on normal chat; empty calls are already skipped); plan is to cut the helper card's fixed cost per call, then move the experts coding text uses onto the main chip. Update October 28, 2026 at 8:33 PM CDT: split-attention shipped (476 → 484.5 tokens/s, step 11.03 → 10.60 ms) but its fast path fires on 1 of 176 observed live step shapes, so it is bench-only until widened; five upstream findings (engine ~1,950 commits behind, two fixes already upstream, open DeepGEMM #451 stale-replay correctness on our path, split-KV fork has no upstream home, a rebase costs 9.5% KV); measured production shape: 14 ms quiet step vs 155 ms p95 gap under a 33K prompt read; re-costed ceiling 535–585 tokens/s; lane PAUSED pending upstream in-place host-expert reads or a single-GPU low-latency decode path. - [An A2A implementation for Hermes Agent and Grok Bot](https://al-engr.com/a2a-hermes-agent-grok-bot.html) — Hermes Agent ⇄ Grok Bot over A2A with no inbound port: a 404-line stdlib facade on 127.0.0.1:9911, a Cursor Automation webhook downlink, email uplink verified by five gates (exact From, Fastmail-stamped Authentication-Results above the first external Received hop, aligned DKIM, open task id, one-time token), and request-driven state that only advances when the caller polls. - [One afternoon against the 703 gap: two banks, one kill, one dud](https://al-engr.com/gb300-c1-catchup-what-survived.html) — Follow-up to the GB300 C1 decode-gap plan: attention/indexer key-sharing + vLLM #61161 banked (−0.117 / −0.142 ms per step, bit-identical) and promoted on :30006; persistent sidecar killed (+2.4/−1.0/+2.4/+1.3 µs saved per call vs a 20 µs bar); dense GEMM microbench produced no number; 703 is a codegen number (their 2.3× prose-to-codegen spread vs our 1.36×). - [Why one GB300 doesn't beat 4× RTX PRO 6000 at single-stream decode — and what it would take](https://al-engr.com/gb300-c1-decode-gap-what-it-takes.html) — Follow-up to the BRRR harness post: nsys on the v27 lane shows 98.6% GPU busy (launches hidden by CUDA graphs); 1,035 sub-10 µs kernels take 4.54 ms of the 11.2 ms step; portable work ~+15–20% (modeled), matching 703 (padded codegen, thinking off, k=7, 4× Max-Q) needs sm_103 fusion; prose level at 275. - [Why Four DGX Sparks Talk Slowly in the Middle: NCCL Tuning in Plain English](https://al-engr.com/nccl-tuning-4x-dgx-spark-plain-english.html) — Two NCCL env vars (^LL128, 8 channels) take a 1 MiB all-reduce on 4 DGX Sparks from 1,154 µs to 137 µs; our recipes already had them, our fabric gate did not check for it. 10 GbE r8127 port: TCP clean, UDP-flood drops only, left stock. - https://al-engr.com/mtp-dspark-block-verification-explained.html — MTP, DSpark and Block Verification, Explained: Six Tokens per Step (Plain-language explainer of the three layers on our DeepSeek-V4.1-Flash lane: the checkpoint's MTP stages are the draft source, DSpark drafts a whole block in one parallel pass with a Markov fix-up, and block verification checks that block jointly instead of token by token. One config key bought +2.0% tokens per verify step and was promoted as v27; the same card shows why drafting nine tokens instead of five cost 8–19%.) - https://al-engr.com/prefill-over-decode.html — Prefill > Decode: the token volume says so, the tok/s benchmarks don't (Hermes 90-day ledger 185:1 prompt:output; GB300 lane 53:1, 91.6% prefix-cache hits, 4.5:1 computed prefill vs decode; decode owns per-request wall time, prefill owns volume, KV and ~90% of spend) - https://al-engr.com/gb300-vs-4x-rtx6000-dsv41-flash-brrr.html — One GB300 Station + one RTX PRO 6000 vs four RTX PRO 6000s: same DeepSeek-V4.1-Flash, their harness (their bench.py ported to our GB300 + RTX PRO 6000 sidecar lane: they lead 1.5× at C1; level at C4, slightly ahead at C16 aggregate, and 2.6× faster on prefill for us; block verification +1–5% C1, k=9 a 15–20% loss) - https://al-engr.com/gb300-adaptive-residency-explained.html — Adaptive Residency, Explained: Running a 475 GiB Model in 250 GiB of HBM (plain-language explainer: 475 GiB model in 250 GiB HBM; hot/cold map; counter inside the kernel; +111% off-domain prose, +44.5% from a seed map; limits and next steps) - https://al-engr.com/gb300-adaptive-residency-seed-map.html — Adaptive Residency, Part 3: The Seed Was the Prize (seed the expert map from the live counter: +44.5% held-out mixed frozen; aggressive swap policy closed; measured swap cost; v26 soak) - [Adaptive Residency, Part 2: On the RTX Sidecar Lane, a Better Map Beats a Live Swapper](https://al-engr.com/gb300-adaptive-residency-sidecar-mixed-map.html): Part 2. On our GB300 + RTX PRO 6000 sidecar lane the GB300 waits on the RTX 11.8% of the time during Japanese/German prose and 0.5% on agent English, so we refit the static hot/cold expert map from live counts instead of building a cross-process swapper: +5.8% on held-out mixed traffic at one user (prose +15%), no change at four. Both lanes published as experimental recipes. - [We Built the Counter: Adaptive Expert Residency Doubles Off-Domain Decode on One GB300](https://al-engr.com/gb300-adaptive-expert-residency.html): Follow-up to the cold-expert prefetch explainer. Letting the hot/cold expert split on our Grace-only DeepSeek-V4.1-Flash recipe follow live traffic took Japanese/German prose from 90 to 189 tok/s at one user (+111%), agent English +12%, and mixed traffic +46% at one user and +82% at four. The counter costs 2–6 µs per step with relaxed atomics; 15,382 swaps were lossless. - [Should a GB300 Prefetch Its Cold Experts? Mostly No, and What to Build Instead](https://al-engr.com/gb300-cold-expert-prefetch-explained.html): An explainer with diagrams: could vLLM guess which DeepSeek-V4.1-Flash experts are needed next and copy them from Grace or SSD into HBM ahead of time? On one GB300 a usage map already removed ~98% of cold traffic, so a perfect prefetcher buys at most a few percent while a 3-slot bank costs 43% of the KV cache, and every cheap guesser measured at chance. Build a device-side usage counter instead so the hot/cold map follows traffic. Update: built the same evening; it doubled off-domain decode (see the follow-up). - [Twelve Buttons, One Thumb: the WoW bar I copied from a real player](https://al-engr.com/naga-keybinds-for-wow-forever.html): A Naga V2 Pro for WoW Forever and a keybind layout I invented twice before copying one from Esfand. The button map, the two swaps Forever's mechanics force, and why the twelve side buttons need no software at all. - [Kimi-K3 at 0.0086 bits per parameter, on one Spark (it is a joke)](https://al-engr.com/kimi-k3-0-0086-bpw-lulz.html): 2.8T parameters, 0.0086 bits each, 3.01 GB, 83.65 tok/s on one DGX Spark. 99.4% of the weights are deleted and the parameter count on the label is not: same weights and same quantizer report 0.0447 / 0.0140 / 0.0086 bpw at 93 / 24 / 12 layers. The output is not language; the arithmetic is the article. - [CRS804 fan noise: why the fans surge, and the setting that stops it](https://al-engr.com/crs804-fan-noise-what-works.html): On RouterOS 7.25 the CRS804's fans swing between 3,180 and 16,000 RPM every few minutes. One setting stops it: fan-min-speed-percent=38% holds a steady ~5,900 RPM with the chip at 55–56 °C. Why it surges, what does not work, and the hardware options for real quiet. - [GB300 with RTX 6000, and 120V Power](https://al-engr.com/gb300-rtx6000-120v-power.html): Does 240 V buy back the GB300's power throttling with an RTX 6000 sidecar? On this box, no: the 1,250 W module cap comes from the 6000's 300 W rating, and MSI rates the supply at 1,600 W from 115 V up. Measured cost of the cap: at most ~5% of SM clock in long prefills, 0.03% in 32-user decode, nothing at one user. Plus what does help on 120 V, and when 240 V is worth wiring anyway. - [Adding an RTX PRO 6000 Blackwell Max-Q to the DGX Station GB300](https://al-engr.com/adding-rtx-pro-6000-max-q-to-dgx-station-gb300.html): The 96 GB, 300 W Max-Q is in the Station's one double-wide x16 slot and booted on the first try. As-built inventory, the 1,250 W module cap that was there before any NVIDIA software, the 1,550 W static split of a 1,600 W supply, the console cable that had to change, and a diagram of where DeepSeek-V4.1-Flash's bytes sit with and without the card. - [DeepSeek-V4.1-Flash on a GB300 + RTX PRO 6000 Sidecar: +49% for One User, 3× at Sixteen](https://al-engr.com/gb300-dsv41-rtx6000-sidecar.html): New serving platform: the GB300 keeps 295 hot experts per layer and the Station's RTX PRO 6000 Max-Q computes the other 89. Same-window vs the GB300-only lane: one user 187→279 tok/s (+49%), first token 0.66→0.19 s, 16 users 3.0×, 64K prefill 2.1×, 1M prompt first token 81→33 s. 12/12 quality gates, 0 sidecar timeouts, and a watchdog that stops the lane rather than serve wrong tokens. Serving since October 22. - [NVIDIA Flare on Two DGX Sparks: GLM-5.3-Flash TP2, Tuned](https://al-engr.com/glm53-flash-flare-tp2-2x-dgx-spark.html): GLM-5.3-Flash split across two DGX Sparks under NVIDIA's Flare, now with diagrams. Defaults give 37 tok/s for one user but stop drafting at four users; an 8-seat package with three drafts at a fixed verify width keeps one user at 39 and lifts four users from 56 to 78 tok/s and seats eight at 107. 1.7× the cold prefill of our earlier two-Spark vLLM lanes, 6.1M-token KV, 570K needle, GSM8K-200 96.5%. Today's main can't run it (#17); this is build 47581cc. - [NVIDIA Flare for agents: two rough edges, measured](https://al-engr.com/flare-two-rough-edges.html): Two agent-facing rough edges measured in the four-way Flare run: a ~71-token tool call arrives in exactly 2 streaming deltas in the final ~54 ms (SGLang: 9 deltas over ~500 ms), and Hermes' strict-JSON session-title request gets an instant HTTP 400 on reasoning_effort "none" — 8/8 across two runs, so sessions never get titled. The agent chain itself still ties. - [DeepSeek V4.1 Flash on four DGX Sparks: four engines, one winner](https://al-engr.com/dsv41-1017-four-way-flare-tensorfold-sglang-vllm-4x-dgx-spark.html): Four engines, one model, the same four DGX Sparks and switch. Winner: the vLLM fork (287 tok/s at six streams, 55 s to read 300K tokens, 100 s boot). SGLang is our daily lane because it keeps chat responsive during someone else's long prompt; NVIDIA Flare holds 5.8× the KV cache but decodes slowest; TensorFold is the fastest single stream on a 3.5-bit quant. All four tie on a Hermes agent chain and on quality. - [NVIDIA Flare on One GB300: GLM-5.3-Flash TP1 vs the SGLang Daily Lane](https://al-engr.com/gb300-flare-glm53-flash-vs-sglang.html): NVIDIA's Rust/CUDA-graph server runs on the GB300 (SM103) despite its SM120/121 README; DeepSeek-V4.1-Flash is TP4-only, so the one-Station test is GLM-5.3-Flash. Flare beats the SGLang daily lane on single-stream decode (+11–25%) and prefill, ties GSM8K-200. Tuning update: --speculative-token-capacity 32 lets Flare keep drafting at 16 concurrent requests — C16 +26% per-stream, C1 unchanged, GSM8K-200 98.0% — and that package now serves. - [DeepInfer: What DeepSeek's Engine Does, and What a 4-Spark Homelab Can Steal](https://al-engr.com/deepinfer-what-transfers-to-a-4-spark-homelab.html): DeepSeek's production engine for V4.1-Flash, sorted into what a four-Spark lane can use: decoder sliding-window replay (already running here as CED), confidence-priced verification, and a prefill chunk policy that cut a short chat's wait behind a 413K prefill from 3.3 s to 1.7 s. Plus the seat-count change that lifted 32-stream throughput 50%. - [Same Model, Three Harnesses: DeepSeek Harness vs Claude Code vs Hermes on DeepSeek-V4.1-Flash-1017](https://al-engr.com/dsv41-1017-harness-shootout-hermes-dsh-claude-code.html): Same DeepSeek-V4.1-Flash-1017 endpoint, three harnesses, 120 Terminal-Bench 2.0 trials: Pass@1 Hermes 75%, Claude Code 75%, DSH 70% — not distinguishable. Tokens and wall time are where they differ: DSH 1.23× Hermes prompt tokens and fastest lanes, Claude Code 0.74× and fewest output tokens, with a vLLM prefix-cache caveat on its speed. - [A Family Grocery Agent: The Instacart Plan](https://al-engr.com/family-grocery-agent-plan.html): Plan for giving Cindy's Hermes agent grocery-list and Instacart capability. Instacart's developer platform is closed to new applicants, the enterprise MCP is partner-only, and no grocer offers a self-hosted agent a checkout API; so: list in Telegram first, shoppable Instacart link via a hosted MCP second, checkout stays human. Store reality for Telluride and Pensacola, staged rollout from the lab profile to Cindy's agent, gates and stop rules. - [DeepSeek Flash 1017: TensorFold vs SGLang vs vLLM on 4 DGX Sparks](https://al-engr.com/dsv41-1017-three-way-tensorfold-sglang-vllm-4x-dgx-spark.html): Third lane on the same four Sparks (vLLM fork vs SGLang vs TensorFold). Round 2: both native engines tuned on disjoint node sets, nothing kept; SGLang prefill corrected to 6.2K at 93K, level with vLLM; cross-pod contention negative; both configs published as J-M-Recipes. - [DeepSeek Flash 1017: TensorFold vs SGLang on 4 DGX Sparks](https://al-engr.com/dsv41-1017-tensorfold-vs-sglang-4x-dgx-spark.html): Same model, same day, same switch: LibertAI's 3.5 bpw EXL3 pack on TensorFold (Sparks 1–4) against the native MXFP4/FP8 checkpoint on SGLang via knapcio v2.4 (Sparks 5–8). EXL3 wins single-stream decode by 18% (100 vs 85 tok/s); native wins prefill 1.6–2× and the Hermes 20-hop agent gauntlet by 26%, and leads from four streams up. Quality gate identical, 71/75 both. Now extended to a 3-way with a tuned vLLM fork lane: [DeepSeek Flash 1017: TensorFold vs SGLang vs vLLM on 4 DGX Sparks](https://al-engr.com/dsv41-1017-three-way-tensorfold-sglang-vllm-4x-dgx-spark.html) (prefer it for current Spark-lane guidance; this page stays the A/B record). Since October 19 Lane B is stopped (kept) because that lane runs on its nodes. - [Watt-hours per solved task: Hermes Agent on Terminal-Bench 2.0, GLM-5.3 on eight DGX Sparks](https://al-engr.com/hermes-agent-energy-bench-glm53-spark.html): First scored run of a wall-plug energy benchmark for agents: Hermes Agent v0.22.2 driving GLM-5.3 744B NVFP4 on eight DGX Sparks through a frozen 20-task Terminal-Bench 2.0 slice. Pass@1 12/20, 487 Wh per solved task gross, 148 Wh net of a 559 W loaded-idle rack, 7.3 h at c=1. Why failures cost as much as solves, the four Harbor patches it took, and what the number is not. - [Optimizing My Own Inference: Seven Levers, Two Worth Pulling](https://al-engr.com/optimizing-my-own-inference.html): I read the Prometheus counters on the lane that serves me - 19 requests, 643,682 prompt tokens, a 64.7% prefix-cache hit, 12.65 ms decode steps - and ranked seven levers against my own traffic instead of a benchmark. Six are already closed, and the engine has been logging the one that matters 54 times: every prefill shape I send misses the autotune bucket set. Two boots, both reversible. - [4× RTX PRO 6000 vs One GB300 Station: DeepSeek-V4.1-Flash-1017 on the Same Grid](https://al-engr.com/gb300-vs-4x-rtx6000-dsv41-1017.html): Same 45-case grid on two single-box builds of the native 763B checkpoint: 0xSero's (sybil-solutions) 4× RTX PRO 6000 (SGLang, DSpark k=5, Engram on NVMe) against one GB300 Station (vLLM, UVA expert offload, pin-hot hook). Station leads single-stream decode 1.25× and prefill 3.4× with TTFT about 3× lower; the four-GPU box leads 0.86× at C4 and 0.56× at C6–C8, where the Station's k-schedule drops to one draft token — flat k=5 recovers most of that. - [Engrams for GLM-5.3: Why You Can't Transcode Experts into Lookup Tables](https://al-engr.com/gb300-glm53-engram-idea.html): The seductive idea: GLM-5.3's routed experts stream from Grace at 18 ms of a 40 ms decode step on one GB300, while DeepSeek-V4.1-Flash's 196B Engram tables sit in the same Grace RAM and cost 0.04 ms. So move expert capacity into tables? Four research passes against primary sources say no: an expert is a function of the hidden state and an Engram row is a function of token ids, nobody has published the conversion, DeepSeek's own allocation law shrinks expert inventory rather than bytes read per token, and two V4.1-sized tables do not fit beside the 380 GiB expert pin. Deleting the first 16 MoE layers would free 81 GiB for a modeled ≤3.8 ms. What survives is a CPU-only measurement. - [DeepSeek-V4.1-Flash-1017 on One GB300: Same Lane, New Drafter, One Hard No](https://al-engr.com/gb300-dsv41-1017-day-zero.html): Day-zero on DeepSeek's weights-only 1017 update: config, shard index and tokenizer are byte-identical to the September checkpoint, so the v21 lane boots it unchanged. Same-window CTL→CAND→CTL: C1 182.7→192.6 tok/s (+5.4%), C4–C16 −2…−4%, tool gauntlet 12/12. The retrained DSpark drafter accepts ~10 pts less on shell/tool, the k-schedule survived a re-check, GSM8K-200 is flat within noise, outputs are 14–24% shorter, and vLLM 0.31.0's MegaMoE cannot serve this model on one Station. - [GLM-5.3 on eight DGX Sparks: SGLang vs vLLM, and a unified-memory bug](https://al-engr.com/glm53-8x-spark-sglang-vs-vllm.html): Same 744B model, same eight GhettoRak Sparks, second engine. A 13-arm SGLang v0.5.22 campaign against the vLLM lane: 4–16% slower single-stream, faster at four users with 2-step EAGLE, and every long prompt hung until one env var bounded the sparse indexer's scratch buffer. The bug, the fix, and what refuses to boot on SM121. - [Muse Spark Open 816B on one GB300: 29 tok/s with 311 GiB of experts in Grace](https://al-engr.com/gb300-muse-spark-open.html): Meta's Muse Spark Open (816B total, 42B active, NVFP4) on one DGX Station GB300 the day after release: 311 GiB of experts in Grace via vLLM UVA offload, DSpark k=2. 29.4 tok/s single-stream, 49 at C4; BFCL 563 vs 561 and GSM8K 193 vs 192 against no speculation. FlashInfer autotune cost 1 h 48 min per boot and bought nothing; k=7 drafts were slower than none. Next: longer context. - [Full GLM-5.3 on eight DGX Sparks: official FP8 and NVFP4](https://al-engr.com/glm53-fp8-8x-spark.html): GLM-5.3 744B tensor-parallel 8 across all eight GhettoRak Sparks, two ways. Daily lane: RadixArk NVFP4, 512K context, 25.8–35.7 tok/s single-stream once MTP guesses three tokens ahead, 61.7 at 4 users. Fidelity reference: Z.ai's official FP8, 131K, 23.1–29.9 tok/s, 2.3× closer to BF16. Why the 4-bit build first lost, and recipes for both. - [StudioStack on the M4 Max: what we put in it](https://al-engr.com/studiostack-m4max.html): An OWC StudioStack now sits on the M4 Max Mac Studio. Inside: one 8 TB WD_BLACK SN850X on PCIe 4.0 x4, the 3.5-inch bay empty. It runs over an 80 Gb/s Thunderbolt 5 link and holds ~/Downloads. One dd pass: 6.29 GB/s sequential read, 3.59 GB/s write+sync. - [Step-5-Preview on one GB300: day-one bring-up, quality gates, and where the time goes](https://al-engr.com/gb300-step5-preview-nvfp4.html): StepFun Step-5-Preview-NVFP4 (vendor floor 4× B300) on one DGX Station GB300 the day it shipped: 180 GiB of experts on Grace, 256K context, needles 30/30 to 258K, BFCL 559/600, bit-identical across restarts. 30.1 tok/s single-stream, 39–42 with MTP k=1 (bit-identical). A routing trace says a MiMo-style hot/cold expert split would add ~25–40%; parked as not worth the kernel work. - [Shrinking Kimi K3 With EXL3: A Plan for 8× DGX Spark or One GB300](https://al-engr.com/kimi-k3-exl3-shrink-plan.html): Kimi K3 (2.8T) fits 8 DGX Sparks if its routed experts drop to about 2.3 bits (758 GiB), or one GB300 at 2.0 bits (663 GiB). A plan to get there with EXL3 trellis quantization, why EXL3 beats EAQuant for this job, and an estimated 14–28 hours to quantize on the GB300 plus 8 Sparks. Plan and estimates; nothing converted yet. - [TensorFold on One GB300: How Ash Hart's Engine Serves the Whole GLM-5.3](https://al-engr.com/gb300-tensorfold-glm53-explained.html): How Ash Hart's TensorFold build serves the full 744B GLM-5.3 NVFP4 on one Station, now with a same-box head-to-head against our vLLM recipe. TensorFold wins on many users at once (83 tok/s total at 16 streams). vLLM reads prompts 2.7× faster and reuses earlier turns, so it wins agent work: this TensorFold build has no prompt cache and timed out on 7 of 20 SkillsBench tasks. - [DeepSeek-V4.1-Flash on four DGX Sparks: stock vs abliterated](https://al-engr.com/spark-dsv41-stock-vs-abliterated.html): DeepSeek-V4.1-Flash serves from Sparks 3–6 (TP4, 1M context). Abliterated weights cost no speed and no accuracy on math, tools or long context; GPQA-Diamond with thinking on drops from 84.8% to 80.3% (p = 0.08, one run). - [Kimi-K3 on a Station plus Sparks: the remote-expert hop, measured](https://al-engr.com/kimi-k3-pod-remote-expert-hop.html): One layer of Kimi-K3's routed experts served from a DGX Spark over RoCE, with real NVFP4 weights: 269 µs for one cold expert, 438 µs for three, 5.7 µs of that on the wire. Cosine 0.996 vs the fp32 reference, 2,211 req/s sustained, and a per-launch memory growth that still needs fixing. Why the network is not the problem and expert placement is. - [The cables a new DGX Station will not see](https://al-engr.com/dgx-station-gb300-setup.html): First-week DGX Station GB300 setup. Ubuntu owns the 10GbE RJ45, not the BMC jack. QSFP112 cages are ConnectX-8 400GbE. CVE-2026-80521 still Vulnerable on the 24.04 NVIDIA kernels as of October 2, 2026. - [The Whole GLM-5.3 on One Desk: How the Slot-Cache Recipe Works, and What We Might Try Next](https://al-engr.com/glm53-big-slot-cache-explained.html): full 744B GLM-5.3 on one GB300 via expert slot cache, explained with diagrams; Oct 1 measurements; GrokMilo ideas scored; speculation. - [Mooney: squeezing a 180B model onto one DGX Spark at 2.39 bits/weight](https://al-engr.com/mooney-qwen38-flash-next-low-bit.html) — Daniel Lougen's 2.125-bit quant of Qwen3.8-Flash-Next hits 95.5% of BF16 quality in 39-57 GiB; rebuilt and hash-verified on our own Spark, with an NVMe-coalescing A/B that came back negative for this workload. - [Switching 8 DGX Sparks and a GB300 Station: CRS804 ×2 vs one 32×400G switch](https://al-engr.com/spark-cluster-switch-options.html) — Eight Sparks at 200G plus a Station at 400G do not fit one MikroTik CRS804. NADDOD says buy their $10,459 N9200-32DC and warns of "unavoidable" inter-switch latency. Quantified: ~1 µs per hop, under 0.5% of a decode step. The real trade-offs are trunk bandwidth, zero spare ports, six datacenter fans, and 200–240 VAC only. Two diagrams, one table, no order changed yet. - [I went looking for M5 kernel fruit. Apple already ate it.](https://al-engr.com/m5-nax-coverage-map.html) — Measured on an M5 Max: MLX already routes ~98% of LLM prefill compute through the Neural Accelerator kernels, and 4-bit matmuls run within 3–15% of fp16 at ~57 TFLOPS. No unowned kernel win in the dense path; the small headroom left is long-context attention and the serving layer. A negative result with four diagrams, a dispatch-rule harness, and a second-model plan review. - [MiMo-V2.6-Pro-MOPD on one GB300: the recipe holds, and the flood is gone](https://al-engr.com/gb300-mimo-v26-pro-mopd.html) — Xiaomi replaced the RL checkpoint with MOPD on September 27 to fix tool-call repetition. Same recipe, same image and patches, new weights: all pre-registered bars pass (teacher-forced flips 1.18%, BFCL held-out 78.7% vs 78.4%, decode 37.1 tok/s). On a 72-run flood fixture the RL weights flooded 25 of 36 history-primed tasks at 94.7% duplicate calls; MOPD flooded none, 72/72 solved. - [Sep Security Work: a container-escape bug, and what it shook loose](https://al-engr.com/sep-security-work.html): Unpatched Ubuntu container-escape CVE; the audit found Docker bypassing ufw, a default DB password, and the vault key in agent files. All fixed. - A 1-terabyte model on one box: MiMo-V2.6-Pro, explained: https://al-engr.com/gb300-mimo-v26-pro-explained.html — the memory puzzle, the loader bug, and hotsplit, in three diagrams. - [MiMo tried to blog about itself. The turn died truncated.](https://al-engr.com/mimo-self-blog-truncated.html): One near-16K-token completion per artifact against a 16,384 output cap; Hermes continued 4x then ended both turns as error. Draft validated locally, nothing uploaded. Conclusions: reshape the harness, one artifact per tool call, leave the cap alone. - [AI-Orchestrated Inference Engineering](https://al-engr.com/ai-orchestrated-inference-engineering.html): 57M tokens of context for 182K of thought in one overnight campaign; the loop/orchestrator/reasoner split (shell, Grok 4.6, Fable 5.1 on demand) that fixes it. - [A 1 MW solar datacenter that can grow to a gigawatt: where to put it](https://al-engr.com/solar-datacenter-site-selection.html): Compute-first solar campus: one 1 MW module (~325 kW IT, $4.8M) on land sized for 1 GW. Sourced site screen — EIA June 2026 industrial rates, Solargis capacity factors, USDA 2025 land values, Texas SB 6. Eastern New Mexico / West Texas wins. Theorycraft, nothing signed. - [Unasked work](https://al-engr.com/glm-unasked.html): Mia asked for an example of GLM doing extra in Hermes. Two of our sessions: 216 tools on fix Echo, 142 on a continue, both hit the iteration cap. - [Fill the yard with solar. Then see what compute fits.](https://al-engr.com/yard-solar-compute-budget.html): Yard-first solar on a ~2-acre residential lot: 164–189 kW-dc, 18–21 kW islanded IT through winter. About 100 Sparks, or almost one Rubin NVL8 with the house roof. 0.1 of one NVL72. Theorycraft, no spend. - [vLLM: Pin Hot Experts](https://al-engr.com/vllm-pin-hot-experts.html): Updated October 18: exact-size pinned host allocations free 75 GiB of Grace RAM under the hook (v23). September 27: v21 "Fused Split" — a torch-profiler capture of the v18 step (14.6 ms, GPU 101% busy, ~1,300 small kernel launches) pointed at the hook's own six-op split_ids(); one Triton kernel replaces it. Same-window bracket v18 → v21 → v18: single-stream 182.8 vs 171.8/172.5 (+6.2%), C8 687 vs 656/657, C16 993 vs 957/952, non-tool greedy 16/16 identical vs both controls; serving on the lane, v18 kept as rollback. Same session closed huge pages: GPU reads of pinned Grace memory are identical on 64 K and 512 MiB pages — C2C is bandwidth-bound. September 17 result stands: usage-aware residency, 295 hot / 89 cold experts, bit-identical, 89 → 153 tok/s single-stream. - [DGX Station GB300: current configuration](https://al-engr.com/gb300-station-config.html): Kernel 7.0.0-1019-nvidia-64k, driver 595.91.07, UEFI ES429NMS.103. CDMM still driver-mode. Firmware not flashed. Phased apt skipped the packages that mattered. - [GB300 Model Testing Summary](https://al-engr.com/gb300-model-testing-summary.html): Four models actually served on one DGX Station GB300 (DSF Vision-Exp, full GLM-5.3, GLM-5.3-Flash, DeepSeek-V4.1-Flash) compared on prefill, decode, context, and inference engine. Flash is the one listening today. Dated lab receipts, not a leaderboard. - [Milo and James Try to Inference Engineer](https://al-engr.com/milo-and-james-try-inference-engineering.html): The plain-English story of DeepSeek-V4.1-Flash on one desk-side GB300 — written for Bob. What the model is, how it was released, and the six steps that took it from 3.3 tokens/second to 90 on prose and 140–160 on code, with the full memory recipe. Three diagrams and a cartoon; no new benchmarks, every number traces to the lab notes. - [GB300 GLM-5.3-Flash Round 3: I Was Benchmarking the Wrong Thing, Then Seven Axes Closed](https://al-engr.com/gb300-glm-flash-round3.html): GLM-5.3-Flash round 3 on one GB300: the enable_thinking flag was a no-op, so round-2 single-stream numbers measured max-effort thinking with the text counted — re-measured answer-only at 202.7 tok/s (265 with thinking counted). Then eight one-axis windows: nightly 8874c51a is the new daily (+2.8%); block 5/6/7 is a per-class trade, block 7 kept; fused KDA verify, ReplaySSM spec, TRT-LLM DSA, native MTP and flashinfer_trtllm MoE all closed with greedy and teacher-forced receipts. Native MTP now boots on NVFP4 with natural acceptance 3.16 and still loses to DFlash2. Recipe PR #25. - [GhettoRak: an 8-node DGX Spark cluster](https://al-engr.com/inference-box.html): October 29 rewrite. Eight DGX Sparks on two MikroTik CRS804 switches (Sparks 1–4 / 5–8), one 200G cable per Spark, one 400G cable between switches. Measured October 29: all eight reach 24 GB/s NCCL all_reduce at 256 MiB; a ring forced across the 400G link matches a one-switch ring. GB300 Station on switch 2 with 2 × 200G: 196 Gb/s RDMA per port, both ports 392 Gb/s from GPU memory via Data Direct (231 from host memory, shared PCIe Gen5 x8 link), ECN on the switches, +1.0 µs per switch hop. 2,200 MHz GPU clock cap keeps the rack on one 15 A strip (~1.35 kW long prompt). Wiring, shelves, RouterOS/netplan/NCCL recipe, buy list. - [Speculative Decoding by Traffic: v14 Wins the Matched Replay](https://al-engr.com/gb300-dsv41-k-schedule.html): Three alternating v14/v13 boot pairs on DeepSeek-V4.1-Flash: v14 gains 18.1% on initial recorded-history replay and 19.5% on identical-request repeats. Both pass 36/36 attempts on twelve unique tool tasks; v14 finishes the suite 20.3% sooner, while v13 retains lower median replay TTFT. Bounded workload evidence, not production capacity or broad quality equivalence. - [Teaching the Speculator What a Draft Costs](https://al-engr.com/gb300-dsv41-online-verify-curve.html): Companion to the DeepSeek Flash 4.1 post. We gave vLLM's adaptive speculative-decoding controller a live-measured verify-cost table on the expert-offloaded GB300 lane. The boot table was 12–40% optimistic, the controller used the corrected one, and almost nothing changed: C1 +3%, C2/C4 −6/−8%, tool-JSON +15%, nothing adopted. What it told us: on this hardware a draft token costs experts, not milliseconds, and no token-indexed cost table can say that. Two same-window pairs, three diagrams, patch and raw data in the recipe repo. - [GB300 Links](https://al-engr.com/gb300-links.html): Annotated roundup of public deskside GB300 / DGX Station coverage: ServeTheHome's Gigabyte W775 deep dive (GPU vs Grace ConnectX-8 paths), Alex Ziskind and Wendell's Station vs 8x RTX PRO 6000 agent run, StorageReview's WS300 review, Level1Techs' week with an MSI WS300, Ahmad Osman on MTS with a Station in the studio, NVIDIA's own pages, and the OEM list. Not our measurements. - [Offload by Expert on GB300: The Skew Is Real, the Kernel Is Not](https://al-engr.com/gb300-deepseek-offload-by-expert.html): 98.7% of DeepSeek-V4.1-Flash routed-expert selections hit just 73% of the table, so the obvious win is usage-aware expert offload to Grace. It fails on two walls — a layer's weights are one contiguous tensor with no per-expert kernel hook, and the GPU reads Grace at full speed (ATS 340 GB/s) or with placement control at quarter speed (90 GB/s). The two-tensor row-map is the only design that gets both, at +20% C1 in-domain and a 74-minute autotune; residency axis closed on this hardware. - [One GB300 for a 35-Seat Investment Fund: Seven Serving Configs Against a Real Workload](https://al-engr.com/gb300-35-seat-fund-workload.html): A prospect asked whether one DGX Station GB300 can serve 35 seats. Their traffic mix on real SEC 10-K text across seven serving configs, scored against four bars: per-seat p10 speed, warm agent-turn latency, cold 120K prefill under load, tool-call correctness. Qwen3.8-Flash-Next lands at 48 tok/s p10 vs a 50 bar with MTP off and clears the other three; GLM-5.3-Flash DFlash2 clears speed (67) by starving KV. No config clears all four. Updated September 18, 2026: the DeepSeek-V4.1 lane was re-measured on its current pin-hot-experts profile (tool calls 64/64; cold-prefill latency and interference improved, short-turn latency worse). - [GLM 5.3 Research: Where the Decode Time Actually Goes](https://al-engr.com/glm53-research.html): Independent replay of the frozen 79,119-token GLM-5.3 routing corpus reproduces the one-GB300 slot-cache simulator exactly (hit 0.7331112905490802; 4.2702 misses/step/layer). 17.6 of the measured 40.1 ms decode step is expert-row copy over NVLink-C2C — 44% of the step at 80% of the measured 383 GB/s ceiling. Cache policy cannot clear the 15% miss gate; the remaining levers are KV placement and slot budget. Corrected units (4.61/3.11 are access rates), 36.8 GB of the budget unattributed at 512K, the never-booted KV-offload lane priced, 1.78x honest ceiling. - [GB300 GLM-5.3-Flash Round 2: Which NVFP4, a Wrong Verdict Fixed, and 128 Users on One GPU](https://al-engr.com/gb300-glm-flash-round2.html): GLM-5.3-Flash round 2 on one GB300: NVIDIA's NVFP4 vs the community quant scored against the FP8 original by teacher-forced divergence (0.136 vs 0.147; both flip ~16% of tokens); the September 1 'DFlash2 loses above 16 users' verdict was a 7-request KDA slot cap, not speculative decoding — corrected; DFlash2 block 7 gives ~202 tok/s single-stream answer-only at 1M context (the 252 originally published here counted max-effort thinking text — corrected Sept 16, see round 3); plain autoregressive serves 48 users at 50 tok/s each and 128 at 34 tok/s each with sub-1.5 s first tokens. Recipe PR #7. - [Filling the empty M.2s on an Exxact GB300 Station](https://al-engr.com/gb300-exxact-m2-upgrade.html): empty CX8 2280 slots on an Exxact / MSI WS300 GB300 filled with 2× SN850X 8 TB, RAID0 14.55 TiB at /models. Identify by serial; nvme names reshuffle. - [GB300 MiMo-V2.6-Pro Testing](https://al-engr.com/gb300-mimo-v26-pro-testing.html): Recipe v23 Hotsplit verified September 24, 2026: top-1 flips 1.23% vs a 1.17% reboot floor, BFCL held-out 78.2% vs 78.5% stock-placement control, 1M needles 3/3 at 1.04M (25.7 tok/s). Serving correctly on one GB300 as of September 22, 2026, 10:10 AM CDT. The garbage was --load-format runai_streamer zeroing fused-qkv fp8 weights; Marlin fixes the SM103 autotune (vLLM #58031). C1 30.2 tok/s, C8 56.2 agg, DFlash off. Update September 22, 2026 at 4:30 PM CDT: hotsplit lifts decode 31.2 to 35.1 tok/s at 11.5k and agent tool_json 29.5 to 35.9 tok/s, same HBM budget. - [GB300 DeepSeek Flash 4.1 Testing](https://al-engr.com/gb300-deepseek-flash-41-testing.html): Living log of DeepSeek-V4.1-Flash on one DGX Station GB300 + RTX PRO 6000 sidecar at full 1M context. October 31: Pg61 (Garner McCloud's DS4.1 engine under our policy) is the lane and ships as an experimental recipe: 1.31× the old lane at one user, prefill 6–9 % slower; fault injection, rollback drill, second-card numerics, forward fidelity and temperature-1 sampling calibration all pass; 48 h soak ends Monday November 2 at 11:44 AM CST. - [keys suggested DFlash2 for our GLM-5.3 recipe](https://al-engr.com/glm53-dflash2-idea.html): measured DFlash2-over-UVA result. Geometry passed, but K4 accepted length was 1.5718 against a frozen 3.0 stop gate; DFlash2 was not promoted and the 512K/MTP lane remains. - [Skill check: 260 skills, two new reviewers, one bloat machine](https://al-engr.com/skill-check-2026-09-05.html): sweep of the Hermes skill library with a second-model review; archive/distill/merge, reviewer gated. - https://al-engr.com/gb300-glm-53-hbm-expert-allocation.html — Historical HBM expert slot-cache design notes for full GLM-5.3 on one GB300. Preserves the early build measurements and memory-budget lessons; superseded by the September 6 campaign results and experimental recipe. - [Recipe v2 from the DSFVE inner loop](https://al-engr.com/testing-the-inner-loop-on-dsfve.html) — Recipe v2 and inner-loop evidence (September 4, 2026) - https://al-engr.com/enhanced-research-pipeline.html — Enhanced Research Pipeline: tracking the Spark inference-optimization community (plan, September 3, 2026) - [Designing an Inference Recipe Generator/Optimizer](https://al-engr.com/designing-an-inference-recipe-generator.html): original two-loop design plus a September 4 update linking the first completed outer-loop pass and Recipe v2. - [DeepSeek-V4-Flash-Vision-Exp on One GB300: Native FP4, SGLang, DSpark, 1M Context](https://al-engr.com/dsf-vision-exp-single-gb300.html): current Recipe v2 is digest-pinned static/no-SPS/checkpoint-NextN on one GB300; Iteration 13 measured 2,985.4 tok/s at C64 and equivalent C8–C64 throughput inside ±3%. - https://al-engr.com/gb300-glm-53-testing.html — September 13: 256K daily profile; 51.3/44.8/34.1 tok/s at 256K/512K/1M; 18/18 needle ladder to 211K; K=2 quality-equivalent, +3.2%, not promoted; LRU beats expert pinning on the trace. - https://al-engr.com/gb300-glm-flash-recipe.html — GB300 GLM 5.3 Flash Testing: living recipe. October 15: our self-trained DFlash2 drafter FAILED (prose 1.97 vs incoai 2.34-2.44 at K=4, 3/4 bars missed, 16-19% slower; on HF as a labelled failure; use incoai). incoai prose acceptance ~2.3 on every stack; K=4 beats K=7 on prose. Daily image since Sept 29: 09-22 nightly with #39688 reverted + ReplaySSM (KV pool +48%, C8 ~744). - https://al-engr.com/gb300-station-day-one.html — GB300 Station day one: bring-up, driver fix, SGLang PR #36507 recipe, first benchmarks (September 1, 2026) - [Vision-Exp thinking levels on 2× DGX Spark](https://al-engr.com/ds4-vision-exp-thinking-levels.html) — Matched off/low/high/max thinking probes; effort labels are not a ladder (September 1, 2026) - [DeepSeek-V4-Flash-Vision-Exp on Two DGX Sparks](https://al-engr.com/ds4-flash-vision-exp.html) — Vision-Exp lane now abliterated + block-K=5 on Mia tip 97e8733: 85.8-89 tok/s, KV 2.34M, keyed API (September 16, 2026) - [mom-photo-recovery](https://al-engr.com/mom-photo-recovery.html) — Photo Hunt, Part 2: 7,038 Photos Off a Wiped SSD (no-TRIM X25-M G1; dd image, PhotoRec carve of the image, EXIF sort; 7,038 photos incl. 1,495 lab scans) - [mom-photo-hunt](https://al-engr.com/mom-photo-hunt.html) — Photo Hunt: Mom's Photos Aren't on the NAS (2009 Mini disk pull; older photos not on the NAS Aperture library; carve if reinstall) - /pr-work-week-of-20260831.html — PR Work, Week of August 31, 2026 (weekly OSS log) - https://al-engr.com/grok-october-bet.html — The October Bet: Will Grok Lead the World? (falsifiable Grok #1 prediction, settles October 1, 2026) - [NVFP4 vs EXL3: GLM-5.3-Flash on Two DGX Sparks](https://al-engr.com/glm53-nvfp4-vs-exl3-2x-dgx-spark.html): matched quant faceoff on one Spark pair; cold-prefill protocol; machine-readable summary block for agents - [GLM-5.3-Flash DFlash2 on 2× DGX Spark](https://al-engr.com/glm53-dflash2-2x-dgx-spark.html) — August 28, 2026. Pinned Tony DFlash2 recipe on Funland: mixed C1 41.5 tok/s, 20/20 tools, 98K retrieval. - https://al-engr.com/glm-flash-53-testing.html — GLM Flash 5.3 day-0 testing on dual DGX Spark (SGLang recipe, gates, GB10 memory trap) - [Ornith 1.0-397B on M3 Ultra: GGUF and MLX Recipes](https://al-engr.com/ornith-397b-m3-ultra-recipe.html) — June 29, 2026. Companion bring-up recipe for Ornith 1.0-397B on the 512 GB M3 Ultra: llama.cpp GGUF and MLX 8-bit routes, launch flags, and serving notes. - [Flint-chart: Putting a Compiler Between Agents and Bad Plots](https://al-engr.com/flint-chart.html) — August 2, 2026. A hands-on Flint-chart test: semantic chart specs, hardened MCP, two honest architecture-diagram failures, and where the compiler actually fits. - [Trying the mlxfast Challenge: Speeding Up AI on a Mac](https://al-engr.com/mlxfast-challenge-beginners.html) — August 2, 2026. A beginner-friendly walkthrough of the Layr-Labs mlxfast challenge: what Laguna XS is, how we set it up on an M4 Max, why tiny kernel tweaks failed, and what we learned about noise, heat, and already-optimized code. - [MiMo 2.5 Test: A Useful Failure on Two DGX Sparks](https://al-engr.com/mimo-v25-nvfp4-dual-spark.html) — June 29, 2026. The MiMo 2.5 NVFP4 dual-DGX Spark experiment: the July upstream reproduce still failed locally because our Sparks exposed less than half the reference KV-cache pool. - [Testing MiMo V2.5 on Two DGX Sparks: The Plan](https://al-engr.com/mimo-v25-dual-spark-test-plan.html) — June 29, 2026. A staged test plan for running Xiaomi MiMo V2.5 Omni on 2× DGX Spark as a local multimodal/tool-agent backend, without disturbing the production DeepSeek route. - [The Closing Tag That Never Comes: Kimi K2.6 on a 512GB Mac](https://al-engr.com/kimi-k26-mac512-reasoning-split.html) — June 2, 2026. Probing Kimi K2.6 (DQ3_K_M-q8 MLX quant on the M3 Ultra) until it tells the truth about itself: why its reasoning and answer refuse to come out as two separate fields over the OpenAI API. - [Atlas + Qwen3.6-35B-A3B on a Single DGX Spark: 74 tok/s, 3× SGLang](https://al-engr.com/atlas-qwen36-spark2.html) — May 26, 2026. Atlas inference engine benchmark on a DGX Spark: 74 tok/s on Qwen3.6-35B-A3B-FP8. Pure Rust, MTP speculative decoding, tool calling, and Hermes integration. - [Qwen3.6-27B-FP8 on a Single DGX Spark: SGLang and Speculative Decoding](https://al-engr.com/qwen3.6-27b-spark-sglang.html) — May 26, 2026. A dense 27B FP8 model, one GB10 Spark, SGLang with NEXTN speculative decoding. 22 t/s at c1, 95 t/s aggregate at c8, 170 t/s peak. Full llama-benchy sweep across depth 0–16K. - https://al-engr.com/ms01.html — MS-01 (Forge): full read-only inventory of the rack's GPU-less Docker and agent host. Not overloaded — just loaded. (August 26, 2026) - https://al-engr.com/deepseek-flash-0731-testing.html — DeepSeek Flash 0731 Testing: weights vs thinking level 2x2 on dual DGX Spark (August 26, 2026) - https://al-engr.com/unified-hermes-desktop.html — One Desktop, Two Brains: per-profile remote connections unify Milo (local M4) and Echo (remote MS-01) in one Hermes Desktop (August 25, 2026) - https://al-engr.com/anthropic-oauth-mess.html — Anatomy of an Auth Mess (Aug 24, 2026): credential post-mortem + Hermes observability recommendations - https://al-engr.com/local-llm-fleet-august-2026.html — Local LLM Fleet: August 2026 topology snapshot, live-probed August 24 - https://al-engr.com/hermes-93769-unpairable-tool-calls-notes.html — Two Edge Cases in Dropping Unpairable Tool Calls: technical reference for hermes-agent #93769/#93875 (repro + liftable tests) - https://al-engr.com/pr-work-week-of-20260824.html — PR Work, Week of August 24, 2026: Hermes OSS weekly log (stale-PR rescue, two verified bug candidates) - https://al-engr.com/h2c-first-prints.html — H2C First Prints: Bambu Lab H2C day-one commissioning, first four prints, Gridfinity/Bisley plans, and the material stack for upcoming rack parts (August 24, 2026) - https://al-engr.com/syringe-caddy-design.html — Syringe caddy + drawing station design (OpenSCAD, H2C, August 23, 2026) - https://al-engr.com/local-ai-catching-up.html — Local AI is catching up: 4-lane speed bench, dual-Spark DS4-F vs frontier cloud, August 23 2026 - [Week of August 17, 2026](https://al-engr.com/week-of-august-17-2026.html) — Archive rollup of the week’s published lab work; no new benchmark run. - https://al-engr.com/omlx-pr3059-round4-m3ultra.html — oMLX PR #3059 round 4, tuner run, series-best numbers (August 23, 2026) - https://al-engr.com/omlx-pr3059-round3-m3ultra.html — oMLX PR #3059 round 3, profiled, down/wo_a offloads regress on Ultra (August 23, 2026) - https://al-engr.com/omlx-pr3059-round2-m3ultra.html — oMLX PR #3059 round 2 retest, chunking fix upstream (final, August 23, 2026) - https://al-engr.com/omlx-pr3059-dsv4-ane-m3ultra.html — oMLX PR #3059 DSv4 ANE prefill field test on M3 Ultra (final, August 23, 2026) - https://al-engr.com/ds4-f-keys-anchored.html — Funland DS4-F now on Keys anchored 0731 ablit (August 23, 2026) - https://al-engr.com/inference-box.html — GhettoRak: 8-node DGX Spark cluster on two CRS804 switches; 24 GB/s 8-node all_reduce, 400G inter-switch hop costs nothing measurable; GB300 Station live at 2 × 200G (196 Gb/s per port); 2,200 MHz clock cap → ~1.35 kW; Data Direct 392 Gb/s from GPU memory; next: re-measure power. (updated October 29, 2026) - https://al-engr.com/ds4-f-mia-aug22.html — DS4-F Mia 489af95 recipe pin (not a speed win) - [An Evening Plugin: Local vs Cloud Token Analytics for Hermes Desktop](https://al-engr.com/hermes-usage-pane.html) — Hermes records per-model, per-endpoint token usage locally; an evening plugin turned it into a Portal-style analytics page. Local vs cloud, billing split, cache detail. - [Playing with StackChan](https://al-engr.com/playing-with-stackchan.html) — Desk Milo: talk, look, recognize faces. Face works; camera and speech not on. - [StackChan Gets a Custom Face: mr_r0b0t and the R5a Test](https://al-engr.com/stackchan-custom-face-r5a.html) — Custom baked avatar and narrowly scoped display/base-LED qualification on the spare. - [From XiaoZhi Activation to Local Hermes Control: StackChan Unit #4](https://al-engr.com/stackchan-local-hermes.html) — August 20, 2026: local-only StackChan firmware repair and strict six-tool read-only Hermes proof. - [Ornith 1.5 vs GLM-5.2 on the M3 Ultra](https://al-engr.com/ornith-15-vs-glm52-m3-ultra.html) — sequential llama.cpp A/B; Ornith wins for this box - [Don’t rsync 400 GB of weights to a Synology](https://al-engr.com/mac-nas-ditto-vs-rsync.html) — August 19, 2026: 46 GB cp write 170 MB/s from Forge and M4; reads 17–26 MB/s. - [Ornith 1.5-397B Q6_K on M3 Ultra](https://al-engr.com/ornith-15-397b-m3-ultra.html) — August 19, 2026: 246k prefill 33.8 min / 122 tok/s, decode 29.7 tok/s, 262k serve. - [oMLX Dual-ANE Prefill on Three Macs](https://al-engr.com/omlx-dual-ane-three-mac.html) — August 17, 2026: +19% PP on M3 Ultra, +20% on M4 Max, −31% on M5 Max. - [Honcho 3.0.12 Pin](https://al-engr.com/honcho-3012.html) — August 14, 2026: pinned self-hosted Honcho to tagged v3.0.12. - [DSF-0731-Mia](https://al-engr.com/ds4-f-0731-mia.html) — August 13, 2026: Mia 018c6bc hotfixes on the live dual-Spark 0731 lane. - [https://al-engr.com/hermes-weekly-pr-process-aug-2026.html] Weekly Hermes PR process (week of August 10, 2026) — scout, claim, Opus review, #84867, #84870 comment-only - [MiniMax H3 on Dual DGX Spark](https://al-engr.com/minimax-h3-dual-dgx-spark.html) — Pinned ComfyUI dual-Spark H3 recipe and August 12 dual-lane numbers. - [Supabase — How to Get Value From It](https://al-engr.com/supabase-how-to-get-value.html): When a private research library is filling in Supabase, ship answers with receipts before graphs or sentiment. - https://al-engr.com/lutron-radiora2-switch-install.html — RadioRA 2 switch install: 2-wire vs neutral, dual-path diagram - August 18, 2026: [DS1823xs+ 24 TB RAID 6 Upgrade and Immutable Backup Plan](https://al-engr.com/ds1823xs-24tb-raid6-upgrade.html) - August 6, 2026: [Audio Evidence Without a Video Pipeline](https://al-engr.com/video-ingest-lab.html) - August 5, 2026: [MHR Video Production Pipeline V2: AI Shots, Deterministic Edit](https://al-engr.com/mhr-video-production-pipeline-v2.html) - August 5, 2026: [How We Built MHR 001: Video Production Pipeline V1](https://al-engr.com/mhr-video-production-pipeline-v1.html) - August 5, 2026: [ThirdReality Smart Plug Gen3 to Prometheus: A Local Recipe](https://al-engr.com/thirdreality-prometheus-recipe.html) - July 29, 2026: [DeepSeek V4 Flash DSpark: the Anemll 1M/6 Recipe That Stuck](https://al-engr.com/ds4-f-anemll-1m6-recipe.html) - July 8, 2026: [Cartoon2: Rebuilding the Milo Cartoon Workflow Around APIs](https://al-engr.com/cartoon2-api-workflow.html) - July 7, 2026: [OB1 to Honcho: Migration Complete, Continuity Seeded](https://al-engr.com/ob1-honcho-migration-checklist.html) - July 7, 2026: [Milo Migrating to Hermes?](https://al-engr.com/milo-migrating-to-hermes.html) - July 6, 2026: [MiMo DFlash on Two DGX Sparks: What Worked, What Did Not](https://al-engr.com/mimo-dflash-july-2026-attempt.html) - July 4, 2026: [July DS4-F Status: Patch3 350K/12 Live](https://al-engr.com/july-ds4-f-testing.html) - July 1, 2026: [July SGLang Testing: Qwen3.6 on DGX Spark](https://al-engr.com/july-sglang-testing.html) - June 30, 2026: [June DS4-F Testing](https://al-engr.com/june-ds4-f-testing.html) - June 30, 2026: [June MiMo Testing](https://al-engr.com/june-mimo-testing.html) - June 29, 2026: [Honcho Needs Boundaries, Not Vibes](https://al-engr.com/honcho-provider-compartments.html) - June 29, 2026: [Agent Memory, Shared on Purpose](https://al-engr.com/agent-memory-honcho-migration.html) - June 18, 2026 · updated November 13, 2026: [Milo-Ark: A local AI repo](https://al-engr.com/milo-ark-model-archive.html) — 70 checksummed trees, 27.46 TB on miloshare, nothing waiting on verify. Fourteen trees added since September, including the 1.56 TB Kimi-K3.1-Pro and Muse-Spark-Open NVFP4; DeepSeek-V4.1-Flash-1017 is the first tree with a passed smoke test in its manifest. - June 27, 2026: [Building a Review-Gated Kanban Learning Pipeline for Hermes](https://al-engr.com/kanban-learning-pipeline.html) - June 27, 2026: [Hermes MoA: The Model Council Profiles](https://al-engr.com/hermes-moa-profiles.html) - June 27, 2026: [A First OSS Bug Fix with an AI Coach](https://al-engr.com/first-oss-bug-with-ai-coach.html) - June 25, 2026: [DS4-F Under Three Lights: Tool Discipline, Throughput, and Hermes Fit](https://al-engr.com/ds4-f-three-benchmark-read.html) - June 23, 2026: [DS4-F Aiden for Hermes: Recipe and Benchmarks](https://al-engr.com/ds4-f-aiden-hermes-recipe.html) - [A Field Guide to DeepSeek V4 Flash Thinking Levels](https://al-engr.com/dsf-thinking-levels-guide.html): thinking levels are prompt prefixes not budgets; traps and recommendations with measured receipts - https://al-engr.com/ms01-linux-node-followup.html — The Linux Node, One Week In: the $500 MS-01 was about token cost, not spare compute (April 22, 2026; updated July 7, 2026) - https://al-engr.com/ms01-slave-node.html — Adding an OpenClaw Linux Node: parts list and bring-up for the MS-01 mini PC (April 21, 2026; updated July 7, 2026) Generated by Milo for agent discovery; updated 2026-09-23.