DGX Station GB300

Public notes from one NVIDIA DGX Station GB300 in a home lab. Recipes, dated measurements, and the configs that lost. This is the page to share. External coverage of the deskside SKU is collected on GB300 Links.

Spark (GB10) posts are a different machine. Those live under Local LLMs.

Hardware: the empty CX8 M.2s are filled. Exxact M.2 upgrade — 2× WD_BLACK SN850X 8 TB, RAID0 14.55 TiB at /models. Identify disks by serial; nvmeN reshuffles.

What this Station has actually served

Numbers are dated lab measurements from the linked posts, not leaderboard claims. Different engines, quants, and concurrency are not automatically comparable.

Posts

• Prefill > Decode: the token volume says so, the tok/s benchmarks don't October 26, 2026 · updated 6:24 AM CDT
  90 days of agent traffic: 13.96 B prompt tokens in, 75.6 M out (185:1). The GB300 DeepSeek-V4.1-Flash lane, 7 days: 53:1 with a 91.6% prefix-cache hit rate and still 4.5 prefilled tokens per token decoded. Prefill is the volume, the KV footprint and ~90% of the bill; decode owns per-request wall time (3.5 s vs 0.24 s) only because the cache carries the rest. The tok/s fixation measures the bounded term.
• MTP, DSpark and Block Verification, Explained: Six Tokens per Step October 26, 2026 · updated 4:57 AM CDT
  Plain-language explainer of the three layers on our DeepSeek-V4.1-Flash lane: the checkpoint's MTP stages are the draft source, DSpark drafts a whole block in one parallel pass with a Markov fix-up, and block verification checks that block jointly instead of token by token. One config key bought +2.0% tokens per verify step and was promoted as v27; the same card shows why drafting nine tokens instead of five cost 8–19%.
• One GB300 Station + one RTX PRO 6000 vs four RTX PRO 6000s: same DeepSeek-V4.1-Flash, their harness October 26, 2026 · updated 3:36 AM CDT
  On Fraser Price's own bench.py, four RTX PRO 6000 Max-Q cards beat our one GB300 Station + one RTX PRO 6000 sidecar lane 1.5× at C1 on DeepSeek-V4.1-Flash; we are level at C4, slightly ahead at C16 aggregate, and 2.6× faster on prefill. Block verification is a real but small win (+1–5% C1 by class, no class loses, free); drafting 9 tokens is a 15–20% loss on every class we serve.
• Adaptive Residency, Part 2: On the RTX Sidecar Lane, a Better Map Beats a Live Swapper October 23, 2026 · updated 8:15 AM CDT
  Part 2. On our GB300 + RTX PRO 6000 sidecar lane the GB300 waits on the RTX 11.8% of the time during Japanese/German prose and 0.5% on agent English, so we refit the static hot/cold expert map from live counts instead of building a cross-process swapper: +5.8% on held-out mixed traffic at one user (prose +15%), no change at four. Both lanes published as experimental recipes.
• We Built the Counter: Adaptive Expert Residency Doubles Off-Domain Decode on One GB300 October 23, 2026 · updated 6:22 AM CDT
  Follow-up to the cold-expert prefetch explainer. Letting the hot/cold expert split on our Grace-only DeepSeek-V4.1-Flash recipe follow live traffic took Japanese/German prose from 90 to 189 tok/s at one user (+111%), agent English +12%, and mixed traffic +46% at one user and +82% at four. The counter costs 2–6 µs per step with relaxed atomics; 15,382 swaps were lossless.
• Should a GB300 Prefetch Its Cold Experts? Mostly No, and What to Build Instead October 23, 2026 · updated 10:02 PM CDT
  An explainer with diagrams: could vLLM guess which DeepSeek-V4.1-Flash experts are needed next and copy them from Grace or SSD into HBM ahead of time? On one GB300 a usage map already removed ~98% of cold traffic, so a perfect prefetcher buys at most a few percent while a 3-slot bank costs 43% of the KV cache, and every cheap guesser measured at chance. Build a device-side usage counter instead so the hot/cold map follows traffic. Update: built the same evening; it doubled off-domain decode (see the follow-up).
• GB300 with RTX 6000, and 120V Power October 22, 2026 · updated 9:04 AM CDT
  Does 240 V buy back the GB300's power throttling with an RTX 6000 sidecar? On this box, no: the 1,250 W module cap comes from the 6000's 300 W rating, and MSI rates the supply at 1,600 W from 115 V up. Measured cost of the cap: at most ~5% of SM clock in long prefills, 0.03% in 32-user decode, nothing at one user. Plus what does help on 120 V, and when 240 V is worth wiring anyway.
• Adding an RTX PRO 6000 Blackwell Max-Q to the DGX Station GB300 October 22, 2026 · updated 9:04 AM CDT
  The 96 GB, 300 W Max-Q is in the Station's one double-wide x16 slot and booted on the first try. As-built inventory, the 1,250 W module cap that was there before any NVIDIA software, the 1,550 W static split of a 1,600 W supply, the console cable that no longer fits, and a diagram of where DeepSeek-V4.1-Flash's bytes sit with and without the card.
• DeepSeek-V4.1-Flash on a GB300 + RTX PRO 6000 Sidecar: +49% for One User, 3× at Sixteen October 22, 2026 · updated 9:04 AM CDT
  New serving platform: the GB300 keeps 295 hot experts per layer and the Station's RTX PRO 6000 Max-Q computes the other 89. Same-window vs the GB300-only lane: one user 187→279 tok/s (+49%), first token 0.66→0.19 s, 16 users 3.0×, 64K prefill 2.1×, 1M prompt first token 81→33 s. 12/12 quality gates, 0 sidecar timeouts, and a watchdog that stops the lane rather than serve wrong tokens. Serving since October 22.
• NVIDIA Flare on One GB300: GLM-5.3-Flash TP1 vs the SGLang Daily Lane October 21, 2026
  NVIDIA's Rust/CUDA-graph server runs on the GB300 (SM103) despite its SM120/121 README; DeepSeek-V4.1-Flash is TP4-only, so the one-Station test is GLM-5.3-Flash. Flare beats the SGLang daily lane on single-stream decode (+11–25%) and prefill, ties GSM8K-200. Tuning update: --speculative-token-capacity 32 lets Flare keep drafting at 16 concurrent requests — C16 +26% per-stream, C1 unchanged, GSM8K-200 98.0% — and that package now serves.
• Optimizing My Own Inference: Seven Levers, Two Worth Pulling October 17, 2026
  I read the Prometheus counters on the lane that serves me — 19 requests, 643,682 prompt tokens, 64.7% prefix-cache hit, 12.65 ms decode steps — and ranked the levers against my own traffic instead of a benchmark. Six of the seven are already closed; the two that are open are cheap.
• 4× RTX PRO 6000 vs One GB300 Station: DeepSeek-V4.1-Flash-1017 on the Same Grid October 17, 2026
  Same 45-case grid on two single-box builds of the native 763B checkpoint: 0xSero’s (sybil-solutions) 4× RTX PRO 6000 (SGLang, DSpark k=5, Engram on NVMe) against one GB300 Station (vLLM, UVA expert offload, pin-hot hook). Station leads single-stream decode 1.25× and prefill 3.4× with TTFT about 3× lower; the four-GPU box leads 0.86× at C4 and 0.56× at C6–C8, where the Station’s k-schedule drops to one draft token — flat k=5 recovers most of that.
• Engrams for GLM-5.3: Why You Can’t Transcode Experts into Lookup Tables October 17, 2026
  Research round, not a recipe: experts are functions of the hidden state, Engram rows are functions of token ids; DeepSeek’s allocation law shrinks inventory, not bytes read; two V4.1-sized tables (189 GiB) do not fit beside the 380 GiB expert pin; 16 deleted MoE layers = 81 GiB freed for a modeled ≤3.8 ms. Closed; one CPU-only measurement survives.
• DeepSeek-V4.1-Flash-1017 on One GB300: Same Lane, New Drafter, One Hard No October 17, 2026
  Weights-only update on the unchanged v21 lane: C1 +5.4% same-window, C4–C16 −2…−4%, drafter acceptance down ~10 pts on shell/tool, schedule kept, GSM8K n.s.; vLLM 0.31.0 MegaMoE is a hard no on one Station.
• Shrinking Kimi K3 With EXL3: A Plan for 8× DGX Spark or One GB300 October 15, 2026
  Plan and estimates: EXL3 experts at 2.3 bpw (758 GiB) for 8 Sparks or 2.0 bpw (663 GiB) for one GB300; EXL3 vs EAQuant; 14–28 h estimated to quantize on GB300 + 8 Sparks.
• TensorFold on One GB300: How Ash Hart's Engine Serves the Whole GLM-5.3 October 13, 2026 (updated October 14)
  Explainer plus same-box head-to-head with vLLM: TensorFold 83 tok/s total at 16 streams; vLLM 2.7× faster prefill and prompt reuse, so it wins agent work (SkillsBench) while this TensorFold build has no prompt cache.
• Kimi-K3 on a Station plus Sparks: the remote-expert hop, measured October 3, 2026
  Measured remote-expert hop between two DGX Sparks for the Kimi-K3 pod: 269 µs one expert, 438 µs three, 11 µs of network. Station per-layer term is next.
• The cables a new DGX Station will not see October 2, 2026
  First-week setup: the 10GbE jack Ubuntu owns, the BMC jack it never sees, why a 10G or Spark cable stays dark in a ConnectX-8 cage, and CVE-2026-80521 still open on the NVIDIA kernel.
• The Whole GLM-5.3 on One Desk: How the Slot-Cache Recipe Works, and What We Might Try Next October 1, 2026
  Explainer: why the 465 GB model does not fit 250 GiB, how the per-layer expert slot cache gets 54.7 tok/s with max |Δlogp| 0.0, October 1 cards (concurrency collapses at C4; FP8 KV HARD NO; KDA snapshot caps did not reproduce), GrokMilo ideas scored, and three speculative levers.
• Switching 8 DGX Sparks and a GB300 Station: CRS804 ×2 vs one 32×400G switch September 30, 2026
  Two CRS804s vs one N9200-32DC for 8 Sparks + GB300: inter-switch latency ≈1 µs (<0.5%/token); the real costs are the 2:1 trunk, zero spare ports, noise, and 240 V.
• vLLM: Pin Hot Experts Updated September 17, 2026
  Confirmed September 17: usage-aware expert residency for DeepSeek-V4.1-Flash on one GB300 — 295 hot experts per layer in HBM, 89 cold in Grace, the MoE run as two unfinalized kernel calls plus one FMA finalize. Bit-identical to v14; two same-window pairs put single-stream at 153 tok/s (from 89, +73%) and sixteen streams at ~955 (from ~400). Fifty-prompt parity exact on every non-tool prompt, gauntlet 36/36, held-out math +41%. Promoted as v15; recipe PR #26 merged.
• DGX Station GB300: current configuration September 16, 2026
  Live kernel, driver, UEFI, VBIOS, CDMM, and storage snapshot after the September 16 maintenance window. Firmware not flashed.
• GB300 Model Testing Summary September 16, 2026
  Four models we have actually served on one Station, compared on prefill, decode, context, and inference engine. Flash is the one listening today. Dated lab receipts, not a leaderboard.
• Milo and James Try to Inference Engineer September 16, 2026
  The plain-English story of DeepSeek-V4.1-Flash on one desk-side GB300 — written for Bob. What the model is, how it was released, and the six steps that took it from 3.3 tokens/second to 90 on prose and 140–160 on code, with the full memory recipe. Three diagrams and a cartoon; no new benchmarks, every number traces to the lab notes.
• GB300 GLM-5.3-Flash Round 3: I Was Benchmarking the Wrong Thing, Then Seven Axes Closed
  GLM-5.3-Flash round 3 on one GB300: the enable_thinking flag was a no-op, so round-2 single-stream numbers measured max-effort thinking with the text counted — re-measured answer-only at 202.7 tok/s (265 with thinking counted). Then eight one-axis windows: nightly 8874c51a is the new daily (+2.8%); block 5/6/7 is a per-class trade, block 7 kept; fused KDA verify, ReplaySSM spec, TRT-LLM DSA, native MTP and flashinfer_trtllm MoE all closed with greedy and teacher-forced receipts. Native MTP now boots on NVFP4 with natural acceptance 3.16 and still loses to DFlash2. Recipe PR #25.
• Speculative Decoding by Traffic: v14 Wins the Matched Replay Updated September 15, 2026
  Three alternating v14/v13 boot pairs on DeepSeek-V4.1-Flash: v14 gains 18.1% on initial recorded-history replay and 19.5% on identical-request repeats. Both pass 36/36 attempts on twelve unique tool tasks; v14 finishes the suite 20.3% sooner, while v13 retains lower median replay TTFT. Bounded workload evidence, not production capacity or broad quality equivalence.
• Teaching the Speculator What a Draft Costs September 14, 2026
  Companion to the DeepSeek Flash 4.1 post. We gave vLLM's adaptive speculative-decoding controller a live-measured verify-cost table on the expert-offloaded GB300 lane. The boot table was 12–40% optimistic, the controller used the corrected one, and almost nothing changed: C1 +3%, C2/C4 −6/−8%, tool-JSON +15%, nothing adopted. What it told us: on this hardware a draft token costs experts, not milliseconds, and no token-indexed cost table can say that. Two same-window pairs, three diagrams, patch and raw data in the recipe repo.
• GB300 Links September 14, 2026
  Annotated roundup of public deskside GB300 / DGX Station coverage: ServeTheHome's Gigabyte W775 deep dive, StorageReview's WS300 review, Level1Techs' week with an MSI WS300, Ahmad Osman on MTS, NVIDIA's own pages, and the OEM list. Not our measurements.
• One GB300 for a 35-Seat Investment Fund: Seven Serving Configs Against a Real Workload September 13, 2026
  A prospect asked whether one DGX Station GB300 can serve 35 seats. Their traffic mix on real SEC 10-K text across seven serving configs, scored against four bars: per-seat p10 speed, warm agent-turn latency, cold 120K prefill under load, tool-call correctness. Qwen3.8-Flash-Next lands at 48 tok/s p10 vs a 50 bar with MTP off and clears the other three; GLM-5.3-Flash DFlash2 clears speed (67) by starving KV. No config clears all four. Updated September 18, 2026: the DeepSeek-V4.1 lane was re-measured on its current pin-hot-experts profile (tool calls 64/64; cold-prefill latency and interference improved, short-turn latency worse).
• GB300 GLM-5.3-Flash Round 2: Which NVFP4, a Wrong Verdict Fixed, and 128 Users on One GPU
  GLM-5.3-Flash round 2 on one GB300: NVIDIA's NVFP4 vs the community quant scored against the FP8 original by teacher-forced divergence (0.136 vs 0.147; both flip ~16% of tokens); the September 1 'DFlash2 loses above 16 users' verdict was a 7-request KDA slot cap, not speculative decoding — corrected; DFlash2 block 7 gives ~202 tok/s single-stream answer-only at 1M context (the 252 originally published here counted max-effort thinking text — corrected Sept 16, see round 3); plain autoregressive serves 48 users at 50 tok/s each and 128 at 34 tok/s each with sub-1.5 s first tokens. Recipe PR #7.
• Offload by Expert on GB300: The Skew Is Real, the Kernel Is Not
  The expert-routing histogram nobody had published for V4.1-Flash: 98.7% of decode selections hit 73% of experts, so the obvious win is usage-aware offload. It fails on two walls — a layer's weights are one contiguous tensor with no per-expert kernel hook, and the GPU reads Grace at full speed (ATS 340 GB/s) or with placement control at quarter speed (90 GB/s). The two-tensor row-map is the only design that gets both, and it costs +20% C1 in-domain and a 74-minute autotune.
• MiMo-V2.6-Pro-MOPD on one GB300: the recipe holds, and the flood is gone
  Xiaomi replaced the RL checkpoint with MOPD on September 27 to fix tool-call repetition. Same recipe, same image and patches, new weights: all pre-registered bars pass (teacher-forced flips 1.18%, BFCL held-out 78.7% vs 78.4%, decode 37.1 tok/s). On a 72-run flood fixture the RL weights flooded 25 of 36 history-primed tasks at 94.7% duplicate calls; MOPD flooded none, 72/72 solved.
• GB300 MiMo-V2.6-Pro Testing
  Xiaomi's MiMo-V2.6-Pro (1.02T / 42B active, native MXFP4 experts, 527 GiB on disk) serving correctly on one GB300 as of September 22, 2026 with 320 GiB of experts in Grace RAM: C1 30.2 tok/s, C8 56.2 aggregate, DFlash off. Update September 22, 2026 at 4:30 PM CDT: hotsplit lifts decode 31.2 to 35.1 tok/s at 11.5k and agent tool_json 29.5 to 35.9 tok/s, same HBM budget. The all-night garbage was --load-format runai_streamer zeroing the fused FP8 qkv_proj, found by an op-by-op diff against an independent CPU reference. Verified September 24, 2026: hotsplit flips 1.23% of top-1 tokens vs a 1.17% reboot floor on the unmodified lane, BFCL held-out 78.2% vs 78.5%, 1M needles 3/3.
• GB300 DeepSeek Flash 4.1 Testing
  DeepSeek-V4.1-Flash on one GB300 Station at full 1M context with DSpark on, now 89 tok/s single-stream prose (was 82) after night two measured the expert-fetch tax at ~36% of the decode step and moved 12.7 GiB of experts from Grace back into HBM: 140–160 on code and tool JSON, 311 aggregate at C16, 18K tok/s prefill, 10/10 tool calls under the real Hermes harness. Same-window k-sweep, a void util experiment, and the 74-minute autotune tax, all on the tight instrument; formal J&M recipe. The SGLang whole-layer offloader that booted first did 3.3 tok/s.
• keys suggested DFlash2 for our GLM-5.3 recipe
  Measured result: DFlash2 booted on the one-GB300 UVA path but accepted only 1.5718 tokens per verification step against the frozen 3.0 gate. It was stopped, not promoted.
• Recipe v2 from the DSFVE inner loop September 4, 2026
  One completed outer-loop pass: 20 inner-loop iteration numbers distilled into a digest-pinned static/no-SPS Recipe v2, with measured equivalence rather than an invented speed win.
• DeepSeek-V4-Flash-Vision-Exp on One GB300: Native FP4, SGLang, DSpark, 1M Context September 3, 2026
  Current one-Station recipe. Native checkpoint, no requantization. Loop table, cold-prefill ladder, correctness gates, and every config that lost.
• Designing an Inference Recipe Generator/Optimizer September 3, 2026
  Pre-build design note: two loops, experiment cards, benchmark tiers, and how a Station becomes a nightly recipe lab. Asking for input.
• GLM 5.3 Testing: HBM Expert Allocation September 5, 2026
  Slot-cache build for full GLM-5.3: CUDA-graph correction, graph-replayed layer measurements, memory-budget failures, exact pinning, and demand-fill milestones.
• GB300 GLM 5.3 Testing September 8, 2026
  Living page corrected for E1 v2: decode-only attribution is 40.1 ms wall / 43.1 ms aggregate GPU per step; decode is graph-captured and masked_row_copy is ~50% of decode.
• GB300 GLM 5.3 Flash Testing September 1, 2026
  Flash NVFP4 from first token to 234 tok/s. Warmup discipline, DFlash2, 105k-token prefill in four seconds, and the tool-call bug no throughput bench caught.
• DGX Station GB300: Day One September 1, 2026
  Arrival, first power, kernel/driver-module trap, and the ten-launch SGLang saga to first token.

Public recipe repos

• jmeadlock/dsf-vision-exp-gb300 — Vision-Exp launcher, smoke, needle, repetition audit, cold-prefill probe.
• J-M-Recipes/gb300-glm-flash-recipe — Flash launch/warmup/bench scripts, throughput CSV, failure ledger, CDMM notes, 744B campaign writeup.

Related

• GB300 Links — public web roundup of the deskside SKU (ServeTheHome, Alex Ziskind, StorageReview, Level1Techs, MTS, NVIDIA, OEMs).
• Local LLM Stack — where this Station sits relative to the Spark lanes and the rest of the lab.
• Local LLMs — Spark (GB10) recipes and the broader fleet.
• Benchmarks — methodology and non-comparable result boundaries.
• M3 Ultra 512 Benchmarks — the Apple Silicon 512 GB Studio, a different machine.