Adaptive Residency, Part 2: On the RTX Sidecar Lane, a Better Map Beats a Live Swapper
1. Two lanes, one question
Both lanes serve DeepSeek-V4.1-Flash: 40 mixture-of-experts layers, 384 experts each, 6 picked per token. Both keep the same 295 experts per layer in GB300 memory, chosen once in September from agent traffic. They differ in where the other 89 go:
- v23, GB300 only. Cold experts sit in Grace memory and are read in place over the chip-to-chip link. Each cold pick costs a full expert read, about 45 µs. This is the lane where adaptive residency took prose from 90 to 189 tok/s.
- v25, GB300 + RTX PRO 6000. Cold experts live on the RTX, which computes them while the GB300 computes the hot ones. Only activations cross PCIe. This is the production lane.
Part 1 ended with "the sidecar uses a fixed map too, so the same counter applies." True. The question is what it's worth there, because on v25 a cold pick isn't a stall on its own: the RTX works in parallel. It only costs time when the RTX finishes after the GB300.
2. The same workloads on all four configurations
Same harness as part 1: 400-token greedy answers, thinking off, medians from 60 seconds into each phase. "Mixed" alternates a Japanese/German essay and an agent-style English answer on every request. For v25, the mixed runs use a held-out prompt set written after the map below was fitted. The in-sample set gave the same v25 numbers within 1% (207.8 and 566.1).
Figure 1. The sidecar lane wins at both concurrencies. A better map adds a little at one user and nothing at four.
| post-60 s medians, tok/s | v23 fixed | v23 adaptive | v25 fixed | v25m mixed map |
|---|---|---|---|---|
| Japanese/German essays, one user (shift protocol) | 90.0 | 189.4 | 186.0 | 206.9 |
| agent English, one user (shift protocol) | 195.7 | 218.5 | 244.3 | 247.9 |
| mixed, one user, aggregate | 133.1 | 194.1 | 208.9 | 221.1 |
| mixed, four users, aggregate | 236.6 | 431.0 | 560.4 | 558.5 |
Read across the first row. A fixed map costs v23 half its prose speed. The same fixed map costs v25 much less: 186 tok/s on prose against 244 on agent English, and part of that gap is just prose being harder for the speculative drafter. The sidecar was already absorbing most of what adaptive residency fixes on v23.
3. Measuring the ceiling before building anything
On v25 every MoE layer hands its cold tokens to the RTX, runs the hot experts, then waits for the RTX's answer. The wait kernel already adds up how long it spins, in nanoseconds, in a shared-memory counter the watchdog reads. Sampling that counter every 5 seconds during the domain-shift protocol gives a direct measurement of what cold experts cost this lane:
| phase, one user | wait per layer | share of wall time spent waiting |
|---|---|---|
| v25, Japanese/German essays | 32 µs | 11.8% |
| v25, agent English | 2.9 µs | 0.5% |
| v25m, Japanese/German essays | 6.1 µs | 0.5% |
| v25m, agent English | 3.6 µs | 0.4% |
If a perfect live swapper drove the wait to zero, prose could gain about 1/(1 − 0.118) ≈ +13%, and agent English almost nothing. Building one is feasible. We checked that both expert layouts can be transformed one expert at a time: on the GB300, DeepGEMM's MegaMoE transform of a single expert produces exactly the bytes of that expert's slice of the whole bank, and on the RTX, b12x prepares its experts in place. But a swap on this lane crosses two processes and two weight formats. It needs a swap mailbox in the shared memory both sides poll, raw expert reads from the checkpoint, and an ordering protocol so neither side computes a half-moved expert. That's two to three days of careful work for a ceiling of 13%.
4. The cheap version: refit the map
Part 1's mixed-traffic run had a useful side result: with both domains alternating, adaptation settled at under 1% cold picks. So one set of 295 experts per layer covers Japanese/German prose and agent English at the same time. And the counter had already been recording every decode pick through that boot, about half an hour of traffic: about 100k prose tokens and 105k agent and tool tokens, 128.7 million expert picks in all.
We took the top 295 experts per layer by that count. Experts that stayed hot kept their rows, and ties went to the old map. On average 42.5 experts per layer changed sides. On the traffic it was fitted on, the cold share of picks falls from 18.2% to 0.88%. Then we booted v25 with that one file swapped, for both the server and the sidecar. Same image, hook, kernels, flags and KV pool (4,400,013 tokens).
On held-out prompts it gives +5.8% on mixed traffic at one user, coming from prose (179.5 → 206.7 tok/s, +15%), with agent requests flat (250.8 → 246.4). The RTX wait on prose drops from 11.8% to 0.5% of wall time, so a refit map captures nearly the whole ceiling a live swapper could reach on this traffic. At four users it's flat. With four requests in a step, the GB300's own work runs longer than the RTX's, so the RTX was rarely the one making it wait.
The obvious limit: a fitted map is fitted to a traffic mix. A third domain this map has never seen (Korean, say, or a codebase with unusual vocabulary) would bring the wait back. Covering that is the live swapper's job, and that's why it stays on the list rather than in the bin.
5. Why this one stays experimental
On v23 we could show the swaps are lossless with greedy parity: 50 fixed prompts, compare outputs, and v23 repeats itself 49/50. On v25 that test is useless. Two captures of the same map, minutes apart, agree on only 14 of 50 prompts. The lane drafts with probabilistic sampling, and the cold experts run on a different GPU with different kernels, so tiny numeric differences change which draft tokens are accepted, and outputs drift from there. v25 against v25m reads 15/50, at that noise floor. It shows no sign of damage, but it can't show the absence of damage either.
The v25 recipe was promoted on stronger gates: a per-layer cosine agreement check, quality benchmarks and a soak. None of those has been re-run on the new map. A refit map changes only which GPU computes each expert, not any expert's weights or math, so we expect them to pass. "Expected to pass" is exactly what the experimental label is for. The sidecar recipe's own 48-hour soak was also cut at about 25 hours tonight to free the lane for this work, so that recipe stays experimental until a clean soak runs.
6. Which recipe to run
| you have | run | why |
|---|---|---|
| a GB300 only | v23 + adaptive residency (experimental) or v23 (verified) | adaptive is +46% / +82% on mixed traffic at one / four users, +111% on off-domain prose |
| a GB300 + RTX PRO 6000 | v25 (experimental) or v25m (experimental) | the sidecar is the faster lane at every concurrency. v25m adds +15% on single-stream off-domain prose |
| traffic that changes a lot | v23 + adaptive | the only one that follows traffic it has never seen |
The Station's :30006 lane is serving v25m tonight, with the same fail-stop watchdog as v25. The v25 containers are stopped and kept for rollback.
7. Recipes and receipts
- GB300 + RTX PRO 6000 sidecar recipe, with the v25m variant. Map file, fitter, held-out and in-sample runs, stall samples, parity captures and the layout-separability test are in
results/2026-10-23-ar5-mixed-map. To launch v25m, setROWMAP=rowmap-mixed-ar-v1.jsonfor both launch steps. - GB300-only recipe, with the adaptive-residency variant: hook, diffs against v19, harness and all receipts in
results/2026-10-23-ar-adaptive-residency.
Credits
The sidecar lane's production shape (the b12x peer protocol, the two-step launch and the harness it was first measured on) is RonanLabsAI's dgx-station-frontier, built on original-el8's hook. The cold experts on the RTX run on b12x, and the hot ones on DeepGEMM's MegaMoE. The device-side planner idea behind part 1 came from vLLM #56177.