For the last two weeks the overnight vLLM campaigns on the GB300 Station have been run by an AI agent end to end: research the upstream changes, write the plan, launch each container, watch it boot, bench it, read the ledger, pick the next rung, and write the recipe in the morning. It works. The bisect that pinned a 1.77% agent-tool regression to vLLM PR #56633 came out of one of those nights. It also cost roughly a hundred times more than it should have, and the reason is instructive enough to write down.
Session 20260920_191542, Sunday September 20, 19:15 to Monday 09:40 CDT. The orchestrator was Claude Fable 5.1 on the Anthropic Max subscription. James's brief was one line: research the upstream vLLM/DeepSeek changes, update the recipe repo, propose a night's ladder, run it on the Station with Grok 4.6 as the worker. Then "go for 24 hour window", "confirm", "yes", and a handful of "status and eta plz" pings in the morning.
The engineering output was real: two bisect rungs, a control rerun, a held-out regression test, a resolved culprit commit, and a ledger with per-rung concurrency curves at C1/C8/C16. Nothing about the result is in question. The cost is.
All four numbers are from the Hermes session store (sessions.cache_read_tokens etc.), not estimated. 62.25M total.
Look at the bar. Output, the tokens the model actually generated, is 182K. That is the thinking: the plan, the bisect interpretation, the ETA tables, the recipe text. It is 0.3% of what was billed. The other 99.7% is the same growing transcript being replayed into the model on every API call, because an agent session is stateless at the API boundary and every wake-up re-reads everything it has seen.
The wake-ups were the killer. The wait pattern was for i in $(seq 1 1400); do ssh ... grep -q "CHAIN END" && break; sleep 30; done running as a background process. Correct, robust, and fine for a shell script. But when that process completes, the agent session wakes with its full ~280K-token context to read a two-line result and decide "next rung". With 194 terminal calls in the night, that is a lot of 280K reads to make decisions a bash case statement could make.
The context itself was bloated for the same structural reason. The research phase pulled full web pages (three of them over 30K characters each) and two complete skill runbooks into the transcript. They were useful for twenty minutes at 19:30. They were then billed on every call until 09:40.
None of this is a Fable problem. It is the natural cost curve of using a single long-lived agent as the loop, the orchestrator, and the reasoner at once. The three jobs have wildly different requirements:
Run a rung, detect exit, append a ledger line. Needs zero intelligence and zero context. Should never touch an LLM.
Read the ledger delta and the plan, pick the next rung, answer "status?", know when it is out of its depth. Needs reliable tool calling and a small, stable context. Cheap or flat-rate.
Write the plan, interpret a bisect that contradicts the previous rung, decide whether an anomaly is real, write the recipe. Needs the best model available, about ten times a night, with a fresh context each time.
Solid arrows are the per-rung cycle; dashed violet arrows are the only paths that touch the expensive model.
Layer 1 already existed. The Station has cardA through cardF runner scripts, a campaign log, and results/ledger.md. The runners boot containers, bench, run the tool-call harness, and write structured lines: cardD rc=0, NIGHT2 CHAIN END, BOOT_FAIL, VOID. Fable spent the night watching files that shell was already writing.
Layer 2 is a Grok 4.6 session on the xAI subscription route, with a PLAN.md as its anchor. James's call: Grok is allowed to improvise inside the plan's budget and stop rules. It does not need per-rung sign-off from the reasoner. It launches rungs, waits with exactly one background process per rung that notifies on completion, answers status pings from the ledger rather than from transcript memory, and escalates when a trigger fires.
Layer 3 is Fable 5.1 as a one-shot. Not a session, not a subagent inheriting context: hermes -m fable51 -t "" -z "$(cat brief.md)" with a usage file to prove the route. Each call sees a brief of at most ~20K tokens (the relevant plan section, the ledger delta, one question) and returns a decision. Fresh context every time, so there is nothing to replay.
The reasoner gets three kinds of touchpoint, and the middle one is rate-limited by design.
| When | Fable does | Input | Calls / night |
|---|---|---|---|
| Front | Writes PLAN.md from James's one-line brief plus a research digest: rungs, budgets, win bars, stop rules, escalate-if conditions. James confirms. | research digest (~50–80K) | 1 |
| Middle | Answers an escalation with a decision and a revised rung list. | plan section + ledger delta + one question (≤20K) | ≤5 |
| End | Turns the finished ledger into recipe PR text and the blog body. | ledger (~30K) | 1 |
Escalation triggers, fixed up front so the orchestrator does not have to reason about whether to reason:
VOID or BOOT_FAIL results on the same card;Budget arithmetic: about ten Fable calls at ~30K each is ~300K tokens, against 57M. Even if that estimate is off by five times, it is still two orders of magnitude. And the reasoner's quality goes up, because each call sees a curated brief instead of a 280K-token transcript with three stale web pages in it.
| Candidate | Verdict | Reason |
|---|---|---|
| Grok 4.6 (xAI subscription) | chosen | Already the profile default; flat rate; proved itself as the worker on the reference night; reliable tool calling over long loops. Matches the standing role split (Grok default autonomous route, GPT-5.5 conservative verifier). |
| GPT-5.6 (Codex subscription) | verifier seat | Good model, wrong job. Use it as the second opinion when Fable and the ledger disagree; also flat-rate, so consulting it is free. |
| DeepSeek-V4-Flash on the Spark pair | ruled out for now | $0 is tempting, but the Sparks are themselves an experiment target, and 131K context is thin for a 24-hour loop that has to survive compaction. Revisit when the 8-node Spark cluster is up and stable. |
| Sonnet via Claude subscription | no | Same Max pool the split exists to protect. |
| GLM-5.3 on the Station's :30001 | no | Shares the box with the experiments. |
The model choice is worth maybe 10× on its own. The session discipline is worth the other 10×. These are the rules baked into the orchestration-split skill the orchestrator loads:
ssh … until grep -q 'CHAIN END' with completion notification. Never a sleep-poll whose every iteration is a full-context call.PLAN.md, do not re-load it.PLAN.md + the last ledger delta. Everything else is disposable.The first night under the split gets a cost line in its ledger: orchestrator session cache-read tokens plus the sum of every Fable usage.json, against the 56.9M baseline. That number gets published here whether or not it is as good as the arithmetic above says it should be. Two things I expect to learn: how many escalations a realistic plan actually produces (the ≤5 budget is a guess), and whether Grok holds a 24-hour plan through compaction without drifting, which is the property that decides whether a local model can ever take the seat.
Token counts, call counts, and timestamps are read from the Hermes session database for 20260920_191542_7c138e on September 21, 2026. Layer-1 artifacts (cardA–F-*.sh, results/ledger.md) were listed live on the Station the same morning. The 300K-token projection for the split is arithmetic, not a measurement, and is labelled as such above. Decisions (Grok orchestrates and may improvise; Sparks ruled out until 8 nodes; separate fable51 alias so the ordinary subagent route stays cheap; Fable writes the plan) are James's, September 21, 2026.