Recipe v2 from the DSFVE inner loop

We gave one GB300 to an autonomous experiment loop overnight. It tested plausible serving changes, rebuilt parts of its own harness when the harness failed, and found no material speed promotion. The outer pass still produced a better artifact: Recipe v2, the Iteration 13 static/no-SPS/checkpoint-NextN recipe, digest-pinned and correctness-hardened.

James Meadlock + Milo · outer pass: gpt-5.6-sol-900k via openai-codex · September 4 evening update: claude-fable-5-1 via anthropic · DSFVE / SGLang / DSpark / GB300

The day before, we published a design for an inference recipe generator: an outer loop would find ideas, choose experiments, and turn receipts into public recipes; an inner loop would run one card at a time on the Station.^[1] The first real campaign narrowed the scope. We skipped the kanban layer, used the filesystem as the handoff, kept the human in charge of stop and restore, and tested the part that mattered first: could the loop produce evidence we would believe? The answer is yes, and the outer pass used that evidence to select the recipe below instead of pretending a loss was a win.

The recipe that came out

Recipe v2 is the configuration exercised by Iteration 13, with Iteration 16 supplying the reversal evidence that the checkpoint-default three-layer NextN setting should stay. It is not a throughput promotion. It is the best supported DSFVE single-GB300 recipe because it removes ineffective or harmful machinery, pins the exact artifacts, and keeps the correctness gates that caught real serving failures.

This is the exact v2 server launch. Adjust only the host-side model, patch, and cache paths; supply the API key at runtime rather than committing it:

docker run -d --name dsfv-dspark --gpus all --ipc host --network host \
  --ulimit memlock=-1 --ulimit stack=67108864 --cap-add IPC_LOCK --cap-add SYS_NICE \
  -e SGLANG_RAGGED_VERIFY_MODE=static \
  -v ~/models/DeepSeek-V4-Flash-Vision-Exp/6821d6ad3681a4b137b066b76094fa82ebd0a380/original:/model:ro \
  -v ~/dsf-vision-exp-gb300/patches/encoding_dsv4.py:/sgl-workspace/sglang/python/sglang/srt/entrypoints/openai/encoding_dsv4.py:ro \
  -v ~/sgl-cache/root-cache:/root/.cache -v ~/sgl-cache/tilelang:/root/.tilelang \
  -v ~/sgl-cache/triton:/root/.triton -v ~/sgl-cache/nv:/root/.nv \
  lmsysorg/sglang@sha256:7ac467a50508b7029a23e846c150998fdd26d95c1cfd377ea7e74e28374486a6 \
  python3 -m sglang.launch_server --trust-remote-code --model-path /model --tp 1 \
    --context-length 1048576 --mem-fraction-static 0.90 \
    --chunked-prefill-size 8192 \
    --cuda-graph-max-bs-decode 64 --cuda-graph-bs-decode 1 2 4 8 16 32 64 \
    --max-running-requests 64 --speculative-algorithm DSPARK \
    --swa-full-tokens-ratio 0.1 \
    --reasoning-parser deepseek-v4 --tool-call-parser deepseekv4 \
    --served-model-name dsf-vision-exp --api-key YOUR_KEY --host 0.0.0.0 --port 30003

The public release carries the human recipe, machine-readable recipe, evidence extraction, and exclusion ledger as RECIPE_V2.md, recipe-v2.json, research/recipe-v2-evidence.json, and research/do-not-retry.md.

First, make the machine boring

Production on port 30003 was stopped and renamed under a dated hold name instead of being removed. The production launcher was left untouched. The experiment target was pinned to model revision 6821d6ad3681a4b137b066b76094fa82ebd0a380, image digest sha256:7ac467a50508b7029a23e846c150998fdd26d95c1cfd377ea7e74e28374486a6, and SGLang source revision 40b3e15ddbd9a1067e181283d9900dd3f4d76ed7. Every experiment used a different container name, and teardown had to leave the port offline, the GPU idle, and the lock free.

That separated two verbs which agents tend to blur:

• STOP cancels the active experiment and starts nothing else.
• RESTORE runs the locked production launcher and its production gates.

The campaign ended on STOP. We did not turn a successful test of the process into a routing change; Recipe v2 is a release artifact, not an unattended restore side effect.

A finished workload is not necessarily an experiment

Twenty iteration numbers appeared in the ledger. Only ten became valid performance evidence. Two were calibration runs, and eight were blocked or aborted. That classification came from frozen decisions and persisted receipts, not from whether a GPU happened to do work.

Three performance conclusions

1. Compact verify-all is neutral here

Iteration 1 made compact verify-all look promising: +5.280% on the primary geometric mean versus the static baseline. One row disagreed, with C16 down 6.303%, so the frozen rule correctly called it inconclusive.

The repeat mattered. Iteration 12 was 3.883% below Iteration 1 on the same primary rows. We then ran static and compact in the opposite order. The compact–static–compact ABA estimate was only +0.901% primary and +0.217% prefill. Both sit inside the neutral band. Compact verify-all was closed as neutral rather than promoted on the strength of one flattering comparison.

2. One NextN layer is a real loss

The official checkpoint declares num_nextn_predict_layers: 3.^[2] A Hugging Face discussion proposed overriding it to one, which gave us a clean, testable community lead.^[3]

Iteration 15 proved the override was active through server information and the exact launch command. It lost 4.700% primary versus the preceding compact control. We immediately removed the override in Iteration 16. The checkpoint default then beat the one-layer run by 3.608%. The default–one–default ABA estimate was −4.093% primary and −1.538% prefill.

That is enough. Keep the checkpoint default of three.

3. A beautiful SPS fit can still make the server slower

DSpark's SPS planner needs a table that estimates step cost as batch size and verify budget change. The first valid legacy calibration produced a table, but its frozen runtime candidate lost 4.327% against the older compact reference. A same-hour no-SPS control improved only 0.464%, so we refused to blame SPS for every bit of historical drift.

Iteration 17 tested the planner's additive cost model across 11 batch-size probes, four fractions and three repeats: 132 cells. The fit looked excellent on paper: R² 0.9996, P95 relative error 0.0200, maximum relative error 0.0487, and every lookup positive.

The shape was suspicious. The verify-budget term contributed at most 0.7196% of median predicted step time, and 22 of 33 adjacent timing transitions ran in the wrong monotonic direction. We kept the artifact because it passed the calibration contract. We did not call it a candidate winner.

Iteration 18 completed a workload but used the wrong decision field name. The evaluator required winner_rule; the card supplied decision_rule. That run is blocked, so its throughput cannot be used to rescue or condemn the table. The corrected Iteration 19 passed runtime and correctness gates and then lost 7.145% primary, with prefill at −0.833%. The additive table is mathematically valid and operationally harmful on this workload.

Alternating controls changed the story

C1 and C4 moved around enough to make good screenshots and bad science. Even C8 could swing hard between nearby runs. The original compact result, the later compact repeat, the static control and the opposite-order compact control made the ordering effect visible.

An adjacent comparison answers, "what happened next?" It does not automatically answer, "what did this flag cause?" Brackets and reversals were the cheapest way to separate mechanism from time. For compact mode we used compact–static–compact. For the NextN override we used default–one–default. Any future apparent speed winner should get the same treatment before promotion.

The harness failed usefully

Eight iteration numbers did not become performance evidence. Most of them made the next run safer.

The classification rule matters: a harness failure is not a model loss. Iteration 18 is the cleanest example. The model served, the workload finished, and correctness passed. The formal comparison was impossible under the frozen card, so the result stayed blocked. Anything else would have taught the loop that enough completed GPU work can waive its own contract.

Correctness gates before speed

Every valid performance run had to pass protocol smoke, real vision, a six-turn replay, C64 repetition, GPU-error scanning, repository sanitation, and teardown. The opaque-identifier gate generated 64 adversarial IDs and required exact byte-for-byte fidelity. One changed, duplicated or missing identifier was an immediate loss.

That gate came from a concrete SGLang failure report where one character in an opaque tool identifier changed.^[4] The multi-turn replay came from an earlier DSFVE integration bug: the image contained the serving-side normalization from SGLang PR 28035 but not the matching DSML encoder change, so nested tool arguments grew one level per replayed turn.^[5] A one-turn tool smoke looked fine. The sixth turn did not.

These are not benchmark decorations. This endpoint sits behind agents. A configuration that returns more tokens per second while corrupting identifiers or replay history is slower in the only sense that matters: it makes the system fail.

Research fed cards, not conclusions

We reviewed fresh SGLang work from September 3 before selecting later cards. The broad multimodal implementation PR remained open.^[6] A merged change added DSpark/DSv4 Flash CI coverage rather than a clean runtime optimization.^[7] Other merged changes were cookbook or configuration work for single-GPU Vision-Exp, multi-node serving and DS4-F deployment.^[8][9][10]

None of those was mixed into the pinned runtime. That would have destroyed the one-delta comparison. Posts from Mia AI Lab and @plotarmordev were recorded as community leads, with exact URLs and verification questions, not promoted to facts because they appeared in a social feed.^[13][14]

What the next inner loop should keep

1. Immutable experiment cards. Pin the image, revision, workload, reference, threshold and one allowed delta.
2. Local validation first. A malformed card should fail before it can touch SSH, Docker or the ledger.
3. Runtime receipts. Prove what the server did, not what the launch request asked it to do.
4. Correctness before throughput. Include opaque IDs and replayed tool turns for agent endpoints.
5. Separate evidence classes. Performance experiments, calibration artifacts and harness failures answer different questions.
6. Alternating controls. Any apparent winner gets a reversal or bracket before promotion.
7. Detached execution. A recurring agent decides what may start; a durable process owns the long benchmark.
8. Teardown as a gate. Every card returns the target to idle. Restoration stays explicit.

Update, September 4 evening: the reopen condition fired, and SPS still lost

The plan was narrow on purpose: collect two independent additive-SPS profiles (P0, P1) on the pinned image, fit tables at M-bin widths 64, 6 and 1 on P0 only, score them on P1, and only then bracket a candidate against fresh Recipe v2 anchors and compact controls. The selection gate was written before any data existed: the chosen width had to cut held-out mean absolute error by at least 50% versus width 64 without worsening RMSE, maximum error, or bias. If nothing passed, the campaign stops at calibration.

Both profiles completed cleanly — 120 cells each (per-rank batch 1–8, five verify fractions, three repeats, M from 2 to 48), and both passed the full tier-0 stack first: multimodal smoke, six replayed tool turns, 64 opaque identifiers, and a 128-case repetition audit with zero flags. The held-out result:

Within-run repeat spread on P1 was 0.296 ms, roughly twice the entire prediction error. The finer widths were worse on every metric. On the P0 self-fit they had looked marginally better (0.1146 versus 0.1215 ms at width 1), which is the signature of bins fitting noise; the independent profile exposed it.

The reason is physical rather than statistical. The additive table models verify-step time as a bias plus a per-request term in batch size plus a per-token term in M. On one GB300 at TP1 the per-token term is essentially zero: batch 8 takes 16.3 ms at M=16 and 16.2 ms at M=48; batch 1 moves from 8.65 to 8.66 ms across its whole M range. A scheduler that budgets verification by M is budgeting a variable that does not move the clock here. That is consistent with all three earlier SPS losses (−3.6%, −4.3%, −7.1%) and it is a better explanation than any of them offered.

The harness earned its keep again, in the unglamorous way. The first P0 attempt died at round 36 because the dispatcher was a child of the agent session that launched it; when that session hit its context limit, SSH closed, the remote tee took SIGPIPE, and the cleanup trap fired. The user-scope hard-stop timer died the same minute for the same reason. The rerun moved dispatch under launchd and the hard stop to two system-scope transient timers (a STOP latch at 06:00 and container enforcement at 06:15). Then the gate itself was wrong: it derived the expected M with a formula that disagreed with the pinned profiler at batch 2, so a perfectly good P0 came back BLOCKED. Each fix was a contract revision with re-frozen hashes and 29 passing tests; the released P0 card stayed byte-identical. A second model (GPT-6 Astra) reviewed the recovery plan before it ran and correctly insisted that the salvaged 36 rounds were evidence, not a run.

Campaign contracts, the pinned profiler, all three fitted tables, the held-out manifest with hashes, and the revised do-not-retry entry are in the campaign directory of the public repository.

Why this is an outer-loop success

The inner loop found no production speed win. The outer pass still produced the thing a recipe generator is supposed to produce: a recipe whose claim boundary is honest enough to run.

It refused the first compact number because one primary row disagreed. It refused synthetic correctness. It refused to turn a generated SPS table into a speed promotion. It refused an otherwise completed run because the evaluator contract was malformed. It rejected a community suggestion only after proving the override was live and reversing it immediately. Then it preserved the usable conclusion: choose the simpler static/no-SPS/checkpoint-NextN recipe and say exactly why.

The purpose of an autonomous experiment loop is not to keep producing green cards until dawn. It is to make unattended work less likely to fool the person reading it in the morning. On that standard, the inner loop worked, and the outer loop did the review job it was designed to do.

The sanitized harness, audit generator, full campaign retrospective, and Recipe v2 files live in the public DSFVE GB300 recipe repository.^[11] The locked September 3 production recipe and its original performance/correctness work remain documented separately.^[12]

Sources

1. Designing an Inference Recipe Generator/Optimizer, J&M Labs, September 3, 2026.
2. DeepSeek-V4-Flash-Vision-Exp checkpoint config, Hugging Face.
3. DeepSeek-V4-Flash-Vision-Exp discussion 11, Hugging Face.
4. SGLang issue 34959: corrupted opaque tool-call identifier.
5. SGLang PR 28035: tool-call argument normalization.
6. SGLang PR 37253, broad DeepSeek V4/Vision implementation work.
7. SGLang PR 37665, DSpark/DSv4 Flash CI coverage.
8. SGLang PR 37492, Vision-Exp single-GPU cookbook.
9. SGLang PR 37737, Vision-Exp multi-node cookbook.
10. SGLang PR 37301, DS4-F deployment cookbook.
11. jmeadlock/dsf-vision-exp-gb300, public recipe and campaign repository.
12. DeepSeek-V4-Flash-Vision-Exp on One GB300, locked production recipe and measurements.
13. Mia AI Lab community lead, September 2026.
14. @plotarmordev community lead, September 2026.

Measured September 3–4, 2026 on one DGX Station GB300. The outer-pass synthesis and post revision were authored by Milo using gpt-5.6-sol-900k via openai-codex, from persisted machine receipts and a programmatically generated 0–19 audit. Credentials and connection details are excluded. Experimentation ended after Iteration 19; no Iteration 20 was authorized.