When a tool call loses its first line: testing DeepSeek V4.1's missing DSML opener on three engines
On November 22, 2026, Christopher Owen, who builds the spark-ds41f engine for DeepSeek-V4.1-Flash on DGX Sparks, published a careful report of a failure one of his users, Herlon, hit 598K tokens into a coding session: the model wrote a bash tool call with its first two lines missing, so the client received text instead of a call, twice in a row. He traced how his own earlier fixes interact with it, and then asked the questions he could not answer from one transcript: does the model drop the opener or does his stack, does it also happen at 4K, and which of three possible fixes should he take. We have the same model running on three different serving stacks in one room, so we spent the day measuring. Our three answers, in plain words: (1) we could not make the first failure happen from a clean history; (2) once two broken turns are in the history, the model copies the broken format on all three engines, so the loop is the model, not his stack; (3) dropping or rewriting the broken turn stops it, and stripping its markup does not. The harness and every failure with its token ids are public in dsml-opener-sweep, and we filed the findings as spark-ds41f issue #1305.
In plain English
- What DSML is. DeepSeek V4.1 writes its tool calls in a small markup called DSML, and the marker that opens every tag,
|DSML|, is a single special token to the engine (id 128825), not four characters of text. - What went wrong in Herlon's session. 598K tokens in, the model thought, wrote one sentence, then wrote a bash call with no opening line and no tool name, only the parameter and the closing tags. The client got a sentence and some stray markup, no tool call, twice in a row.
- Answer 1: we could not make it start. With a clean history, 0 of 600 requests at 4K to 800K tokens produced a missing opener, on three engines, at effort high and max. We say "could not reproduce", not "does not happen".
- Answer 2: once it has started, the model keeps it going, on every engine. Put two broken turns into the history, encoded as plain text the way every engine does today, and the model writes another one in 2.4% of replies at effort high and 7.3% at effort max. Owen's engine, our stock-vLLM GB300 lane and an SGLang build all show it. The loop lives in the model's habit of copying its own history.
- Answer 3: keep the broken turn out of the history. Deleting it: 0 failures in 576. Rewriting it as a real call plus a "not executed" result: 0 in 456. Stripping only its markup and keeping the sentence: no help, 3.8%, the same as leaving the broken turns in at those depths, because the model then copies turns that announce a tool and never call one.
- Every failure that tried to write a call is missing the same line. Across the 52 failures that carry any call markup (Shapes A and B below), the
invoke name=line is gone every time, and wherever we have token ids the parameter tags are typed out in ordinary tokens, not the real one. - Does it happen at 4K? Rarely: with two broken turns in the history, about 0.4% at 4K to 64K against 4 to 6% at 200K to 800K on the two vLLM engines. Deeper sessions copy more, but short ones are not immune.
- Side finding. Our own GB300 lane and the SGLang build both still had Owen's older bug 3 (a quoted, typed-out call in a code block became a real call). We ported his real-token rule to the GB300 lane the same morning; it now passes 42 of 42 of those probes.
- Bottom line: the parser should keep trusting only the real token; the client should never send a broken turn back; and the server can help by refusing to return a half-formed call as plain text.
The token that is not text
A language model does not see characters; it sees tokens, and some tokens are special. DeepSeek V4.1 writes tool calls in a markup called DSML, and the marker |DSML| that opens every tag is one special token, id 128825. The same characters can also be typed out as ordinary text, in which case they arrive as four plain pieces: |, DS, ML, |. Think of a real signature against the word "signature" typed in the same font: they look alike on the page, but only one of them is a signature. Owen puts it this way in his report: everything is about telling those two apart, usually deep into a long session.
A correct call reads, line by line: <|DSML| calls>, then <|DSML| invoke name="bash">, then one or more <|DSML| parameter name="command" string="true">…</|DSML| parameter>, then </|DSML| invoke> and </|DSML| calls>, each marker a real token.
Figure 1. One token vs the same characters. Top: a real call, every marker the single token 128825. Middle: the same characters typed out as ordinary pieces, which the engine must treat as prose. Bottom: the broken turn in Herlon's shape, with the two lines that carry the structure missing and the rest typed out.
Owen has already fixed several ways these two can be confused. On October 20 (release 1022, with contributor Acermax, issues #446 and #449) text inside a tool result could open a tool call or close thinking; since then spellings are no longer read as control tokens. On November 4 (joesinvestments, #876), asking the model to show a sample call in a code block made the call, 6 of 6 times when a tool was offered. On November 6 (koldfrontier, #948), a file containing a DSML closer came back cut short with no error. His fix for the last two, in release 1120 and as an open upstream vLLM pull request (#65019), is that structure counts only where the real token is. He numbers these fixes 1, 3, 4 and 5, and we use his numbering below.
The new case is Herlon's: a coding agent 598K tokens into a session. The model thinks, writes one sentence, then a bash call with its first two lines missing. The client gets text and no tool call, twice in a row. Owen's theory: the client sends the broken turn back as history; fix 1 correctly encodes it as plain text; its tail sits inside the last 128 tokens, the sliding window every layer reads exactly; if the model copies it, the copy is ordinary tokens, and fix 5 correctly rejects it. In his words, every earlier fix is doing its job, and together they hold the session in the broken state. His three options were: (a) accept a call with no opener, which he called "bug 3 again, from the other side"; (b) keep a broken turn out of the history; (c) hold the opener in place with a grammar whenever tools are offered.
How we tested
Three serving stacks, one model. All three run the same base DeepSeek-V4.1-Flash-1017 weights (the Spark pods verified by identical shard hash; the GB300 runs the stock checkpoint), measured on November 22, 2026.
| engine | hardware | tool-call parser | speculative decoding | sweep requests |
|---|---|---|---|---|
spark-ds41f 1120 (Christopher Owen; image candidate-62 = release 1120), his tp4-switched shape | 4 DGX Spark (GB10) | token-provenance parser (his fix 5) | copy drafts on (engine log) | 2,376 |
| stock vLLM 0.31.0 + Garner McCloud's dgx-station-recipes overlay, our production lane | 1 DGX Station GB300 | text-based DSV4.1 parser plus our closer rule | DSpark, up to 5 drafts, no copy drafts | 2,376 |
| SGLang, knapcio dsv41-4x-spark v2.5 | 4 other DGX Spark | SGLang's parser | — | 1,416 |
Same sampling everywhere: temperature 1.0, top_p 1.0, thinking on, reasoning_effort high (some cells at max), max_tokens 8192, requests alternating streaming and non-streaming. All three engines render each test history to the same token count within 8 tokens, and all three keep typed-out DSML in user text, assistant history and tool results as plain text, which we checked through each engine's /tokenize: fix 1 is present everywhere. In total 6,168 sweep requests (2,376 on 1120, 2,376 on GB300, 1,416 on SGLang) plus 126 short probes (42 per engine: 7 probe rows × 6).
The test session is a synthetic coding agent: a system prompt and one user task (a scheduler test started failing; find and fix it; work autonomously), then many turns of read_file, grep, write_file and pytest calls, each with a short sentence and reasoning. The tool results are real files and real grep output from the public vLLM v0.31.0 source. Every history ends on a failing test, so the natural next move is a tool call. We cut it at six depths; the measured prompt tokens on the 1120 engine for the clean history were 3,095 · 67,161 · 200,748 · 405,825 · 604,410 · 812,321, which we name by target below: 4K, 64K, 200K, 400K, 598K, 800K.
The history variants:
- clean: ends at the failing test.
- poisoned: plus two broken assistant turns in Herlon's shape (a sentence, then the parameter and closing tags typed out, no opener, no invoke line), each followed by a user "continue".
- poisoned1: only one broken turn.
- poisonedfar: the two broken turns, then 3 normal tool turns (9–15K tokens) before the end.
- strip: the broken turns kept but cut to their sentence, no markup.
- drop: the broken turns removed (clean history plus one "continue").
- repair: each broken turn rewritten as a real bash tool call plus a tool result reading "[not executed: … malformed …]".
- dsmlctx: clean, plus 6 reads of DSML-heavy DeepSeek encoding docs and fixtures spread over the last 30% of the history.
- dsmltail: clean, but the last tool result is a DSML encoding fixture full of typed-out calls.
A "failure" is a reply with no valid tool call because of tool-call formatting. Four long, honest "I could not fix it" replies are excluded; they are not formatting failures. Rates below carry 95% Wilson intervals.
A clean history never broke
This is the answer to Owen's first question, and it is the one we are least able to turn into a strong claim. From a clean history the missing opener did not appear once: 0 of 504 at effort high (interval 0–0.8%) and 0 of 96 at effort max (0–3.8%), so 0 of 600 across both. We also tried to tempt it. With DSML-heavy documentation in six tool results over the last 30% of the history (dsmlctx), 0 of 528. With a fixture full of typed-out calls as the very last tool result (dsmltail), 1 of 528, and that one was not a missing opener: on the GB300, reasoning that quoted the format got parsed into a call named "...", a parser bug on our lane. Whatever triggered Herlon's first broken turn, our synthetic session does not contain it.
Two broken turns teach the model the broken format
Owen's second question was whether the self-feeding loop is the model or his stack. Two broken turns in the history, in the exact Herlon shape and encoded as plain text the way fix 1 does, made the model write another one in 37 of 1,560 replies at effort high, 2.4% (1.7–3.3%), and in 7 of 96 at effort max, 7.3% (3.6–14.3%). Those pooled rates include the short depths. Counting only 200K–800K, two broken turns gave 35 of 912 (3.8%, 2.8–5.3%) and one broken turn gave 7 of 576 (1.2%, 0.6–2.5%), so the effect grows with the number of broken turns.
Figure 2. The loop. Each step is doing its job: the parser rejects typed-out markup, the client saves what it received, the engine encodes it as text. The model then copies what it sees in its own history, and nothing in the stack can tell it not to. The exit is to keep the broken turn out of the history.
And it happened on all three engines. Poisoned at effort high: spark-ds41f 1120, 14 of 564, 2.5% (1.5–4.1%); stock vLLM on the GB300, 20 of 564, 3.5% (2.3–5.4%); SGLang, 3 of 432, 0.7% (0.2–2.0%). At effort max, 4 of 48 on 1120 (8.3%) and 3 of 48 on the GB300 (6.2%). SGLang is lower; we did not investigate why, and the intervals overlap at the low end. The point is what the three have in common and what they do not. The GB300 lane runs DSpark speculative decoding with no copy drafts, so the loop does not need Owen's copy drafts. SGLang is a different engine with a different parser. The 1120 logs show no loop-detector events and no errors during the runs. What is shared is the model and the history, so that is where the loop lives.
Deeper is worse, 4K is not immune
Owen's third question: does it happen at 4K? With two broken turns in the history, yes, but rarely. Pooled over the three engines at effort high:
| depth | pooled n | failures | rate | 95% CI | 1120 | GB300 | SGLang |
|---|---|---|---|---|---|---|---|
| 4K | 324 | 2 | 0.6% | 0.2–2.2% | 0.8% | 0.8% | 0.0% |
| 64K | 324 | 0 | 0.0% | 0–1.2% | 0.0% | 0.0% | 0.0% |
| 200K | 228 | 4 | 1.8% | 0.7–4.4% | 2.6% | 2.6% | 0.0% |
| 400K | 228 | 11 | 4.8% | 2.7–8.4% | 2.6% | 10.3% | 1.4% |
| 598K | 228 | 9 | 3.9% | 2.1–7.3% | 2.6% | 9.0% | 0.0% |
| 800K | 228 | 11 | 4.8% | 2.7–8.4% | 9.0% | 2.6% | 2.8% |
Figure 3. Failure rate by session depth with two broken turns in the history, pooled over three engines at effort high, with 95% intervals. The per-engine dots are drawn lighter on purpose: those cells hold 72 to 126 requests each, too few to rank engines by depth.
The per-engine, per-depth cells are small, so we make claims from the pooled column and from two bands. At 4K–64K: 1120 1 of 252 (0.4%), GB300 1 of 252 (0.4%), SGLang 0 of 144. At 200K–800K: 1120 13 of 312 (4.2%), GB300 19 of 312 (6.1%), SGLang 3 of 288 (1.0%). So the short-context rate on the vLLM engines is about a tenth of the long-context rate, not zero. Owen's theory leaned on the last 128 tokens, the sliding window every layer reads exactly. We tested that too, at the same three depths (200K, 400K, 598K): with the broken turns ending about 200 tokens before the reply, 24 of 684 (3.5%, 2.4–5.2%); with 9–15K tokens of normal tool turns after them, 5 of 672 (0.7%, 0.3–1.7%). Being inside the window amplifies the copying, but imitation also reaches past it.
What the failures look like
The two vLLM engines return token ids (return_token_ids), so for most failures we can see exactly which pieces the model wrote. SGLang returns none, so its shapes come from text. Three shapes account for 76 of the 79 counted failures (45 + 7 + 24 = 76; plus 2 empty replies and 1 bad-shape call, the GB300 parser bug above, makes 79). The 45 of Shape A and the 7 of Shape B are the 52 that carry any call markup.
Figure 4. Three ways it broke, with real decoded token pieces from results/failures.jsonl. The counts 45, 7 and 24 are the three shapes; 2 empty replies and 1 bad-shape call make up the rest of the 79 failures.
Shape A, 45 cases, Herlon's symptom. A sentence, then <|DSML| parameter …>…</|DSML| parameter></|DSML| invoke></|DSML| calls> as text, no calls opener and no invoke name= line. In 34 of them the command is new (for example sed -n 1180,1270p tests/v1/core/test_scheduler.py); 11 are verbatim copies of the broken turn in the history. Of the 42 with token ids, the parameter tags are typed-out pieces in all 42; 39 contain no real DSML token at all; 3 switch to the real token for the last two closers only.
Shape B, 7 cases (1120: 3, GB300: 4). The real opener token <|DSML| calls>, then no invoke line, then typed-out parameter tags. Both vLLM parsers enter the call, cannot build one with no tool name, and drop the body silently; the client gets only the sentence, with finish_reason "stop" and no error. The decoded pieces of one, as stored, with the single real token in bold:
[.\n\n] [<] [|DSML|] [ calls] [>\n] [<] [|] [DS] [ML] [|] [ parameter] [ name] [="] …
That bold piece is token 128825; every |, DS, ML after it is an ordinary token.
Shape C, 24 cases. A short line announcing a tool action, then nothing, mostly in the strip variant. Common to A and B: the invoke name= line is always missing. And the model often knows. One reply's reasoning, verbatim from the record:
My previous tool call had malformed tags (fullwidth characters), so it never actually executed. Let me redo it properly with plain ASCII.
Its visible content was "My previous tool call was malformed; redoing it." and no call followed. The model diagnosed the problem correctly and still wrote the broken format.
What fixes it
Three ways of handling the broken turn before sending history back, all measured at 200K–800K and effort high. The untouched poisoned history at those same depths fails 35 of 912 times, 3.8% (2.8–5.3%):
- drop (remove the broken turns): 0 of 576 (0–0.7%), and 0 of 192 on each engine.
- repair (rewrite each as a real bash call plus a "not executed" tool result): 0 of 456 (0–0.8%); 0 of 192 on 1120, 0 of 192 on the GB300, 0 of 72 on SGLang.
- strip (keep the sentence, cut the markup): 22 of 576, 3.8% (2.5–5.7%); 8 of 192 on 1120 (4.2%), 10 of 192 on the GB300 (5.2%), 4 of 192 on SGLang (2.1%). That is no better than leaving the broken turns in (3.8% at the same depths); only the shape changes. The stripped history now shows the model turns that announce a tool action and make no call, and it copies those instead.
Figure 5. Failure rate by history variant, pooled over three engines (effort high unless noted), with 95% intervals. Blue: no broken turns, all depths. Red: broken turns present, 200K–800K (the max-effort row is 598K only). Green: the three treatments, also 200K–800K, so they compare directly with the red poisoned row. Zero bars are drawn as a baseline marker with their 0/n label.
Drop and repair together: 0 of 1,032 (576 + 456). We did not find a run where keeping the broken turn out of the history failed.
Owen's three options, against the data
| option | what the data says |
|---|---|
| (a) accept a call with no opener | Not safe. In all 42 Shape A cases with token ids the failing tags are typed-out text, so accepting them is accepting spelled structure: bug 3 again. Owen's instinct was right. |
| (b) keep the broken turn out of the history | The only option our data supports: drop 0 of 576, repair 0 of 456. Do not do it by stripping the markup; that leaves "announced but did not call" turns and the model copies those (3.8%). On the server side, Shape B (real opener, no invoke line) is an unambiguous signal to resample or return an error rather than return text with finish_reason "stop", so the client never saves it. |
| (c) hold the opener in place with a grammar | It would have to enforce the whole chain, calls → invoke name= → parameter, not just the opener: in Shape B the opener was present and the invoke line was the part missing. |
Side finding: bug 3 on two of the three engines
Before the sweep we ran Owen's older cases as short probes, 6 per row per engine, half streaming. These are the probes that the 1120 release was built to pass, and it does. The two engines without his real-token rule do not.
| probe | 1120 | GB300 | SGLang |
|---|---|---|---|
| #876: quote a sample call in a code block, tool offered, thinking off | 6/6 pass | 0/6 (real call made) | 0/6 (real call made) |
| same, thinking on | 6/6 | 0/6 | 0/6 |
| #876: no tool offered | 6/6 | 0/6 (content cut at the code fence) | 6/6 |
| #948: closers inside a write_file argument (thinking off / on) | 6/6, 6/6 | 6/6, 6/6 | 6/6, 6/6 |
| #446: a DSML call injected in a tool result (off / on) | 6/6, 6/6 not obeyed | 6/6, 6/6 | 6/6, 6/6 |
So our GB300 lane and the SGLang build both have bug 3: a quoted, typed-out DSML sample in a code block becomes a real call when a tool is offered. SGLang's logprobs show what happened in its #876 failure: the model wrote |, DS, ML, | inside a markdown code fence and ended with FINISHED, and the parser still produced get_weather(Boston). This is not a ranking of engines. It is one specific rule that Owen's engine had and the other two did not.
Update, November 22, 2026, 08:37 CST: our GB300 lane now has the real-token rule. We ported it (upstream pull request #65019, adapted to our vLLM 0.31.0, merged with our closer rule) and put it live on the lane the same morning. The same probes now pass 42 of 42 on the GB300, including both #876 rows that failed 0 of 6 above. Structured-output burst 64 of 64, one-user decode 546 tok/s per stream, 32-user aggregate 3,622 tok/s (3,566–3,664 before). The overlay and its tests are in the same repo. Every sweep number in this post was measured before the change, and none of the loop results depend on it: the failing tags were ordinary tokens either way.
Limits
- The history is synthetic. We did not reproduce the first occurrence (clean 0 of 600). Whatever triggered Herlon's may be absent from our session: client-side history rewriting, compaction, earlier turns that quote markup.
- One prompt family. Repeated requests reuse near-identical prompts, so they are not independent sessions, and the intervals should be read with that in mind.
- SGLang returns no token ids; its failure shapes come from text alone.
- Per-engine, per-depth cells hold 72 to 126 requests. We make claims from pooled rows and bands, not from those cells.
Run it yourself
The harness, the session builder, every result row and every failure with its token pieces are in milo-the-machine/dsml-opener-sweep (MIT). It needs a vLLM v0.31.0 source tree for the tool-result contents and any DeepSeek-V4.1 endpoint with /tokenize to size the sessions. From the README:
python3 build_sessions.py --src /path/to/vllm-v0.31.0/vllm \
--tokenize-root http://HOST:PORT --model MODEL_ID
python3 run.py sweep --engine a=http://HOST_A:PORT/v1 --variants clean,poisoned \
--depths 4000,64000,200000,400000,598000,800000 --reps 48 --conc 8 --tag main
python3 final_tables.py
Two rules from the README worth repeating: size the sessions with the engine's own /tokenize, not a local tokenizer, and run the depth cells in ascending order so each one is mostly a prefix-cache hit. Rates of 1–7% need at least 48 reps per cell per variant before comparing engines.
Credits and links
Christopher Owen: spark-ds41f, the report and the questions this post answers (his X post, repo, our issue #1305). Herlon, for the transcript behind it. joesinvestments (#876), koldfrontier (#948), Acermax (#446 and #449) and Jordi (the earlier loop detector) for the DSML bugs that came before this one. Garner McCloud: the DeepSeek-V4.1 engine our GB300 lane is built on. knapcio: the dsv41-4x-spark SGLang recipe. DeepSeek for the model and for the encoding docs we used as test content (MIT). Data and harness: dsml-opener-sweep.
Earlier posts this one leans on: why we are not porting spark-ds41f to the GB300 (November 21, 2026), eight Sparks vs one GB300 (November 20, 2026), and Pg62, the recipe our GB300 lane grew from.