← Back to Home

Ornith 1.5-397B Q6_K on M3 Ultra

M3 Ultra 512 GB · llama-server b9700 · operational probes, not a harness score

What we ran

Official Ornith 1.5-397B is a reasoning MoE on a Qwen 3.5-class backbone. The card’s llama.cpp one-liner is -c 262144. We are on that window now. Weights are the official GGUF Q6_K (~303 GB on disk) on stock Homebrew llama-server b9700 (9724f664e), localhost only, no LaunchAgent, no Hermes route.

Bring-up was 8k (HTTP 400 on an 8,229-token request), then 32k (12.6k prefill passed), then native 262k. No YaRN. The card says static YaRN hurts ordinary-length quality.

/opt/homebrew/bin/llama-server \
  -m Ornith-1.5-397B-Q6_K.gguf \
  -a ornith-15-397b-q6k \
  --host 127.0.0.1 --port 8035 --no-webui --jinja \
  -ngl 999 -fa on -c 262144 --no-context-shift \
  --parallel 1 --cache-ram 0 --no-cache-idle-slots \
  -b 2048 -ub 2048 -ctk q8_0 -ctv q8_0 \
  --threads 8 --threads-batch 24 \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  --reasoning off \
  --chat-template-kwargs '{"enable_thinking":false}'

After the 262k restart, /v1/models reported n_ctx=262144 equal to n_ctx_train. RSS was 307.6 GiB at idle and 308.5 GiB after the 64k prefill. Wired limit on this box is already 480 GB. Context is a prefill-time cost, not a RAM problem.

Measured probes (August 19, 2026)

Single client. Streamed completions used stream_options.include_usage=true. Decode rate is completion_tokens / (total − TTFT). Prefill rate is prompt_tokens / wall with a nonce at the start of the prompt. A 21k run that reused an earlier start nonce looked like 769 tok/s and is not published as cold prefill.

Prefill slowed from 459 tok/s at 12.6k to 294 at 64k to 121.6 at 246k. The 15-minute linear guess was wrong. The measured dump was 33.8 minutes. llama-server logged 2,024,783 ms / 246,435 tokens (121.71 tok/s).

How to read it

• Decode matches Ornith 1.0 Q4_K_M’s speed class on this machine (~31 tok/s in the June 29 GGUF recipe). Q6_K costs about 90 GiB more RSS for no decode win. That 1.0 figure is from the published recipe, not a same-session re-run.
• 8k was the real bug, not Metal. First serve logged n_ctx_seq (8192) < n_ctx_train (262144). 32k unblocked 12.6k. 262k unblocked 64k.
• 246k works without a RAM step-change. 310 GiB RSS after the dump versus 308 GiB after 64k. Context is still a time cost, not a RAM problem.
• Thinking-off works with --reasoning off plus the chat-template kwargs.
• Tools work at the HTTP layer. That is not an agent score.

What this is not

• Not a full 262,144-token prompt. The dump used 246,435 prompt tokens plus 10 completion tokens so the template would fit under the window.
• Not Terminal-Bench, SWE-bench, or a Hermes canary.
• Not a reason to displace DS4 as the default agent route. A cold 246k turn costs half an hour before decode starts.

Next

1. Hermes canary only, advertised 262k, long timeouts. Keep it off the default route.
2. Optional --spec-type ngram-mod on a short-context A/B, not during a 246k fill.

Related: the Ornith 1.0 GGUF + MLX recipe from June 29–30. Card: ornith-ai/Ornith-1.5-397B. GGUF: ornith-ai/Ornith-1.5-397B-GGUF.

All numbers in the probe table came from direct HTTP calls on August 19, 2026 against the live localhost server. Official card scores are not reproduced here.