One Milo, Many Bodies

Milo choosing among multiple embodiment options while James watches.
One Milo, many bodies: desktop, robot, browser, and whatever comes next.
Created Last updated

Originally July 28, 2026 · updated July 29, 2026 — Phase 0/1 stop · by James & Milo

Current call — July 29: The frozen Phase 0/1 contract spine remains the rollback-safe stop point, while five bounded local slices were added behind it: strict loopback admission/host, server-owned text turns, a browser-avatar render adapter, provider-neutral model-context construction, and an API-family-pinned provider translation plan. 225 tests green; final independent fail-closed review passed. No provider client or call, token-budget approval, streaming path, audio, robot motion, persistent listener, or deployment cutover was activated.
Separate Reachy baseline update — July 28: this does not reopen or cut over the frozen Phase 0/1 runtime. The working Reachy baseline now has a deterministic P0 greeting path behind a loopback-only, token-authenticated voice-control boundary. It refuses remote callers and interruption bypasses; the greeting remains idle-only and literal, not LLM-rephrased. No forced greeting, robot motion, or live audio was used as the security smoke test.

Separate Reachy baseline: hardened, not promoted

The adjacent production baseline is still the rollback path, not an implementation of the private contract spine. We hardened it because a robot speech-injection endpoint is voice authority: an unauthenticated LAN endpoint was not acceptable in a home. The work is intentionally narrow and reversible.

BoundaryCurrent verified state
Greeting injectionLoopback-only, token-authenticated, idle-only; response.created precedes output, and a cancellation fence refuses any literal enqueue after barge-in.
Greeting policyDeterministic literal TTS, quiet-hours aware, and separate from the LLM. Unknown people remain silent.
Knowledge authorityOrdinary spoken turns cannot attach web lookup. Only the explicit ask big Milo path may authorize read-only research.
Honcho recallHoncho v3 search uses JSON POST only. A stale GET fallback returned 405 and was removed; writes stay serialized and receive bounded in-memory retry. Read timeouts remain separately observable.
Evidence612 tests, Ruff, lock check, package build, and live non-speaking rejection probes passed. The JSON-POST repair has a focused regression test within that suite; the brain restart rebuilt Honcho-backed context without a new stale-GET or 405 error.
Promotion rule: a hardened legacy baseline is not a migration. The frozen runtime remains stopped at its contract boundary until a later, explicit promotion decision and live-profile evidence justify a cutover.
Suite225pytest green
Phaselocal slicescontracts only
Live cutovernoneReachy baseline intact
Rust rewritenohybrid later if needed

Goal is still the same: one Milo mind and voice runtime, many bodies. Reachy Mini, a browser avatar, PuppyPi, and future surfaces should share identity, turn-taking, STT/LLM/TTS policy, memory, and safety — not each reimplement a brain.

What changed since the original planning post: the architecture is no longer only a diagram. A private Python package now holds reviewed, fail-closed contracts for handshake, isolation, memory visibility, delivery styles, reactions/banter, TTS planning, thin Reachy/browser adapter translation, bounded model-context assembly, and a pure provider-plan translation seam.

Target architecture

Milo voice runtime target architecture M4 Max owns the voice runtime. Reachy, browser avatar, and PuppyPi are thin adapters. Production Reachy remains rollback baseline. One Milo runtime · many bodies M4 is authority · adapters render only · production Reachy stays rollback M4 Max · Milo Voice Runtime session · STT · LLM · TTS · memory · safety · telemetry Admission handshake Policy delivery · banter I/O plans TTS · model VoiceRuntime owner path · per-session isolation · memory/context barriers Reachy adapter contracts shipped expression + playback intents no live robot I/O Browser avatar display contract shipped visual cues only · no media I/O PuppyPi adapter later allowlisted poses only Working Reachy deployment = rollback baseline until promotion gates pass

Target topology. Solid Reachy and browser boxes = contracts exist. Dashed PuppyPi box = planned. Amber strip = production rollback baseline.

What is real code today

Phase 0/1 contract spine modules Shipped Python packages from protocol through Reachy adapter and banter pack, with live path still outside the box. Contract spine + bounded local slices Private package milo_voice · frozen context + provider plans before live bindings protocol milo.* events runtime admit · turns memory visibility barrier policy styles · banter pure plans tts · provider VoiceRuntime owner path connection_id only · admitted tags · no raw workspace/policy escape hatches adapters.reachy hello/caps · expression cues playback metadata · no locomotion Not activated or connected yet STT/LLM/TTS vendor bindings · audio/media paths · robot drivers persistent listener · Honcho writeback · voice enrollment · cutover

Shipped module spine vs still-gated live path. Rose dashed box is intentional non-work at the stop point.

Shipped layers

LayerWhat it locksKey commit
Protocol / handshakeStrict milo.* events, two-step admit→readyearlier Phase 1
Admission + isolationCapacity, surface bind, per-connection workspace5c87a08 / 577e178
Memory visibilitySource-tagged durable writes + generation barrier8cbbf45
VoiceRuntime + turnsOwner path, sticky cancel, turn reset6ea96ac
Delivery stylesdefault/pirate, scoped expiry, soft fallback83cca02
Reactions + banterSpeaker observations; bruh + sparse TWSS56d6d09 / 24c9e4b
TTS seamFrozen TtsPlan, no audio/network37ff5cc
Reachy adapterRender intents only; camera=false honesty; locomotion rejected04cd4d3
Loopback admission + hostStrict JSON-only loopback host; server-created admission state; no persistent servicelocal verified slice
Text turn lifecycleServer-issued turn IDs; stale, forged, cancelled, and late transcript writes fail closedlocal verified slice
Browser avatar adapterFrozen display intents only; no microphone, speaker, camera, motion, or visemes claimlocal verified slice
Model context seamFrozen, bounded, server-owned packet; policy, visible memory, completed history, and one finalized current turnlocal reviewed slice
Provider translation planAPI-family-pinned immutable plan; references remain framed user-role data and provider token fit remains unchecked5b54144

Invariants that stayed true

Deliberately not done

Rust port?

Research conclusion: full rewrite does not make sense at this stop point. The codebase is still mostly contracts/policy (~1.6k LOC), not a media plane. Keep Python core. Revisit a hybrid Rust gateway/audio edge only after live multi-session profiling proves a bottleneck. Full note lives in-repo: docs/research/2026-07-28-rust-port.md.

Resume checklist

  1. uv run pytest -q && uv run ruff check .
  2. Pick one separately authorized next slice only; a future live provider binding must consume the reviewed plan, add a real tokenizer/window check, and still does not authorize a robot cutover.
  3. Keep fail-closed review before any verified commit.
  4. Do not touch live Reachy services until pre-deployment gates pass.
Stopping here is the feature. The spine is reviewable and implementable against; production remains undisturbed.