Milo is an AI agent running on a Mac Studio. It can search the web, manage files, write code, send messages. What it couldn't do was turn on a light.
The house has six systems that don't talk to each other: Lutron Caséta switches, Philips Hue bulbs, two Lennox S30 thermostats, an Eight Sleep pod, a Roomba, and a network of presence sensors. Each has its own protocol, its own auth model, its own idea of what status means. None of them expose a unified API.
The bet: wire all six into a single local FastAPI service that an AI agent can query and control. No cloud. No subscription. No Home Assistant. Just direct protocol integration on the LAN.
The existing app situation was fine, technically. Lutron has a solid app. Roomba works. But "fine" isn't the same as useful when you want an agent to be able to reason about your home.
The goal was simple: give Milo the ability to see what's happening in the house and act on it. Lights, temperature, whether anyone's home, whether the Roomba should run. All through a local API, no cloud dependency.
| Integration | Protocol | Status | Notes |
|---|---|---|---|
| Lutron Caséta | TLS / pylutron-caseta | Live | SmartBridge at 192.168.1.244, 76 devices |
| Roomba | MQTT / roombapy | Live | Static IP, 100% battery, paired via blid |
| Philips Hue | HTTP / Hue v1 API | Live | Bridge at .201, v2 migration queued |
| Lennox S30 | Cloud / lennoxs30api | Live | Cloud mode only; local API behind dealer PIN |
| Presence (iPhone) | ARP ping | Live | James + Cindy; Roomba auto-start on all-away |
| Tesla Model 3 | teslacmd / Fleet API | Live | Charge level, location, climate, lock/unlock, departure events |
| Eight Sleep | pyeight / cloud | Connecting | Rate-limited at setup; auto-reconnects |
| HomePod / Apple TV | pyatv / AirPlay | Pending | RAOP broken for HomePod Gen2; owntone queued |
| RainBird | pyrainbird / LAN | Pending | Found at .132; needs password from app |
The stack is five layers:
presence.all_away or presence.someone_home. Lutron pushes lutron.device_changed on every state change.automations.json. Evaluates conditions (time range, cooldown). Executes actions. Declarative, not hardcoded.homectl CLI, OpenClaw/Milo agent calls, SSE stream for real-time monitoring, Apple Reminders, Telegram.The interesting part isn't the individual integrations — those are mostly just library wrappers. The interesting part is the event bus that ties them together.
Before building it, automations were hardcoded in the presence polling loop. "If all away and Roomba connected and cooldown elapsed, send start." That works for one automation. It doesn't scale to five, and it's impossible to configure without editing Python.
The event bus replaces that with a simple pub/sub pattern. Every integration publishes events. Subscribers (the automation engine, SSE clients, any future consumer) receive them:
await bus.publish("presence.all_away", {
"people": {k: v["home"] for k, v in presence_state.items()}
})
The automation engine subscribes to all events and evaluates rules against them. Tesla departure and arrival events slot right in:
{
"id": "roomba_on_departure",
"enabled": true,
"trigger": { "event": "presence.all_away" },
"conditions": [
{ "type": "cooldown_minutes", "value": 30 },
{ "type": "time_range", "start": "08:00", "end": "21:00" }
],
"actions": [
{ "type": "roomba.start" }
]
}
Enable or disable without touching code: homectl automations enable lights_off_on_departure. Test manually: homectl automations fire roomba_on_departure.
The house has two light systems: Lutron Caséta (the main system, 76 devices) and Philips Hue (accent lighting, 10 devices). Before this, controlling "the living room lights" meant knowing which system to call.
The unified lights layer abstracts that away. POST /lights/living room/on?level=60 routes to Lutron for the main lights and Hue group 1 for the accent lamps, in one call. The router checks both maps and fires both:
homectl lights "living room" on --level 60
# → Lutron: 3 devices @ 60%
# → Hue group 1 @ bri 153
32 rooms mapped. New integrations (MyQ garage, RainBird sprinklers) will slot into the same abstraction layer.
Presence detection uses ARP ping against iPhone IP addresses. It's not perfect — iOS randomizes MACs — but both phones have static DHCP leases in UniFi, so the IPs are stable.
The implementation hit one non-obvious issue: LaunchAgent plist environments don't inherit the shell PATH. ping resolved at startup but failed silently inside the service. Fix was using the full path /sbin/ping.
First live test: both phones pinged, Roomba started automatically when we both left. It worked.
HomePod Gen2 TTS was the biggest failure. pyatv speaks RAOP to HomePod, but for Gen2 it sends the RTSP handshake using the source machine's IP instead of the HomePod's. The handshake fails. owntone (formerly forked-daapd) is the right fix — it implements AirPlay 2 properly — but it needs to be built from source. That's queued.
Apple TV Companion pairing needs a PIN that appears on the TV screen. Getting the PIN to show required navigating Settings → Remotes and Devices → Remote App and Devices, and even then it was finicky. Pairing eventually succeeded on one unit but the approach needs more work before it's reliable.
The Lennox S30 local API is locked behind an installer/dealer PIN. The cloud API works fine, but the cloud push model means data arrives asynchronously via a message pump rather than on demand. The integration works, it's just not as clean as a direct LAN connection.
The SSE /events endpoint streams every event in real time:
curl -N -H 'Authorization: Bearer <token>' http://localhost:8401/events
[09:14:01] system.startup {}
[09:14:03] lutron.device_changed {"device_id": "42", "name": "Kitchen Main", "level": 100}
[09:16:22] presence.all_away {"people": {"james": false, "cindy": false}}
[09:16:22] presence.all_away → automation: roomba_on_departure
[09:16:23] roomba.state_changed {"state": "cleaning"}
This stream is what makes the system legible. You can watch the house in real time, debug automations, or pipe events into other tools. The CLI wraps it: homectl events.
All Python. All local. No cloud dependency for core functionality.
The service is about 1,100 lines of Python. It's not minimal, but every line is doing something. A future refactor could split the integrations into modules. For now it all lives in server.py alongside events.py for the event bus and automation engine.
The code is open source. If you're building something similar and want to compare notes, the approach generalizes to any collection of home devices with Python libraries.