Skill check: 260 skills, two new reviewers, one bloat machine

Created September 5, 2026 at 6:23 AM CDT Last updated September 5, 2026 at 8:48 AM CDT

Two new frontier models showed up this week, so James asked for a skill check: sweep every Hermes skill Milo has accumulated, get a second opinion from GPT-6 Astra, discuss, then act. This is what the sweep found and what changed.

James Meadlock + Milo · sweep and writing: claude-fable-5-1 via anthropic · second reviewer: gpt-6-astra via openai-codex · distillation subagents: gpt-5.5 · Hermes Agent 0.21 / profile milo

Verdict: the library was not too big; it was un-curated. Zero-use bundled skills were a cheap per-turn tax, and a background "self-improvement" reviewer had turned the best operational runbooks into append-only session logs. The fix was housekeeping, not a new system: archive the dead weight, rewrite the giants by hand with live verification, merge true duplicates, and put a human approval gate in front of the reviewer. Nothing was deleted.

Measured on the live tree before and after. Archive means moved to .archive/ with a restore manifest, not deleted.

What a skill is, and why the count matters

In Hermes, a skill is a directory with a SKILL.md entrypoint plus optional references/, scripts/, and templates/. Two costs are easy to conflate. Every turn, the system prompt carries an index of every skill's name and a truncated description so the model knows what it can load; that is a recurring tax proportional to the count. Loading a skill then injects its full entrypoint into context; that is a per-use tax proportional to the entrypoint's size. Disk size is irrelevant.

Measured this morning: 260 skills (235 in the profile, 25 mounted from the Matt Pocock engineering set) cost about 22k characters of index per turn, roughly 5.5k tokens before a single skill was loaded. The largest entrypoints were 100k characters.

The inventory

The usage ledger (skills/.usage.json) records per-skill use counts, view counts, patch counts, timestamps, and who created it. Joining that against the tree:

Metric	Value
Skills total	260
Authored by Milo from real sessions	96
Zero uses ever	57 (54 bundled or hub-installed, 3 Milo-authored)
Used, but idle more than 45 days	16
Entrypoints over 20k characters	23
Descriptions truncated in the index (over 120 chars)	66
Secrets in any skill file	0 (placeholders only)

The bloat machine

The interesting finding was not the dead weight; it was where the live weight came from. Hermes runs a post-turn background review, a fork that replays the conversation and decides whether to save a memory or patch a skill. Its ledger (skills/.curator_ledger.jsonl) showed 703 of 892 skill writes over 18 days came from that reviewer, not from Milo in-session. The GB300 runbook alone took 154 reviewer writes in four days and grew from 5.4k to 67k characters. The dual-Spark runbook had a numbered pitfall list that reached #85.

The content was accurate. Diffing the ledger's before/after blobs showed real, measured lessons: "sshpass hang is a missing password file," "cron watchdog resurrected a torn-down stack and ate 108 GiB mid-bind," "hf download with bracket globs 404s." The failure was structural: append-only into the entrypoint, duplicate "updated <date>" blocks stacked under one heading, and restore commands that quietly drifted from what was actually in production. Every load paid for the whole chronology.

One correction worth recording: the knob that controls this is auxiliary.background_review.enabled, not curator.enabled. The weekly curator only does deterministic stale-to-archive transitions and never patched a runbook. The ledger labels both as actor curator, which is how the wrong knob got blamed first.

Second reviewer: GPT-6 Astra

Following the measure-then-consult rule, the brief carried the measured table, the overlap clusters as name uses/last-use-days/k-chars, Milo's own reading, seven numbered questions, and the full 260-row inventory as an appendix (~55k input tokens). Astra's verdict matched Milo's on the shape: archive low-value capabilities, distill the operational giants, merge only after reading contents, never bulk-delete. Where they disagreed, Astra was mostly right:

Topic	Milo	Astra	Disposition
Merge the 18 `hermes-*` skills to ~10	merge	don't force it; distinct failure domains	Astra. Archived the 3 zero-use, left the rest.
Strong models make "how to think" skills less needed	asserted	they encode James's standards, not model hand-holding	Astra. TDD and debugging skills kept.
Review-skill overlap is harmless	asserted	not when permissions differ (read-only vs auto-fix vs approval-gated)	Astra. Kept separate.
patch_count ≈ use_count proves reviewer bloat	asserted	inspect diffs, don't infer	Right on method; the diffs then confirmed it.
Curator config keys	named `skills.guard_agent_created` / `write_approval`	"not on the curator docs page"	Reviewer factual error, minor: they are `skills.*` keys and exist.

What changed

Action	Count	Mechanism
Archived (zero-use bundled/hub + 7 vendor)	46	moved to `.archive/`, usage state set, restore manifest + `archive.py --restore`
Distilled	22	old entrypoint preserved verbatim as `references/history-through-2026-09-04.md`; new 10–14k entrypoint written from live-verified state
Merged	7 → 2 (Reachy), 3 → 1 (cartoon), 3 → 1 (MLX), 2 → 1 (packets), 2 → archive (OB1)	97 reference files moved, 8 cross-references rewritten, absorbed dirs archived with README stubs
Deleted	0	—
Pinned	23	blocks archive/delete of the rewritten runbooks

The distills ran as parallel subagents on a shared written procedure, with the first hand-distilled runbook (dgx-spark-ds4f-ops, 65.6k → 11.5k) as the shape template: a live-state table with a verification date, hard rules one line each, the operational sequence with gates, a KEEP / MUST / FINAL REJECT / PARKED verdict table, and pointers. Each subagent was allowed read-only probes of the real hosts and had to report anything contradictory in its source. That constraint paid for itself.

What live verification caught

Rewriting from the old text alone would have produced confident, wrong runbooks. Probing first found:

The dual-Spark restore section still targeted the text-only 0731 checkout and gated on the wrong model name; production had been Vision-Exp since August 31, with a different draft length (MTP_NUM_TOKENS=6, not 5).
The Spark watchdog cron was still commented out from an August 26 maintenance window, so the live lane had no auto-restart. That fact was buried in pitfall #80; it is now in the live-state table.
Every documented LAN port on the M3 Ultra refused; only one llama.cpp process was up, bound to localhost. The placement skill now marks those lanes stale rather than restating them.
The GB300's production port was dark because James is mid-campaign testing a large GLM on it. A subagent flagged the mismatch as "reconcile before any change" instead of fixing it, which is the correct behavior for a reviewer with no authority over production.
The dashboard reports "degraded" solely because Telegram is disconnected on the milo gateway; the runbook now says not to read that as a desktop outage.

The reviewer, going forward

Turning the background reviewer off permanently would lose the capture that produced 96 of the best skills. Leaving it as it was reproduces the bloat. Hermes has the middle option built in: skills.write_approval: true makes every skill write, from the reviewer or from Milo in-session, stage under pending/skills/ instead of committing. Review is /skills pending, /skills diff <id>, /skills approve or reject; reviewer proposals are tagged [auto]. The gate was proven immediately: Milo's own note recording the new configuration was staged as pending cdcade60 rather than written.

Exit condition: if the pending queue sits unread for two weeks, the reviewer gets turned off rather than rubber-stamped. A gate nobody reads is worse than no gate.

What was deliberately not done

Bundled upstream skills over 20k characters (humanizer, claude-code, unbroker, comfyui, claude-design, research-with-quarantine) were left alone. Distilling them would mark them user-modified and fork them from hermes update.
No automatic consolidation or merge mode. The Reachy 7 → 2 merge took a subagent nine minutes of judgment about which of five overlapping stacks was current; no heuristic does that.
No semantic-search layer, no retrieval system, no new tooling. Astra's phrase: "this is housekeeping, not a new retrieval-system project."

Reproducibility

Everything is reversible. Each rewritten entrypoint has a pre-copy in the sweep work directory and a verbatim history file in its own references/. The 58-entry archive manifest restores in one command. The sweep procedure, the fan-out procedure, and the verify script are recorded in a new pinned skill, skill-library-curation, so the next sweep starts from the ledger instead of from memory.