Oleks
17b5b15956
An audit that forgets every run makes the analyst re-derive the same judgment calls from scratch each time, and loses the declined-candidate reasoning that is the most expensive thing a run produces. Two new audit-snapshot subcommands, both with `pages`' contract — filesystem in, JSON out, no MCP call ever made here: memory-queries what to search before grading memory-notes the drawers and kg facts to write after the run Placement: target-scoped findings go to the audited plugin's own wing (room `inference-arbitrage-audits`), mirroring FR-6.1's "the finding belongs on the target's surface"; cross-target calibration goes to `claude-plugins` (room `inference-arbitrage-lessons`) on every run regardless of outcome. Rooms are namespaced by this plugin's name because plugin-named wings really do collide with unrelated ones (`cluster`, `imagex`), and a namespaced room makes that a shelving overlap rather than a content collision. Reading is advisory and never a gate: nothing recalled may override boundary-classify (FR-4.2) or stability-classify (FR-4.5), excuse skipping the marker search, or re-grade a returned verdict. The memory renderers deliberately do NOT reuse render_run/render_latest. `snapshot.notes` is caller-supplied free text that legitimately reaches a wiki page in this plugin's own repo, and must never reach a drawer in a wing shared with other projects. They read from a fixed allowlist of structured fields — allowlist, never blocklist, since a blocklist is one new snapshot field away from leaking. The MCP calls are not covered by tests/memory.test.sh and are not faked there. That is the point of putting all the judgment in the script: what remains untestable is one verbatim tool call with nothing in it to get wrong. The live round trip was verified by hand against the real anxious snapshot.
inference-arbitrage
Audits a Claude Code plugin — its skill/agent definitions and its real usage transcripts — to find steps that are being done by raw LLM inference but pass every test of a deterministic script, and files the well-evidenced ones as issues on the target plugin's own repo. Runs on demand; each run accumulates into a snapshot history so cost and candidate status can be tracked over time.
The name echoes builder-arbitrage: route each unit of work to the cheapest
executor that can do it correctly — here, "script vs. model" instead of
"which build node."
Status: scaffolding, pre-implementation. See design/ for the full
specification:
design/spec.md— why, scope, functional requirementsdesign/plan.md— architecture and file layoutdesign/rubric.md— the script-vs-inference boundary rubricdesign/tasks.md— build order
Tracked as issues on this repo under the v0.1.0 milestone.