The stability gate treated every evidence flip as symmetric, and that
produced a false downgrade on the candidate it was written for
(kotkan/claude-plugin-inference-arbitrage#15). Re-measuring
anxious/agent-wip/release-policy-derivation over four real windows with
the same script gave 0 -> 2 -> 12 -> 15 invocations at 7/14/27/60 days:
monotonic accumulation of a step that runs about once every two days, not
a label flipping about. The 7-day window had not caught an unstable
candidate, it had failed to observe a real one.
So a thin/unmeasured reading now only overturns a `measured` one when its
window is at least 90% as wide (observed since->until span). Below that
the comparison takes a third verdict, `insufficient-window`: not filed,
not downgraded to a boundary question, no FR-6.5 self-report, and carried
forward with the wider snapshot left as the standing comparison point.
The parity is relative rather than an absolute day count because the
adequate width is a property of the candidate's invocation rate, which
the auditor does not know in advance.
Unchanged, and tested: the reverse direction (thin prior -> measured now)
still downgrades, a flip between comparably wide windows still downgrades,
and a threshold crossing with `measured` on both sides is never excused by
narrowness. An unreadable window is not an exemption either — the flip
stands and window_parity records that the check could not run.
- bin/stability-classify: window_days/window_parity/deferrable, the
insufficient-window verdict, stability_deferrals, window_parity on
findings too, insufficient_window + window_parity_ratio in the gate block
- bin/audit-snapshot: record deferrals in the snapshot notes so the row is
visible on the wiki page instead of silently dropped
- design/spec.md: FR-4.5.1 as a testable requirement; S4 restated
- design/rubric.md: the third boundary in §5b
- skills/offload-audit/SKILL.md: do not file and do not self-report a deferral
- tests: real 60-day anxious fixture; cases (f)/(f2) on real data, (j)-(n)
synthetic — 7 test files pass
Verified against the live store (~/.cache/inference-arbitrage/wiki): the
real 7.0d-vs-26.49d comparison now reports insufficient-window with zero
findings, while the real 33.4d-vs-26.49d one reports stable at 13.87% and
files normally.
An independent spec-quality review (spec-review) found real drift and one real
gap, but also overstated FR-9.2 as an unenforced 'discipline' when three of its
four prohibitions are already mechanical (filing-plan only files verdict=='file',
requires --existing so a failed marker search can't silently duplicate, and
stability-classify rewrites rather than re-asks). Only 'what the analyst writes
into the judgments JSON' is genuinely unchecked — narrowed the claim to that.
The review also asserted an obligation — 'a run that leans on a recalled
conclusion must say so in the summary' — that didn't exist in any procedure.
Rather than strip a good idea, implemented it: skills/offload-audit/SKILL.md §7
and agents/offload-analyst.md §5 now require exactly that disclosure.
Same class of error, caught independently: FR-7.3's new stability/tracker-
separation clause describes what the analyst's spoken report can do, not what
the rendered wiki page does — snapshot_candidate carries no stability field, and
findings from stability-classify never enter the snapshot. Documented the gap
and its mechanical cause (widening the snapshot schema, not the renderer) rather
than leaving the spec's claim wrong.
FR-6.5, the FR-9.3 wing/room table, and the Q1/Q3 resolutions the review added
were checked against the shipped code and are accurate; left unchanged.
Suite: 159 assertions, exit 0.
An audit that forgets every run makes the analyst re-derive the same judgment
calls from scratch each time, and loses the declined-candidate reasoning that is
the most expensive thing a run produces.
Two new audit-snapshot subcommands, both with `pages`' contract — filesystem in,
JSON out, no MCP call ever made here:
memory-queries what to search before grading
memory-notes the drawers and kg facts to write after the run
Placement: target-scoped findings go to the audited plugin's own wing (room
`inference-arbitrage-audits`), mirroring FR-6.1's "the finding belongs on the
target's surface"; cross-target calibration goes to `claude-plugins` (room
`inference-arbitrage-lessons`) on every run regardless of outcome. Rooms are
namespaced by this plugin's name because plugin-named wings really do collide
with unrelated ones (`cluster`, `imagex`), and a namespaced room makes that a
shelving overlap rather than a content collision.
Reading is advisory and never a gate: nothing recalled may override
boundary-classify (FR-4.2) or stability-classify (FR-4.5), excuse skipping the
marker search, or re-grade a returned verdict.
The memory renderers deliberately do NOT reuse render_run/render_latest.
`snapshot.notes` is caller-supplied free text that legitimately reaches a wiki
page in this plugin's own repo, and must never reach a drawer in a wing shared
with other projects. They read from a fixed allowlist of structured fields —
allowlist, never blocklist, since a blocklist is one new snapshot field away
from leaking.
The MCP calls are not covered by tests/memory.test.sh and are not faked there.
That is the point of putting all the judgment in the script: what remains
untestable is one verbatim tool call with nothing in it to get wrong. The live
round trip was verified by hand against the real anxious snapshot.
Phase 7 filed anxious/agent-wip/release-policy-derivation as `measured` at
11.94% of audited spend on a 30-day window. The same candidate over 7 days came
back `thin` at 0.00%. Only the date range differed. A human reading either issue
body alone cannot know the other exists, which is rubric.md §4a's failure mode
reached through the plugin's own headline number.
That was caught by a human running a second skeptical review — luck, not a
control. It is now mechanical, in the same register as FR-4.2's overrule-case
gate:
- bin/stability-classify matches each `file` candidate against the most recent
prior snapshot recording it (FR-5.2 identity) and downgrades it to a boundary
question if `measurement_strength` changed or the share crossed the 2% filing
threshold between windows. Movement within a label is a trend, not an
instability; no prior window is `unchecked`, not unstable (FR-2.4).
- The downgrade is a finding against THIS repo, not the target's — the target
did not change, the auditor described it two ways. Carries an `ia-stability`
marker so a re-run comments rather than duplicating.
- bin/ia_store.py factors the store and identity resolution out of
audit-snapshot so both scripts hash signatures identically. A divergence there
would make the gate match nothing and fail open.
Documented as spec FR-4.5 and rubric §5b (shipped verbatim in references/).
Wired into skills/offload-audit as step 4b, before filing-plan; steps 5-6 now
consume its output rather than classified.json.
S4 is retired in its old form — a human triage catching a bad candidate is not a
repeatable test — and re-passed as: the gate catches the real
release-policy-derivation case with no human in the loop. Asserted in
tests/stability.test.sh case (f) against the verbatim two-window Phase 7 output.
Closeskotkan/claude-plugin-inference-arbitrage#11
Suite: 178 assertions, exit 0.
Oleks