FR-4.5: evidence-stability gate — refuse to file a candidate whose confidence moved with the window

Phase 7 filed anxious/agent-wip/release-policy-derivation as `measured` at
11.94% of audited spend on a 30-day window. The same candidate over 7 days came
back `thin` at 0.00%. Only the date range differed. A human reading either issue
body alone cannot know the other exists, which is rubric.md §4a's failure mode
reached through the plugin's own headline number.

That was caught by a human running a second skeptical review — luck, not a
control. It is now mechanical, in the same register as FR-4.2's overrule-case
gate:

- bin/stability-classify matches each `file` candidate against the most recent
  prior snapshot recording it (FR-5.2 identity) and downgrades it to a boundary
  question if `measurement_strength` changed or the share crossed the 2% filing
  threshold between windows. Movement within a label is a trend, not an
  instability; no prior window is `unchecked`, not unstable (FR-2.4).
- The downgrade is a finding against THIS repo, not the target's — the target
  did not change, the auditor described it two ways. Carries an `ia-stability`
  marker so a re-run comments rather than duplicating.
- bin/ia_store.py factors the store and identity resolution out of
  audit-snapshot so both scripts hash signatures identically. A divergence there
  would make the gate match nothing and fail open.

Documented as spec FR-4.5 and rubric §5b (shipped verbatim in references/).
Wired into skills/offload-audit as step 4b, before filing-plan; steps 5-6 now
consume its output rather than classified.json.

S4 is retired in its old form — a human triage catching a bad candidate is not a
repeatable test — and re-passed as: the gate catches the real
release-policy-derivation case with no human in the loop. Asserted in
tests/stability.test.sh case (f) against the verbatim two-window Phase 7 output.

Closes kotkan/claude-plugin-inference-arbitrage#11
Suite: 178 assertions, exit 0.
This commit is contained in:
oleks
2026-07-29 20:24:31 +03:00
committed by Oleks
parent 105e9f5bae
commit dcec9cabfb
16 changed files with 1119 additions and 123 deletions
+16
View File
@@ -158,6 +158,22 @@ Note the inverse too: if the overrule case is *common*, the candidate is not a
position-1 script — it is position 3, and the cut belongs earlier in the
pipeline.
### 4b. The second hard gate — evidence stability
> **Evidence-strength label or filing-threshold side changes across measurement
> windows → downgrade to a boundary question, never file (FR-4.5).**
`bin/stability-classify` enforces it, against the prior snapshot, and it runs
after `boundary-classify` and before anything is filed. The reason is the mirror
of §4's: the triple checks that the *reasoning* survives scrutiny, this checks
that the *evidence* does. A `measured` 12% that becomes a `thin` 0% on a
different date range was never a measurement of the target, and a human shown
only one of the two has no way to discount it.
When it fires, the finding is about **this plugin**, not the audited one — file
it on `kotkan/claude-plugin-inference-arbitrage`. Do not caveat the instability
into an issue body on the target's repo; a hedge a reader can skip is not a gate.
### 5. Report
- **Rank by measured `offload_value`, and say that you are doing so.** A reader