Phase 7 filed anxious/agent-wip/release-policy-derivation as `measured` at
11.94% of audited spend on a 30-day window. The same candidate over 7 days came
back `thin` at 0.00%. Only the date range differed. A human reading either issue
body alone cannot know the other exists, which is rubric.md §4a's failure mode
reached through the plugin's own headline number.
That was caught by a human running a second skeptical review — luck, not a
control. It is now mechanical, in the same register as FR-4.2's overrule-case
gate:
- bin/stability-classify matches each `file` candidate against the most recent
prior snapshot recording it (FR-5.2 identity) and downgrades it to a boundary
question if `measurement_strength` changed or the share crossed the 2% filing
threshold between windows. Movement within a label is a trend, not an
instability; no prior window is `unchecked`, not unstable (FR-2.4).
- The downgrade is a finding against THIS repo, not the target's — the target
did not change, the auditor described it two ways. Carries an `ia-stability`
marker so a re-run comments rather than duplicating.
- bin/ia_store.py factors the store and identity resolution out of
audit-snapshot so both scripts hash signatures identically. A divergence there
would make the gate match nothing and fail open.
Documented as spec FR-4.5 and rubric §5b (shipped verbatim in references/).
Wired into skills/offload-audit as step 4b, before filing-plan; steps 5-6 now
consume its output rather than classified.json.
S4 is retired in its old form — a human triage catching a bad candidate is not a
repeatable test — and re-passed as: the gate catches the real
release-policy-derivation case with no human in the loop. Asserted in
tests/stability.test.sh case (f) against the verbatim two-window Phase 7 output.
Closeskotkan/claude-plugin-inference-arbitrage#11
Suite: 178 assertions, exit 0.
S1-S3 and S5-S8 pass; S4 is prepared but deliberately unfiled pending user
sign-off (tasks.md 7.4 stays unchecked).
Two bugs surfaced by running the composed pipeline against real data rather
than synthetic fixtures, both in load-bearing places:
audit-snapshot: cost_per_invocation folded to 0 when a candidate had zero
invocations in a window, so a step that simply did not run reported -100% and
direction() called it `shrunk`. That is the quiet-window artefact FR-5.4 exists
to prevent, one level up from absolute tokens, and it contradicted direction()'s
own docstring. It is now None, metric_delta marks the pair `unmeasured`,
direction returns a new `unmeasured` status, and the pages render "not run".
tests/calibration/mask-worktree-audit.py: recomputed script_loc from scripts
alone, silently dropping all 1051 LOC of worktree-discipline's hooks. Masking
one 272-LOC script appeared to remove 1323, taking script_coverage 0.353 -> 0.083
instead of 0.297, which let S3 pass partly on an artefact of the mask. The mask
now matches build_aggregate. S3 still flags the known cut `high` /
llm-over-script-digest on the corrected, harder input.
Both fixes carry regression tests; suite is 156 assertions, exit 0.
FR-6.4 requires a target with no writable repo to record `filing: unavailable`
with its reason in the output. The filing pass knows that; audit-snapshot does
not, and `notes` was derived purely from no-offload/drift-note verdicts, so the
fact could reach the user's terminal but never the wiki page FR-6.4 points at.
Requested by the Phase 6 output-paths work.
bin/audit-snapshot write|list|diff, plus two subcommands that carry the wiki
bridge: `pages` emits what to publish and `import` seeds a cold cache from a
wiki-fetched snapshots.jsonl. The script is offline and filesystem-only — it
mirrors the wiki tree into a local store and never opens a socket, so the
plugin still holds no Gitea credentials and the push stays an explicit caller
action. Resolves the tension in plan §5, where "cloned and pushed" and "never
writes Gitea directly" cannot both be true of one process.
Identity resolves issue -> signature hash -> slug (FR-5.2); a hash change under
a stable identity is reported as signature-drifted rather than a silent
disappearance. Diff verdicts come from cost-per-invocation and share-of-spend
with a 10% noise band (FR-5.4) so a quiet window cannot read as progress.
`resolved` is the strict FR-5.5 conjunction — issue closed AND remeasured under
the same 2%/3-invocation bar boundary-classify uses to file; everything else
that merely stopped being reported is claimed-fixed-unconfirmed, naming the
missing leg.
Two additions to the plan §6 schema, both because a requirement is otherwise
uncomputable: target.skills (stale needs to tell "deleted" from "got cheap")
and measurements_by_signature (resolution needs a later window's cost for a
candidate that is gone).
references/boundary-rubric.md is design/rubric.md verbatim (4.1).
skills/boundary-rubric is usable standalone on a single step before any code
exists — the rubric is more valuable applied before the fact than after (4.2).
agents/offload-analyst classifies pre-computed inventory/scan JSON, leads with
coverage, treats hook/prose drift as its own category, and enforces the hard
gate: no overrule case -> boundary question, never filed (4.3).
bin/boundary-classify applies rubric §6's confidence table and the FR-4.2 filing
gate. The determinism-test outcomes and the falsifiability triple are inference;
grading them is a lookup, so it is a script rather than agent prose (FR-8.1).
Calibration gate (4.4) passes on real runs:
- token-budget -> 0 high-confidence, "nothing to offload here"
- worktree-discipline with bin/worktree-audit masked out
-> worktree-safety classification at high,
position llm-over-script-digest
tests/classify.test.sh asserts both gates plus synthetic table cases; 67
assertions green across the suite.
Streams ~/.claude/projects/**/*.jsonl once, filters to turns attributed to the
target plugin (attributionPlugin / attributionSkill / attributionAgent),
de-duplicates by (message.id, requestId) taking max on every usage field, and
computes the spec.md §5.2 dynamic signals: MTR, offload_waste, recurring
tool-signature n-grams, read amplification, retry density, judgment density,
fan-out multiplier, and the composite offload_value — plus the FR-3.4
attribution-coverage headline.
Token and cost arithmetic is taken from token-budget's cc-tokens, never
reimplemented. It is imported as a module rather than shelled out with --json,
because cc-tokens' subcommands aggregate by day/session/project/model and none
can answer "weighted tokens for this set of turns" — the attribution fields are
not on its CLI surface. Its Rec.cost / Rec.weighted still compute every number
here, so the pricing table and cache multipliers keep exactly one home.
Two deviations from design/plan.md §4.1, both because the transcripts say
otherwise: agents attribute via attributionAgent on the subagent sidechain
files, not agentName (which is a separate `type: "agent-name"` line mapping a
session to a display label); and attributionSkill is sometimes bare rather than
plugin-qualified, so it is normalized.
Tool arguments are masked to <path>/<n>/<sha>/<url>/<str> before entering any
emitted structure, and Bash commands are reduced to bare executable words —
FR-3.5, asserted in the suite. Fixtures are synthetic by construction; no real
transcript content is in this repo.
Real-history check: 30-day window over a 1.7 GB history, 94s wall, 29 MB peak
RSS, one process.
oleks