Files
claude-plugin-inference-arb…/tests/calibration/mask-worktree-audit.py
T
Oleks 105e9f5bae Phase 7 acceptance: two FR-5.4/S3 fidelity fixes found by running the real pipeline
S1-S3 and S5-S8 pass; S4 is prepared but deliberately unfiled pending user
sign-off (tasks.md 7.4 stays unchecked).

Two bugs surfaced by running the composed pipeline against real data rather
than synthetic fixtures, both in load-bearing places:

audit-snapshot: cost_per_invocation folded to 0 when a candidate had zero
invocations in a window, so a step that simply did not run reported -100% and
direction() called it `shrunk`. That is the quiet-window artefact FR-5.4 exists
to prevent, one level up from absolute tokens, and it contradicted direction()'s
own docstring. It is now None, metric_delta marks the pair `unmeasured`,
direction returns a new `unmeasured` status, and the pages render "not run".

tests/calibration/mask-worktree-audit.py: recomputed script_loc from scripts
alone, silently dropping all 1051 LOC of worktree-discipline's hooks. Masking
one 272-LOC script appeared to remove 1323, taking script_coverage 0.353 -> 0.083
instead of 0.297, which let S3 pass partly on an artefact of the mask. The mask
now matches build_aggregate. S3 still flags the known cut `high` /
llm-over-script-digest on the corrected, harder input.

Both fixes carry regression tests; suite is 156 assertions, exit 0.
2026-07-29 18:15:47 +03:00

52 lines
1.8 KiB
Python
Executable File

#!/usr/bin/env python3
"""Masks bin/worktree-audit out of a plugin-inventory JSON, simulating the state
before that script was written (spec S3).
Reads inventory JSON on stdin, writes the masked inventory on stdout. Removes the
script entry, recomputes script_loc and script_coverage, and drops command blocks
that invoke the masked script — a skill cannot prescribe a script that does not
exist. The target repo is never touched.
"""
import json
import sys
MASKED = "worktree-audit"
def main():
d = json.load(sys.stdin)
before = d["aggregate"]["script_loc"]
d["scripts"] = [s for s in d["scripts"] if not s["file"].endswith(MASKED)]
# Must match build_aggregate's definition, which counts hooks/ as shipped
# executable volume too — recomputing from scripts alone silently deletes
# every hook and overstates how script-starved the masked plugin is.
d["aggregate"]["script_loc"] = sum(s["loc"] for s in d["scripts"]) + sum(
h["loc"] for h in d.get("hooks", [])
)
body = d["aggregate"]["skill_body_words"] + d["aggregate"]["agent_body_words"]
d["aggregate"]["script_coverage"] = (
round(d["aggregate"]["script_loc"] / body, 3) if body else 0.0
)
removed = []
for sk in d["skills"]:
keep = [c for c in sk["command_blocks"] if MASKED not in c["sig"]]
removed += [c["sig"] for c in sk["command_blocks"] if c not in keep]
sk["command_blocks"] = keep
d["_mask"] = {
"masked_script": MASKED,
"script_loc_before": before,
"script_loc_after": d["aggregate"]["script_loc"],
"script_coverage_after": d["aggregate"]["script_coverage"],
"removed_command_blocks": removed,
}
json.dump(d, sys.stdout, indent=2)
sys.stdout.write("\n")
if __name__ == "__main__":
main()