7 Commits

Author SHA1 Message Date
admin 83a74363c3 Merge pull request 'tests: make the worktree-discipline remote-resolution test hermetic (kotkan/claude-plugin-inference-arbitrage#30)' (#32) from fix/issue-30-stale-repo-name into main
ci/woodpecker/push/test Pipeline was successful
2026-07-30 23:40:49 +03:00
Oleks 6901f924c1 ci: install git — required by the new hermetic filing.test.sh fixture (#30)
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
2026-07-30 23:32:53 +03:00
Oleks 832b07b248 tests: make the worktree-discipline remote-resolution test hermetic
ci/woodpecker/push/test Pipeline failed
ci/woodpecker/pr/test Pipeline failed
kotkan/claude-plugin-inference-arbitrage#30 reported "FAIL repo =
kotkan/worktree-discipline (want kotkan/claude-plugin-worktree-discipline)".
Investigation found no stale reference in this repo's source or fixtures —
the expected value already matches the live repo (verified: only
kotkan/claude-plugin-worktree-discipline exists on Gitea; the pre-rename
bare-name slug is a redirect only). The failure came entirely from the live
git remote of the sibling ../worktree-discipline checkout on disk, which is
outside this repo and outside version control, so it can drift independently
(e.g. an old clone never repointed after the 2026-07-22 kotkan rename). That
also meant the assertion silently never ran in CI, since the bare CI
container has no sibling checkout and the test just printed "skip".

Replace the dependency on that external checkout with a throwaway git repo
created inside the test's own tmpdir, with its remote set explicitly to the
current repo slug. Same code path (resolve_repo/parse_remote in
bin/filing-plan) is exercised deterministically, and the assertion now
always runs instead of skipping outside a workspace with that sibling
checkout present.

Verified the new test both passes against the correct remote and correctly
reproduces the exact original failure signature when pointed at the stale
pre-rename slug.
2026-07-30 23:27:11 +03:00
oleks 0482278270 Merge pull request 'Add bin/candidate-digest: script the inv.json/scan.json join' (#28) from fix/issue-24-candidate-digest into main
ci/woodpecker/push/test Pipeline was successful
2026-07-30 19:57:32 +03:00
Oleks 230a44348c Merge remote-tracking branch 'origin/main' into fix/issue-24-candidate-digest
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
# Conflicts:
#	.claude-plugin/plugin.json
2026-07-30 19:55:09 +03:00
oleks 9a25402d37 Merge pull request 'Fix ia-candidate marker HTML-comment wrapper loss (kotkan/claude-plugin-inference-arbitrage#23)' (#27) from fix/issue-23-marker-wrapper into main
ci/woodpecker/push/test Pipeline was successful
2026-07-30 19:54:27 +03:00
Oleks c4389d8bb1 Add bin/candidate-digest: script the inv.json/scan.json join
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
The offload-analyst agent was hand-reshaping bin/plugin-inventory's
PluginInventory and bin/offload-scan's OffloadScan into a candidate
table on every audit run — 40.09% of this plugin's own audited spend
over 26 invocations (kotkan/claude-plugin-inference-arbitrage#24).

candidate_digest(inv, scan) -> list[CandidateRow] joins the two JSON
documents and computes shape: share_of_spend, mtr, judgment_density,
read_amplification, retry_density, fanout_multiplier,
dominant_ngram_recurrences, has_bin_script, and an advisory `flags`
list. It reuses boundary-classify's MIN_INVOCATIONS/
MIN_ATTRIBUTED_TURNS and ia_store's MIN_SHARE_OF_SPEND rather than
reinventing thresholds, and applies the judgment_density brake
(new JUDGMENT_DENSITY_BRAKE=0.3) so a row with real judgment density
never gets flagged even if every mechanical threshold is crossed.

This is a digest, not an auto-filing script (position 3, never
position 1): flags are advisory input to the analyst's own reading of
the actual n-gram/tool-call shape, never a verdict on their own.
bin/boundary-classify remains the only place a verdict is computed.

Wired into skills/offload-audit/SKILL.md step 3 and
agents/offload-analyst.md section 2 so future audit runs call the
script instead of reshaping by hand.

Tests: tests/candidate-digest.test.sh covers the issue's three worked
examples (scripted skill suppressed via has_bin_script, a flagged
agent row, and the judgment_density-brake-suppressed edge case) plus
a below-MIN_SHARE_OF_SPEND case and a full CandidateRow schema check.
Also verified against a real matched inv.json/scan.json pair from a
prior anxious audit in /tmp.
2026-07-30 19:27:09 +03:00
7 changed files with 460 additions and 36 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "inference-arbitrage",
"version": "0.7.1",
"version": "0.8.1",
"description": "Audits a Claude Code plugin's definitions and usage transcripts to find steps that should be a deterministic script instead of raw LLM inference, and files the well-evidenced ones as issues on the target repo.",
"author": {
"name": "oleks"
+3 -1
View File
@@ -14,7 +14,9 @@ steps:
- name: suites
image: alpine:3
commands:
- apk add --no-cache bash python3 wget >/dev/null
# git is required by tests/filing.test.sh's hermetic worktree-discipline
# remote-resolution fixture (kotkan/claude-plugin-inference-arbitrage#30).
- apk add --no-cache bash python3 wget git >/dev/null
# bin/offload-scan (design decision 0.1) deliberately never vendors
# token-budget's pricing/token-accounting logic -- it imports the real
# cc-tokens module and fails loudly if it's missing. This bare alpine
+43 -16
View File
@@ -50,7 +50,9 @@ ${CLAUDE_PLUGIN_ROOT}/bin/offload-scan --plugin <name> --days N --json # dynami
- **Never implement your own recommendations.** You propose; a human or a
separate session builds.
- **Never count, parse, or aggregate by hand.** If you find yourself tallying
something, a `bin/` tool should be doing it.
something, a `bin/` tool should be doing it — including reshaping `inv.json`
and `scan.json` into a candidate table, which is now `bin/candidate-digest`
(kotkan/claude-plugin-inference-arbitrage#24), not something you do inline.
- **Never grade by hand.** Judge the five tests, write the triple, then run
`bin/boundary-classify`.
@@ -129,22 +131,47 @@ re-proposed as if new.
### 2. Gather candidates from three distinct sources
**a. Static smells** (`plugin-inventory`). The loudest is a high verb ratio
**with no `bin/` script behind it** — treat that as a conjunction, never the
ratio alone. `token-budget` has verb ratios of 0.667 and 0.714 and is perfectly
cut, because every command its skills prescribe is an invocation of a 606-line
script that already exists. Also: `duplicated_command_blocks` (a shared script
nobody wrote), `rule_tables` (a dispatch table being narrated at inference time),
low `script_coverage`.
**Never hand-reshape `inv.json` and `scan.json` into a candidate table.** That
was this plugin's own worst offender — 40% of its audited spend
(kotkan/claude-plugin-inference-arbitrage#24) — and it is now a script:
**b. Dynamic smells** (`offload-scan`). High `mtr` with meaningful spend; a
recurring `ngram` (an algorithm observed in the wild, which is the strongest
evidence there is); high `read_amplification` (a digest belongs upstream); high
`retry_density` (often just wants a thin wrapper that gets the invocation right
once); `fanout_multiplier > 1` (mechanical work inside a subagent costs a
multiple). **`judgment_density` is a brake, not an accelerator** — high judgment
density with high spend means the plugin is doing what it should, and must be
reported as **healthy**.
```bash
${CLAUDE_PLUGIN_ROOT}/bin/candidate-digest inv.json scan.json --json
```
It returns one `CandidateRow` per skill/agent with the shape fields below plus
`has_bin_script` and `flags`. **It is a digest, not a verdict** (position 3):
`flags` marks a row that crosses the same thresholds `boundary-classify`
already applies — `MIN_INVOCATIONS`, `MIN_ATTRIBUTED_TURNS`,
`MIN_SHARE_OF_SPEND` — plus the `judgment_density` brake. A flagged row is
still yours to read, not yours to auto-file: the digest can only see
token/turn shape, not intent, and a row can cross every mechanical threshold
yet turn out to be this audit's own necessary two-pass discipline (read the
static inventory, then the dynamic scan, then reshape) rather than a
genuinely wasteful loop. That overrule case is exactly why the cut sits here
and not in an auto-filing script.
**a. Static smells** (`plugin-inventory`, `has_bin_script` in the digest). The
loudest is a high verb ratio **with no `bin/` script behind it** — treat that
as a conjunction, never the ratio alone. `token-budget` has verb ratios of
0.667 and 0.714 and is perfectly cut, because every command its skills
prescribe is an invocation of a 606-line script that already exists — the
digest's `has_bin_script: true` suppresses exactly this shape. Also read
directly from `plugin-inventory` (the digest does not carry these):
`duplicated_command_blocks` (a shared script nobody wrote), `rule_tables` (a
dispatch table being narrated at inference time), low `script_coverage`.
**b. Dynamic smells** (`offload-scan`, most already surfaced in the digest).
High `mtr` with meaningful spend; a recurring `ngram`
(`dominant_ngram_recurrences` — an algorithm observed in the wild, which is
the strongest evidence there is); high `read_amplification` (a digest belongs
upstream); high `retry_density` (often just wants a thin wrapper that gets the
invocation right once); `fanout_multiplier > 1` (mechanical work inside a
subagent costs a multiple). **`judgment_density` is a brake, not an
accelerator** — high judgment density with high spend means the plugin is
doing what it should, and must be reported as **healthy**; the digest's
`flags` already encode this (a row with `judgment_density >= 0.3` is never
flagged), but confirm it by eye on anything you file.
**c. Hook/prose drift — its own category, framed differently.** Entries in
`aggregate.hook_prose_drift` are **restated configuration contracts, not
+198
View File
@@ -0,0 +1,198 @@
#!/usr/bin/env python3
"""Joins bin/plugin-inventory's PluginInventory and bin/offload-scan's
OffloadScan into a candidate table -- the reshaping the offload-analyst agent
used to do by hand on every single audit run (40.09% of this plugin's own
audited spend, kotkan/claude-plugin-inference-arbitrage#24).
This is a DIGEST, not an auto-filing script: position 3 of the rubric
(references/boundary-rubric.md), never position 1. It computes SHAPE -- which
rows cross the codified mechanical thresholds -- and leaves intent to a
human/the analyst. A row can cross every mechanical threshold (share, mtr,
read-amplification, a recurring n-gram) and still turn out to be this audit's
own necessary two-pass discipline (read the static inventory, then the
dynamic scan, then reshape both) rather than a genuinely wasteful loop. The
digest cannot see intent, only shape, so `flags` is advisory input to a
human's judgment call -- never a verdict, and never wired to filing.
`bin/boundary-classify` remains the only place a verdict is computed; this
script does not import it for grading, only for its two evidence-strength
constants (see below).
Usage: candidate-digest <inv.json|-> <scan.json> [--json]
"""
import argparse
import importlib.machinery
import importlib.util
import json
import os
import sys
BIN_DIR = os.path.dirname(os.path.abspath(__file__))
# ia_store.py has no hyphen, so a normal import works once bin/ is on
# sys.path -- which it already is (sys.path[0]) when this file is run as
# `bin/candidate-digest`, per ia_store.py's own docstring.
from ia_store import MIN_SHARE_OF_SPEND, scan_rows # noqa: E402
def _load_bin_module(filename):
"""boundary-classify has a hyphenated filename and can't be `import`ed by
name -- loaded the same way offload-scan loads token-budget's cc-tokens
(resolve_cc_tokens there), so this plugin has exactly one pattern for
"import a sibling bin/ script as a module"."""
path = os.path.join(BIN_DIR, filename)
mod_name = filename.replace("-", "_")
spec = importlib.util.spec_from_loader(
mod_name, importlib.machinery.SourceFileLoader(mod_name, path))
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
_boundary_classify = _load_bin_module("boundary-classify")
# Reused, not reinvented (kotkan/claude-plugin-inference-arbitrage#24 says to
# reuse these): the same bar boundary-classify already applies to call a
# candidate's evidence `measured` rather than `thin`.
MIN_INVOCATIONS = _boundary_classify.MIN_INVOCATIONS
MIN_ATTRIBUTED_TURNS = _boundary_classify.MIN_ATTRIBUTED_TURNS
# New in this script -- no existing catalog threshold covers "how much
# judgment_density is enough to call a recurring shape genuine work rather
# than waste". The issue's own worked example put a real non-flag case at
# judgment_density=0.55 and a real flag case at 0.034; 0.3 sits well clear of
# both and reads naturally as "under a third of this row's turns were
# judgment calls". This is the judgment_density BRAKE
# (agents/offload-analyst.md: "a brake, not an accelerator") -- it can only
# suppress a flag, never add one.
JUDGMENT_DENSITY_BRAKE = 0.3
def _has_bin_script(name, kind, inv):
"""True only when plugin-inventory's OWN static evidence shows this row's
documented procedure already delegates to a bin/ script.
Skills carry `command_blocks` (the fenced code in their SKILL.md); a
block whose first line names a `bin/` path means the skill is *already*
a thin wrapper around an existing script -- the
"verb_ratio=0.71 backed by a 606-LOC script" example from the issue,
where the conjunction (high verb ratio AND no script) fails and nothing
should be surfaced.
Agents carry no body or command-block text in plugin-inventory's JSON at
all -- only skills do -- so this is conservatively False for every agent
row. That is a real limit on what this script can see, not a judgment
call it is dodging: an analyst who knows a given agent already delegates
to bin/ overrides this field by hand, same as any other digest output.
"""
if kind != "skill":
return False
for skill in inv.get("skills", []):
if skill.get("name") == name:
return any("bin/" in b.get("sig", "")
for b in skill.get("command_blocks", []))
return False
def _dominant_ngram_recurrences(row):
ngrams = row.get("ngrams") or []
if not ngrams:
return None
return max(g.get("recurrences", 0) for g in ngrams)
def _flags(share, invocations, turns, judgment_density, dominant_ngram,
has_bin_script):
"""The escalation path documented in kotkan/claude-plugin-inference-
arbitrage#24: flag a row when it crosses the same thresholds
boundary-classify already uses for evidence strength and filing share
(MIN_INVOCATIONS, MIN_ATTRIBUTED_TURNS, MIN_SHARE_OF_SPEND), shows a
dominant recurring n-gram, AND judgment_density sits below the brake.
`judgment_density` is a brake, not an accelerator: a row that crosses
every other threshold but carries real judgment density is the healthy
case (this audit's own two-pass read-then-reshape discipline, not a
wasteful loop) and MUST come back with flags=[] -- that is the low_judgment
check below, and it is the whole reason this is a digest and not an
auto-filing script.
A row already backed by a bin/ script (has_bin_script) is never flagged
at all: whatever mechanical share it carries is already offloaded, so
there is nothing left here to propose.
"""
if has_bin_script:
return []
measured = invocations >= MIN_INVOCATIONS and turns >= MIN_ATTRIBUTED_TURNS
significant_share = share >= MIN_SHARE_OF_SPEND
recurring = dominant_ngram is not None and dominant_ngram > 1
low_judgment = judgment_density < JUDGMENT_DENSITY_BRAKE
flags = []
if measured and significant_share and recurring and low_judgment:
flags.append("high-share-low-judgment-recurring-ngram")
return flags
def candidate_digest(inv, scan):
"""PluginInventory, OffloadScan -> list[CandidateRow]
(kotkan/claude-plugin-inference-arbitrage#24).
Pure function of its two already-computed JSON documents -- no
filesystem access beyond what the caller already read, no network, no
re-derivation of anything plugin-inventory or offload-scan already
computed. This is the entire join the offload-analyst agent used to
reshape by hand every audit run.
CandidateRow = {row_kind: 'skill'|'agent', name: str,
share_of_spend: float, mtr: float, judgment_density: float,
read_amplification: float, retry_density: float,
fanout_multiplier: float, dominant_ngram_recurrences: int|None,
has_bin_script: bool, flags: list[str]}
"""
rows = []
for name, row in scan_rows(scan):
# scan_rows() (ia_store.py) yields every scan["skills"] row keyed by
# row["skill"] and every scan["agents"] row keyed by row["agent"];
# the row itself still carries whichever of those two keys it came
# from, which is the only place kind is recoverable from here.
kind = "agent" if "agent" in row else "skill"
has_bin = _has_bin_script(name, kind, inv)
dominant = _dominant_ngram_recurrences(row)
share = row.get("share_of_plugin", 0.0)
judgment_density = row.get("judgment_density", 0.0)
rows.append({
"row_kind": kind,
"name": name,
"share_of_spend": share,
"mtr": row.get("mtr", 0.0),
"judgment_density": judgment_density,
"read_amplification": row.get("read_amplification", 0.0),
"retry_density": row.get("retry_density", 0.0),
"fanout_multiplier": row.get("fanout_multiplier", 1.0),
"dominant_ngram_recurrences": dominant,
"has_bin_script": has_bin,
"flags": _flags(share, row.get("invocations", 0),
row.get("turns", 0), judgment_density,
dominant, has_bin),
})
rows.sort(key=lambda r: -r["share_of_spend"])
return rows
def main():
ap = argparse.ArgumentParser(prog="candidate-digest")
ap.add_argument("inventory",
help="path to plugin-inventory --json output, or - for stdin")
ap.add_argument("scan", help="path to offload-scan --json output")
ap.add_argument("--json", action="store_true", help="compact single-line JSON")
args = ap.parse_args()
with (sys.stdin if args.inventory == "-" else open(args.inventory)) as fh:
inv = json.load(fh)
with open(args.scan) as fh:
scan = json.load(fh)
rows = candidate_digest(inv, scan)
json.dump(rows, sys.stdout, indent=None if args.json else 2)
sys.stdout.write("\n")
if __name__ == "__main__":
main()
+29 -8
View File
@@ -78,15 +78,36 @@ with `--note` so the wiki page records that this run is missing from memory.
## 3. Gather candidates
From `plugin-inventory`: high verb ratio **with no `bin/` script behind it**
(a conjunction — never the ratio alone), duplicated command blocks, rule tables,
low script coverage. From `offload-scan`: high MTR with real spend, recurring
n-grams, read amplification, retry density, fan-out. `judgment_density` is a
**brake** — high judgment density with high spend means the plugin is healthy
and must be reported that way.
**Never hand-reshape `inv.json` and `scan.json` into a candidate table.**
That reshaping used to be done by inference on every single audit run and was
itself 40% of this plugin's own audited spend
(kotkan/claude-plugin-inference-arbitrage#24). Run the join instead:
`aggregate.hook_prose_drift` entries are **not offload candidates.** Pass them
with `"category": "hook-prose-drift"` so they come back as `drift-note`.
```bash
$IA/bin/candidate-digest inv.json scan.json --json > digest.json
```
This produces one `CandidateRow` per skill/agent (`row_kind`, `name`,
`share_of_spend`, `mtr`, `judgment_density`, `read_amplification`,
`retry_density`, `fanout_multiplier`, `dominant_ngram_recurrences`,
`has_bin_script`, `flags`). It is a **digest, not a verdict** — position 3, the
same as this whole plugin's own thesis. A `flags` entry is a shape the digest
noticed crossing the same thresholds `boundary-classify` already applies
(`MIN_INVOCATIONS`, `MIN_ATTRIBUTED_TURNS`, `MIN_SHARE_OF_SPEND`) plus the
`judgment_density` brake; it is **never** grounds to skip step 4's judgment or
to file directly. Read every flagged row's actual n-gram/tool-call shape before
deciding candidate vs. this audit's own necessary two-pass discipline — the
overrule case `bin/boundary-classify`'s hard gate (§4 below) exists for.
Also read `plugin-inventory`'s own fields the digest does not carry: duplicated
command blocks, rule tables, low script coverage. `judgment_density` is a
**brake, not an accelerator** — a flagged row with real judgment density
(`>= 0.3`) comes back unflagged, and high judgment density with high spend
means the plugin is healthy and must be reported that way.
`aggregate.hook_prose_drift` entries are **not offload candidates** and never
appear in `digest.json`. Pass them with `"category": "hook-prose-drift"` so
they come back as `drift-note`.
## 4. Judge, then grade mechanically
+163
View File
@@ -0,0 +1,163 @@
#!/usr/bin/env bash
# bin/candidate-digest: the inv.json/scan.json join
# (kotkan/claude-plugin-inference-arbitrage#24). Synthetic fixtures for the
# three worked examples from the issue body, plus the has_bin_script suppression.
set -euo pipefail
cd "$(dirname "${BASH_SOURCE[0]}")"
BIN=../bin/candidate-digest
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
fail=0
field() {
python3 -c "import json,sys;print(json.load(open(sys.argv[1]))[int(sys.argv[2])][sys.argv[3]])" "$@"
}
check() {
local what=$1 got=$2 want=$3
if [ "$got" = "$want" ]; then
echo " ok $what = $want"
else
echo " FAIL $what = $got (want $want)" >&2
fail=1
fi
}
# ---------------------------------------------------------------------------
# Example 1: verb_ratio=0.71 skill backed by a bin/ script -> has_bin_script
# True, flags=[] regardless of its dynamic numbers (conjunction fails).
# Example 2: agent row, mtr=0.17 share=0.40 judgment_density=0.034 with a
# dominant recurring 8-gram -> flags=['high-share-low-judgment-recurring-ngram'].
# Example 3 (EDGE): same mtr/share/ngram but judgment_density=0.55 -> flags=[]
# (the judgment_density brake suppresses the false positive).
# ---------------------------------------------------------------------------
cat >"$tmp/inv.json" <<'EOF'
{
"plugin": {"name": "synthetic"},
"skills": [
{
"name": "syn:scripted-skill",
"verb_ratio": 0.71,
"command_blocks": [{"sig": "${CLAUDE_PLUGIN_ROOT}/bin/do-the-thing --json", "count": 1}]
}
]
}
EOF
cat >"$tmp/scan.json" <<'EOF'
{
"skills": [
{
"skill": "syn:scripted-skill",
"invocations": 12,
"turns": 90,
"share_of_plugin": 0.5,
"mtr": 0.8,
"judgment_density": 0.05,
"read_amplification": 10.0,
"retry_density": 0.0,
"fanout_multiplier": 1.0,
"ngrams": [{"sig": ["a", "b", "c"], "recurrences": 9, "invocations": 5, "waste": 1000}]
}
],
"agents": [
{
"agent": "syn:flagged-agent",
"invocations": 26,
"turns": 208,
"share_of_plugin": 0.40,
"mtr": 0.17,
"judgment_density": 0.034,
"read_amplification": 4.0,
"retry_density": 0.02,
"fanout_multiplier": 1.0,
"ngrams": [{"sig": ["x", "y"], "recurrences": 8, "invocations": 6, "waste": 800000}]
},
{
"agent": "syn:healthy-agent",
"invocations": 26,
"turns": 208,
"share_of_plugin": 0.40,
"mtr": 0.17,
"judgment_density": 0.55,
"read_amplification": 4.0,
"retry_density": 0.02,
"fanout_multiplier": 1.0,
"ngrams": [{"sig": ["x", "y"], "recurrences": 8, "invocations": 6, "waste": 800000}]
}
]
}
EOF
$BIN "$tmp/inv.json" "$tmp/scan.json" >"$tmp/out.json"
echo "== example 1: scripted skill, high mtr/share, has_bin_script suppresses everything =="
row=$(python3 -c "
import json
rows = json.load(open('$tmp/out.json'))
print([i for i, r in enumerate(rows) if r['name'] == 'syn:scripted-skill'][0])
")
check "row_kind" "$(field "$tmp/out.json" "$row" row_kind)" "skill"
check "has_bin_script" "$(field "$tmp/out.json" "$row" has_bin_script)" "True"
check "flags" "$(field "$tmp/out.json" "$row" flags)" "[]"
echo "== example 2: agent, judgment_density=0.034 -> flagged =="
row=$(python3 -c "
import json
rows = json.load(open('$tmp/out.json'))
print([i for i, r in enumerate(rows) if r['name'] == 'syn:flagged-agent'][0])
")
check "row_kind" "$(field "$tmp/out.json" "$row" row_kind)" "agent"
check "has_bin_script" "$(field "$tmp/out.json" "$row" has_bin_script)" "False"
check "dominant_ngram_recurrences" "$(field "$tmp/out.json" "$row" dominant_ngram_recurrences)" "8"
check "flags" "$(field "$tmp/out.json" "$row" flags)" "['high-share-low-judgment-recurring-ngram']"
echo "== example 3 (EDGE): same mtr/share/ngram, judgment_density=0.55 -> brake suppresses =="
row=$(python3 -c "
import json
rows = json.load(open('$tmp/out.json'))
print([i for i, r in enumerate(rows) if r['name'] == 'syn:healthy-agent'][0])
")
check "flags" "$(field "$tmp/out.json" "$row" flags)" "[]"
echo "== below MIN_SHARE_OF_SPEND -> never flagged even with a low judgment_density =="
cat >"$tmp/scan-thin.json" <<'EOF'
{
"skills": [],
"agents": [
{
"agent": "syn:thin-agent",
"invocations": 12,
"turns": 90,
"share_of_plugin": 0.005,
"mtr": 0.9,
"judgment_density": 0.01,
"read_amplification": 3.0,
"retry_density": 0.0,
"fanout_multiplier": 1.0,
"ngrams": [{"sig": ["p", "q"], "recurrences": 5, "invocations": 4, "waste": 100}]
}
]
}
EOF
cat >"$tmp/inv-empty.json" <<'EOF'
{"plugin": {"name": "synthetic"}, "skills": []}
EOF
$BIN "$tmp/inv-empty.json" "$tmp/scan-thin.json" >"$tmp/out-thin.json"
check "flags" "$(field "$tmp/out-thin.json" 0 flags)" "[]"
echo "== schema: every CandidateRow key present =="
python3 -c "
import json
rows = json.load(open('$tmp/out.json'))
want = {'row_kind', 'name', 'share_of_spend', 'mtr', 'judgment_density',
'read_amplification', 'retry_density', 'fanout_multiplier',
'dominant_ngram_recurrences', 'has_bin_script', 'flags'}
for r in rows:
assert set(r.keys()) == want, (r.keys(), want)
print('ok')
"
exit "$fail"
+23 -10
View File
@@ -201,15 +201,28 @@ else
fi
echo "== a writable target resolves through its own checkout's remote =="
if [ -d ../worktree-discipline/.git ]; then
$BIN queries --classified "$tmp/cls.json" --judgments "$JUD" \
--target-path ../worktree-discipline >"$tmp/wd-q.json"
check "repo" "$(q "$tmp/wd-q.json" "d['repo']")" "kotkan/claude-plugin-worktree-discipline"
check "filing" "$(q "$tmp/wd-q.json" "d['filing']")" "available"
check "label scan is the authoritative query" \
"$(q "$tmp/wd-q.json" "d['queries']['label_scan']['labels']")" "['token-offload']"
else
echo " skip (no sibling worktree-discipline checkout)"
fi
# Was: read the live remote of a sibling ../worktree-discipline checkout on
# disk. That checkout is outside this repo and outside version control, so
# its remote can drift independently of anything here — e.g. a stale clone
# still pointing at the pre-rename slug `kotkan/worktree-discipline` instead
# of `kotkan/claude-plugin-worktree-discipline` (kotkan org repos were
# renamed to the claude-plugin-<name> convention on 2026-07-22). That drift
# produced a false "repo" mismatch (kotkan/claude-plugin-inference-arbitrage#30)
# with nothing to fix in this repo's source or fixtures — both already agreed
# with the current live repo name. It also meant this assertion silently
# never ran in CI, since the bare CI container has no sibling checkout at all
# and the test just printed "skip". A synthetic git repo with a fixed remote
# tests the same resolve_repo()/parse_remote() code path deterministically,
# with no dependency on what else happens to be checked out next door.
wd_git="$tmp/wd-fixture"
mkdir -p "$wd_git"
git -C "$wd_git" init -q
git -C "$wd_git" remote add origin git@git.oleks.space:kotkan/claude-plugin-worktree-discipline.git
$BIN queries --classified "$tmp/cls.json" --judgments "$JUD" \
--target-path "$wd_git" >"$tmp/wd-q.json"
check "repo" "$(q "$tmp/wd-q.json" "d['repo']")" "kotkan/claude-plugin-worktree-discipline"
check "filing" "$(q "$tmp/wd-q.json" "d['filing']")" "available"
check "label scan is the authoritative query" \
"$(q "$tmp/wd-q.json" "d['queries']['label_scan']['labels']")" "['token-offload']"
exit "$fail"