.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
 	"name": "inference-arbitrage",
-	"version": "0.7.1",
+	"version": "0.8.1",
 	"description": "Audits a Claude Code plugin's definitions and usage transcripts to find steps that should be a deterministic script instead of raw LLM inference, and files the well-evidenced ones as issues on the target repo.",
 	"author": {
 		"name": "oleks"

agents/offload-analyst.md

@@ -50,7 +50,9 @@ ${CLAUDE_PLUGIN_ROOT}/bin/offload-scan --plugin <name> --days N --json  # dynami
 - **Never implement your own recommendations.** You propose; a human or a
   separate session builds.
 - **Never count, parse, or aggregate by hand.** If you find yourself tallying
-  something, a `bin/` tool should be doing it.
+  something, a `bin/` tool should be doing it — including reshaping `inv.json`
+  and `scan.json` into a candidate table, which is now `bin/candidate-digest`
+  (kotkan/claude-plugin-inference-arbitrage#24), not something you do inline.
 - **Never grade by hand.** Judge the five tests, write the triple, then run
   `bin/boundary-classify`.
 
@@ -129,22 +131,47 @@ re-proposed as if new.
 
 ### 2. Gather candidates from three distinct sources
 
-**a. Static smells** (`plugin-inventory`). The loudest is a high verb ratio
-**with no `bin/` script behind it** — treat that as a conjunction, never the
-ratio alone. `token-budget` has verb ratios of 0.667 and 0.714 and is perfectly
-cut, because every command its skills prescribe is an invocation of a 606-line
-script that already exists. Also: `duplicated_command_blocks` (a shared script
-nobody wrote), `rule_tables` (a dispatch table being narrated at inference time),
-low `script_coverage`.
+**Never hand-reshape `inv.json` and `scan.json` into a candidate table.** That
+was this plugin's own worst offender — 40% of its audited spend
+(kotkan/claude-plugin-inference-arbitrage#24) — and it is now a script:
 
-**b. Dynamic smells** (`offload-scan`). High `mtr` with meaningful spend; a
-recurring `ngram` (an algorithm observed in the wild, which is the strongest
-evidence there is); high `read_amplification` (a digest belongs upstream); high
-`retry_density` (often just wants a thin wrapper that gets the invocation right
-once); `fanout_multiplier > 1` (mechanical work inside a subagent costs a
-multiple). **`judgment_density` is a brake, not an accelerator** — high judgment
-density with high spend means the plugin is doing what it should, and must be
-reported as **healthy**.
+```bash
+${CLAUDE_PLUGIN_ROOT}/bin/candidate-digest inv.json scan.json --json
+```
+
+It returns one `CandidateRow` per skill/agent with the shape fields below plus
+`has_bin_script` and `flags`. **It is a digest, not a verdict** (position 3):
+`flags` marks a row that crosses the same thresholds `boundary-classify`
+already applies — `MIN_INVOCATIONS`, `MIN_ATTRIBUTED_TURNS`,
+`MIN_SHARE_OF_SPEND` — plus the `judgment_density` brake. A flagged row is
+still yours to read, not yours to auto-file: the digest can only see
+token/turn shape, not intent, and a row can cross every mechanical threshold
+yet turn out to be this audit's own necessary two-pass discipline (read the
+static inventory, then the dynamic scan, then reshape) rather than a
+genuinely wasteful loop. That overrule case is exactly why the cut sits here
+and not in an auto-filing script.
+
+**a. Static smells** (`plugin-inventory`, `has_bin_script` in the digest). The
+loudest is a high verb ratio **with no `bin/` script behind it** — treat that
+as a conjunction, never the ratio alone. `token-budget` has verb ratios of
+0.667 and 0.714 and is perfectly cut, because every command its skills
+prescribe is an invocation of a 606-line script that already exists — the
+digest's `has_bin_script: true` suppresses exactly this shape. Also read
+directly from `plugin-inventory` (the digest does not carry these):
+`duplicated_command_blocks` (a shared script nobody wrote), `rule_tables` (a
+dispatch table being narrated at inference time), low `script_coverage`.
+
+**b. Dynamic smells** (`offload-scan`, most already surfaced in the digest).
+High `mtr` with meaningful spend; a recurring `ngram`
+(`dominant_ngram_recurrences` — an algorithm observed in the wild, which is
+the strongest evidence there is); high `read_amplification` (a digest belongs
+upstream); high `retry_density` (often just wants a thin wrapper that gets the
+invocation right once); `fanout_multiplier > 1` (mechanical work inside a
+subagent costs a multiple). **`judgment_density` is a brake, not an
+accelerator** — high judgment density with high spend means the plugin is
+doing what it should, and must be reported as **healthy**; the digest's
+`flags` already encode this (a row with `judgment_density >= 0.3` is never
+flagged), but confirm it by eye on anything you file.
 
 **c. Hook/prose drift — its own category, framed differently.** Entries in
 `aggregate.hook_prose_drift` are **restated configuration contracts, not

bin/candidate-digest

@@ -0,0 +1,198 @@
+#!/usr/bin/env python3
+"""Joins bin/plugin-inventory's PluginInventory and bin/offload-scan's
+OffloadScan into a candidate table -- the reshaping the offload-analyst agent
+used to do by hand on every single audit run (40.09% of this plugin's own
+audited spend, kotkan/claude-plugin-inference-arbitrage#24).
+
+This is a DIGEST, not an auto-filing script: position 3 of the rubric
+(references/boundary-rubric.md), never position 1. It computes SHAPE -- which
+rows cross the codified mechanical thresholds -- and leaves intent to a
+human/the analyst. A row can cross every mechanical threshold (share, mtr,
+read-amplification, a recurring n-gram) and still turn out to be this audit's
+own necessary two-pass discipline (read the static inventory, then the
+dynamic scan, then reshape both) rather than a genuinely wasteful loop. The
+digest cannot see intent, only shape, so `flags` is advisory input to a
+human's judgment call -- never a verdict, and never wired to filing.
+`bin/boundary-classify` remains the only place a verdict is computed; this
+script does not import it for grading, only for its two evidence-strength
+constants (see below).
+
+Usage: candidate-digest <inv.json|-> <scan.json> [--json]
+"""
+import argparse
+import importlib.machinery
+import importlib.util
+import json
+import os
+import sys
+
+BIN_DIR = os.path.dirname(os.path.abspath(__file__))
+
+# ia_store.py has no hyphen, so a normal import works once bin/ is on
+# sys.path -- which it already is (sys.path[0]) when this file is run as
+# `bin/candidate-digest`, per ia_store.py's own docstring.
+from ia_store import MIN_SHARE_OF_SPEND, scan_rows  # noqa: E402
+
+
+def _load_bin_module(filename):
+    """boundary-classify has a hyphenated filename and can't be `import`ed by
+    name -- loaded the same way offload-scan loads token-budget's cc-tokens
+    (resolve_cc_tokens there), so this plugin has exactly one pattern for
+    "import a sibling bin/ script as a module"."""
+    path = os.path.join(BIN_DIR, filename)
+    mod_name = filename.replace("-", "_")
+    spec = importlib.util.spec_from_loader(
+        mod_name, importlib.machinery.SourceFileLoader(mod_name, path))
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+_boundary_classify = _load_bin_module("boundary-classify")
+# Reused, not reinvented (kotkan/claude-plugin-inference-arbitrage#24 says to
+# reuse these): the same bar boundary-classify already applies to call a
+# candidate's evidence `measured` rather than `thin`.
+MIN_INVOCATIONS = _boundary_classify.MIN_INVOCATIONS
+MIN_ATTRIBUTED_TURNS = _boundary_classify.MIN_ATTRIBUTED_TURNS
+
+# New in this script -- no existing catalog threshold covers "how much
+# judgment_density is enough to call a recurring shape genuine work rather
+# than waste". The issue's own worked example put a real non-flag case at
+# judgment_density=0.55 and a real flag case at 0.034; 0.3 sits well clear of
+# both and reads naturally as "under a third of this row's turns were
+# judgment calls". This is the judgment_density BRAKE
+# (agents/offload-analyst.md: "a brake, not an accelerator") -- it can only
+# suppress a flag, never add one.
+JUDGMENT_DENSITY_BRAKE = 0.3
+
+
+def _has_bin_script(name, kind, inv):
+    """True only when plugin-inventory's OWN static evidence shows this row's
+    documented procedure already delegates to a bin/ script.
+
+    Skills carry `command_blocks` (the fenced code in their SKILL.md); a
+    block whose first line names a `bin/` path means the skill is *already*
+    a thin wrapper around an existing script -- the
+    "verb_ratio=0.71 backed by a 606-LOC script" example from the issue,
+    where the conjunction (high verb ratio AND no script) fails and nothing
+    should be surfaced.
+
+    Agents carry no body or command-block text in plugin-inventory's JSON at
+    all -- only skills do -- so this is conservatively False for every agent
+    row. That is a real limit on what this script can see, not a judgment
+    call it is dodging: an analyst who knows a given agent already delegates
+    to bin/ overrides this field by hand, same as any other digest output.
+    """
+    if kind != "skill":
+        return False
+    for skill in inv.get("skills", []):
+        if skill.get("name") == name:
+            return any("bin/" in b.get("sig", "")
+                       for b in skill.get("command_blocks", []))
+    return False
+
+
+def _dominant_ngram_recurrences(row):
+    ngrams = row.get("ngrams") or []
+    if not ngrams:
+        return None
+    return max(g.get("recurrences", 0) for g in ngrams)
+
+
+def _flags(share, invocations, turns, judgment_density, dominant_ngram,
+          has_bin_script):
+    """The escalation path documented in kotkan/claude-plugin-inference-
+    arbitrage#24: flag a row when it crosses the same thresholds
+    boundary-classify already uses for evidence strength and filing share
+    (MIN_INVOCATIONS, MIN_ATTRIBUTED_TURNS, MIN_SHARE_OF_SPEND), shows a
+    dominant recurring n-gram, AND judgment_density sits below the brake.
+
+    `judgment_density` is a brake, not an accelerator: a row that crosses
+    every other threshold but carries real judgment density is the healthy
+    case (this audit's own two-pass read-then-reshape discipline, not a
+    wasteful loop) and MUST come back with flags=[] -- that is the low_judgment
+    check below, and it is the whole reason this is a digest and not an
+    auto-filing script.
+
+    A row already backed by a bin/ script (has_bin_script) is never flagged
+    at all: whatever mechanical share it carries is already offloaded, so
+    there is nothing left here to propose.
+    """
+    if has_bin_script:
+        return []
+    measured = invocations >= MIN_INVOCATIONS and turns >= MIN_ATTRIBUTED_TURNS
+    significant_share = share >= MIN_SHARE_OF_SPEND
+    recurring = dominant_ngram is not None and dominant_ngram > 1
+    low_judgment = judgment_density < JUDGMENT_DENSITY_BRAKE
+    flags = []
+    if measured and significant_share and recurring and low_judgment:
+        flags.append("high-share-low-judgment-recurring-ngram")
+    return flags
+
+
+def candidate_digest(inv, scan):
+    """PluginInventory, OffloadScan -> list[CandidateRow]
+    (kotkan/claude-plugin-inference-arbitrage#24).
+
+    Pure function of its two already-computed JSON documents -- no
+    filesystem access beyond what the caller already read, no network, no
+    re-derivation of anything plugin-inventory or offload-scan already
+    computed. This is the entire join the offload-analyst agent used to
+    reshape by hand every audit run.
+
+    CandidateRow = {row_kind: 'skill'|'agent', name: str,
+    share_of_spend: float, mtr: float, judgment_density: float,
+    read_amplification: float, retry_density: float,
+    fanout_multiplier: float, dominant_ngram_recurrences: int|None,
+    has_bin_script: bool, flags: list[str]}
+    """
+    rows = []
+    for name, row in scan_rows(scan):
+        # scan_rows() (ia_store.py) yields every scan["skills"] row keyed by
+        # row["skill"] and every scan["agents"] row keyed by row["agent"];
+        # the row itself still carries whichever of those two keys it came
+        # from, which is the only place kind is recoverable from here.
+        kind = "agent" if "agent" in row else "skill"
+        has_bin = _has_bin_script(name, kind, inv)
+        dominant = _dominant_ngram_recurrences(row)
+        share = row.get("share_of_plugin", 0.0)
+        judgment_density = row.get("judgment_density", 0.0)
+        rows.append({
+            "row_kind": kind,
+            "name": name,
+            "share_of_spend": share,
+            "mtr": row.get("mtr", 0.0),
+            "judgment_density": judgment_density,
+            "read_amplification": row.get("read_amplification", 0.0),
+            "retry_density": row.get("retry_density", 0.0),
+            "fanout_multiplier": row.get("fanout_multiplier", 1.0),
+            "dominant_ngram_recurrences": dominant,
+            "has_bin_script": has_bin,
+            "flags": _flags(share, row.get("invocations", 0),
+                            row.get("turns", 0), judgment_density,
+                            dominant, has_bin),
+        })
+    rows.sort(key=lambda r: -r["share_of_spend"])
+    return rows
+
+
+def main():
+    ap = argparse.ArgumentParser(prog="candidate-digest")
+    ap.add_argument("inventory",
+                    help="path to plugin-inventory --json output, or - for stdin")
+    ap.add_argument("scan", help="path to offload-scan --json output")
+    ap.add_argument("--json", action="store_true", help="compact single-line JSON")
+    args = ap.parse_args()
+
+    with (sys.stdin if args.inventory == "-" else open(args.inventory)) as fh:
+        inv = json.load(fh)
+    with open(args.scan) as fh:
+        scan = json.load(fh)
+
+    rows = candidate_digest(inv, scan)
+    json.dump(rows, sys.stdout, indent=None if args.json else 2)
+    sys.stdout.write("\n")
+
+
+if __name__ == "__main__":
+    main()

skills/offload-audit/SKILL.md

@@ -78,15 +78,36 @@ with `--note` so the wiki page records that this run is missing from memory.
 
 ## 3. Gather candidates
 
-From `plugin-inventory`: high verb ratio **with no `bin/` script behind it**
-(a conjunction — never the ratio alone), duplicated command blocks, rule tables,
-low script coverage. From `offload-scan`: high MTR with real spend, recurring
-n-grams, read amplification, retry density, fan-out. `judgment_density` is a
-**brake** — high judgment density with high spend means the plugin is healthy
-and must be reported that way.
+**Never hand-reshape `inv.json` and `scan.json` into a candidate table.**
+That reshaping used to be done by inference on every single audit run and was
+itself 40% of this plugin's own audited spend
+(kotkan/claude-plugin-inference-arbitrage#24). Run the join instead:
 
-`aggregate.hook_prose_drift` entries are **not offload candidates.** Pass them
-with `"category": "hook-prose-drift"` so they come back as `drift-note`.
+```bash
+$IA/bin/candidate-digest inv.json scan.json --json > digest.json
+```
+
+This produces one `CandidateRow` per skill/agent (`row_kind`, `name`,
+`share_of_spend`, `mtr`, `judgment_density`, `read_amplification`,
+`retry_density`, `fanout_multiplier`, `dominant_ngram_recurrences`,
+`has_bin_script`, `flags`). It is a **digest, not a verdict** — position 3, the
+same as this whole plugin's own thesis. A `flags` entry is a shape the digest
+noticed crossing the same thresholds `boundary-classify` already applies
+(`MIN_INVOCATIONS`, `MIN_ATTRIBUTED_TURNS`, `MIN_SHARE_OF_SPEND`) plus the
+`judgment_density` brake; it is **never** grounds to skip step 4's judgment or
+to file directly. Read every flagged row's actual n-gram/tool-call shape before
+deciding candidate vs. this audit's own necessary two-pass discipline — the
+overrule case `bin/boundary-classify`'s hard gate (§4 below) exists for.
+
+Also read `plugin-inventory`'s own fields the digest does not carry: duplicated
+command blocks, rule tables, low script coverage. `judgment_density` is a
+**brake, not an accelerator** — a flagged row with real judgment density
+(`>= 0.3`) comes back unflagged, and high judgment density with high spend
+means the plugin is healthy and must be reported that way.
+
+`aggregate.hook_prose_drift` entries are **not offload candidates** and never
+appear in `digest.json`. Pass them with `"category": "hook-prose-drift"` so
+they come back as `drift-note`.
 
 ## 4. Judge, then grade mechanically
 

tests/candidate-digest.test.sh

@@ -0,0 +1,163 @@
+#!/usr/bin/env bash
+# bin/candidate-digest: the inv.json/scan.json join
+# (kotkan/claude-plugin-inference-arbitrage#24). Synthetic fixtures for the
+# three worked examples from the issue body, plus the has_bin_script suppression.
+set -euo pipefail
+cd "$(dirname "${BASH_SOURCE[0]}")"
+
+BIN=../bin/candidate-digest
+tmp=$(mktemp -d)
+trap 'rm -rf "$tmp"' EXIT
+fail=0
+
+field() {
+	python3 -c "import json,sys;print(json.load(open(sys.argv[1]))[int(sys.argv[2])][sys.argv[3]])" "$@"
+}
+
+check() {
+	local what=$1 got=$2 want=$3
+	if [ "$got" = "$want" ]; then
+		echo "  ok   $what = $want"
+	else
+		echo "  FAIL $what = $got (want $want)" >&2
+		fail=1
+	fi
+}
+
+# ---------------------------------------------------------------------------
+# Example 1: verb_ratio=0.71 skill backed by a bin/ script -> has_bin_script
+# True, flags=[] regardless of its dynamic numbers (conjunction fails).
+# Example 2: agent row, mtr=0.17 share=0.40 judgment_density=0.034 with a
+# dominant recurring 8-gram -> flags=['high-share-low-judgment-recurring-ngram'].
+# Example 3 (EDGE): same mtr/share/ngram but judgment_density=0.55 -> flags=[]
+# (the judgment_density brake suppresses the false positive).
+# ---------------------------------------------------------------------------
+
+cat >"$tmp/inv.json" <<'EOF'
+{
+  "plugin": {"name": "synthetic"},
+  "skills": [
+    {
+      "name": "syn:scripted-skill",
+      "verb_ratio": 0.71,
+      "command_blocks": [{"sig": "${CLAUDE_PLUGIN_ROOT}/bin/do-the-thing --json", "count": 1}]
+    }
+  ]
+}
+EOF
+
+cat >"$tmp/scan.json" <<'EOF'
+{
+  "skills": [
+    {
+      "skill": "syn:scripted-skill",
+      "invocations": 12,
+      "turns": 90,
+      "share_of_plugin": 0.5,
+      "mtr": 0.8,
+      "judgment_density": 0.05,
+      "read_amplification": 10.0,
+      "retry_density": 0.0,
+      "fanout_multiplier": 1.0,
+      "ngrams": [{"sig": ["a", "b", "c"], "recurrences": 9, "invocations": 5, "waste": 1000}]
+    }
+  ],
+  "agents": [
+    {
+      "agent": "syn:flagged-agent",
+      "invocations": 26,
+      "turns": 208,
+      "share_of_plugin": 0.40,
+      "mtr": 0.17,
+      "judgment_density": 0.034,
+      "read_amplification": 4.0,
+      "retry_density": 0.02,
+      "fanout_multiplier": 1.0,
+      "ngrams": [{"sig": ["x", "y"], "recurrences": 8, "invocations": 6, "waste": 800000}]
+    },
+    {
+      "agent": "syn:healthy-agent",
+      "invocations": 26,
+      "turns": 208,
+      "share_of_plugin": 0.40,
+      "mtr": 0.17,
+      "judgment_density": 0.55,
+      "read_amplification": 4.0,
+      "retry_density": 0.02,
+      "fanout_multiplier": 1.0,
+      "ngrams": [{"sig": ["x", "y"], "recurrences": 8, "invocations": 6, "waste": 800000}]
+    }
+  ]
+}
+EOF
+
+$BIN "$tmp/inv.json" "$tmp/scan.json" >"$tmp/out.json"
+
+echo "== example 1: scripted skill, high mtr/share, has_bin_script suppresses everything =="
+row=$(python3 -c "
+import json
+rows = json.load(open('$tmp/out.json'))
+print([i for i, r in enumerate(rows) if r['name'] == 'syn:scripted-skill'][0])
+")
+check "row_kind" "$(field "$tmp/out.json" "$row" row_kind)" "skill"
+check "has_bin_script" "$(field "$tmp/out.json" "$row" has_bin_script)" "True"
+check "flags" "$(field "$tmp/out.json" "$row" flags)" "[]"
+
+echo "== example 2: agent, judgment_density=0.034 -> flagged =="
+row=$(python3 -c "
+import json
+rows = json.load(open('$tmp/out.json'))
+print([i for i, r in enumerate(rows) if r['name'] == 'syn:flagged-agent'][0])
+")
+check "row_kind" "$(field "$tmp/out.json" "$row" row_kind)" "agent"
+check "has_bin_script" "$(field "$tmp/out.json" "$row" has_bin_script)" "False"
+check "dominant_ngram_recurrences" "$(field "$tmp/out.json" "$row" dominant_ngram_recurrences)" "8"
+check "flags" "$(field "$tmp/out.json" "$row" flags)" "['high-share-low-judgment-recurring-ngram']"
+
+echo "== example 3 (EDGE): same mtr/share/ngram, judgment_density=0.55 -> brake suppresses =="
+row=$(python3 -c "
+import json
+rows = json.load(open('$tmp/out.json'))
+print([i for i, r in enumerate(rows) if r['name'] == 'syn:healthy-agent'][0])
+")
+check "flags" "$(field "$tmp/out.json" "$row" flags)" "[]"
+
+echo "== below MIN_SHARE_OF_SPEND -> never flagged even with a low judgment_density =="
+cat >"$tmp/scan-thin.json" <<'EOF'
+{
+  "skills": [],
+  "agents": [
+    {
+      "agent": "syn:thin-agent",
+      "invocations": 12,
+      "turns": 90,
+      "share_of_plugin": 0.005,
+      "mtr": 0.9,
+      "judgment_density": 0.01,
+      "read_amplification": 3.0,
+      "retry_density": 0.0,
+      "fanout_multiplier": 1.0,
+      "ngrams": [{"sig": ["p", "q"], "recurrences": 5, "invocations": 4, "waste": 100}]
+    }
+  ]
+}
+EOF
+cat >"$tmp/inv-empty.json" <<'EOF'
+{"plugin": {"name": "synthetic"}, "skills": []}
+EOF
+$BIN "$tmp/inv-empty.json" "$tmp/scan-thin.json" >"$tmp/out-thin.json"
+check "flags" "$(field "$tmp/out-thin.json" 0 flags)" "[]"
+
+echo "== schema: every CandidateRow key present =="
+python3 -c "
+import json
+rows = json.load(open('$tmp/out.json'))
+want = {'row_kind', 'name', 'share_of_spend', 'mtr', 'judgment_density',
+        'read_amplification', 'retry_density', 'fanout_multiplier',
+        'dominant_ngram_recurrences', 'has_bin_script', 'flags'}
+for r in rows:
+    assert set(r.keys()) == want, (r.keys(), want)
+print('ok')
+"
+
+exit "$fail"