claude-plugin-inference-arbitrage/skills/offload-audit/SKILL.md

name, description, allowed-tools

name	description	allowed-tools
offload-audit	Run a full offload audit on a Claude Code plugin — measure what its skills and agents actually cost, classify each candidate against the boundary rubric, file the well-evidenced ones as issues on the target's own repo, and write the run summary to this plugin's wiki. Trigger on "audit this plugin for token optimization", "can any of this be a script", "offload analysis", "where is this plugin wasting inference", "check <plugin> for scripting opportunities", "what is this plugin paying inference for", "run an offload audit". Read-only with respect to the audited plugin.	Bash, Read, Grep, Glob, Skill, Agent

Audit a plugin for work that should be a script

Seven steps, in order. Steps 1–3 and 4b are scripts, step 4 is the only judgment, steps 5–6 are delegation. Do not do by hand anything a bin/ tool does — this plugin is subject to its own thesis (FR-8), and an audit that counts things by inference is the finding, not the method.

Full rubric: the boundary-rubric skill. Deeper reporting discipline: agents/offload-analyst.md. Issue-body contract and the marker-search mechanics: ${CLAUDE_PLUGIN_ROOT}/references/issue-template.md.

---

1. Resolve and measure

IA=${CLAUDE_PLUGIN_ROOT}
$IA/bin/plugin-inventory <path|name> --json > inv.json      # static pass
$IA/bin/offload-scan --plugin <name> --days 30 --json > scan.json  # dynamic pass

plugin-inventory refuses a directory that is not a plugin rather than guessing (FR-1.3). A target with no transcript history is still auditable — the static pass stands alone and every candidate is marked unmeasured (FR-2.4).

2. Read the measurement honestly, before looking for candidates

Open the report with window, absolute evidence counts, the named structural blind spots (hooks, agents, direct commands/), and only then the coverage ratio — stated as dilution, not completeness.

Coverage answers "how much of these sessions was about something else", not "how much of this plugin's work did we see."

A real run against anxious measured 0.021. That is not a failed scan; it is long multi-topic sessions. Reporting it as completeness is the easiest way to mislead the reader. Then give the mechanical-share headline — the number that says whether there is anything here at all.

3. Gather candidates

From plugin-inventory: high verb ratio with no bin/ script behind it (a conjunction — never the ratio alone), duplicated command blocks, rule tables, low script coverage. From offload-scan: high MTR with real spend, recurring n-grams, read amplification, retry density, fan-out. judgment_density is a brake — high judgment density with high spend means the plugin is healthy and must be reported that way.

aggregate.hook_prose_drift entries are not offload candidates. Pass them with "category": "hook-prose-drift" so they come back as drift-note.

4. Judge, then grade mechanically

This is the only step that is inference. Invoke the boundary-rubric skill per candidate; write the five determinism tests, the falsifiability triple, the escalation path and digest schema into a judgments JSON (shape: tests/calibration/*.judgments.json), then:

$IA/bin/boundary-classify judgments.json > classified.json

Never grade by hand and never override the verdict it returns. The filing threshold (high confidence and ≥2% of audited spend and the triple complete including the overrule case) lives in that script.

No overrule case → boundary question, reported, never filed. Hard gate, FR-4.2. When genuinely torn, do not file.

4b. Check the evidence against the previous window

The second hard gate (FR-4.5, rubric §5b). It runs on every audit, before anything is filed, and its output replaces classified.json downstream:

$IA/bin/stability-classify --classified classified.json --target <name> \
    --scan scan.json > stable.json

It matches each file candidate against the most recent prior snapshot that recorded it (FR-5.2 identity) and downgrades any whose measurement_strength changed, or whose share crossed the 2% filing threshold, between the two windows. Use stable.json for steps 5 and 6. Never file from classified.json once this has run.

If the store has no history for this target — a first audit — every candidate comes back stability: unchecked and files normally. That is the FR-2.4 outcome, not a failure. If the store is cold but the wiki has history, seed it first, or the gate has nothing to check against:

$IA/bin/audit-snapshot import --target <name> --snapshots <fetched snapshots.jsonl>

4b-i. A downgrade is a finding against this plugin

Each entry in stability_findings is a defect in the auditor — the tool described the same candidate two different ways depending on the date range. File it, implicitly, on kotkan/claude-plugin-inference-arbitrage, not on the target's repo. Idempotency works the same way as FR-6.3: the finding carries self_report.marker (<!-- ia-stability: <candidate_id> -->); search for that marker first and comment on the existing issue rather than opening a second one.

Agent(subagent_type="anxious:issuer-agent", prompt=...)
  "On kotkan/claude-plugin-inference-arbitrage: search issues (state=all) for the
   literal marker <!-- ia-stability: <candidate_id> -->. If it exists, comment the
   new numbers on that issue. If not, open one — the marker must be the first line
   of the body, verbatim. This is an inference-arbitrage self-audit finding: a
   candidate's evidence label changed across measurement windows, so the gate
   refused to file it. Include the headline, both windows, and both figures."

Do not report the instability as a caveat inside a target-repo issue. A hedge a reader can skip is not a gate.

5. Output path A — file the issues

Filing is implicit. Do not ask permission to file a candidate that cleared the gate — this matches the environment's standing bug-handling convention. What you must not do is file anything the gate rejected.

This plugin holds no Gitea credentials and calls no Gitea MCP tool. The chain is fixed:

offload-audit → Agent(anxious:issuer-agent) → cluster:gitea-agent → Gitea

5a. Resolve the repo and get the searches

$IA/bin/filing-plan queries \
    --classified stable.json --judgments judgments.json \
    --target-path <resolved plugin path> > queries.json

If filing is unavailable (FR-6.4 — a third-party or marketplace plugin with no repo this environment can write to), skip the rest of path A entirely, carry the reason into the report and into the run summary, and go to step 6. Path B still runs. This is a normal outcome, not an error.

5b. Run the marker search — the step that prevents duplicates

Delegate the searches in queries.json. Both queries, merged; the label scan is the authoritative one. Bodies are not in listing responses, so each candidate issue needs an issue_read:

Agent(subagent_type="anxious:issuer-agent", prompt=...)
  "Read-only lookup on <owner/repo>, no writes. For an inference-arbitrage
   audit I need to know whether these candidates already have issues.
   1. list_issues(owner, repo, labels=['token-offload'], state='all')
   2. search_issues(query='ia-candidate: <id>', owner, state='all')  [per id]
   3. For each issue number found: issue_read(method='get')  → body
      and issue_read(method='get_comments') → comment bodies
   Return a JSON array of {number, state, title, body, comments:[{body}]}.
   Do not create, edit, or label anything."

Write the returned array to existing.json. If the search fails or you cannot get bodies, stop path A and report it — do not proceed with an empty array. filing-plan requires --existing precisely so that a failed search cannot silently become a duplicate-filing spree (FR-6.3).

5c. Build the plan

$IA/bin/filing-plan plan \
    --classified stable.json --judgments judgments.json \
    --target-path <path> --existing existing.json \
    --audit-page "Audits/<target>/<YYYY-MM-DD>" \
    --window "<since> -> <until>" --run-date <YYYY-MM-DD> \
    --status <snapshot-diff status, if known> > filing.json

Each action is file, comment, or skip, with the exact body already rendered. Execute the plan verbatim — do not rewrite a body, and never turn a comment into a file. Surface any warnings to the user.

5d. Execute through issuer

One Agent call per action (or one call carrying all of them):

Agent(subagent_type="anxious:issuer-agent", prompt=...)
  # for action == "file"
  "File this on <owner/repo>. It is an inference-arbitrage audit finding.
   Proposed title: <proposed_title>   (yours to rewrite per your taxonomy)
   Body — file VERBATIM, the first line is an idempotency marker that a later
   audit searches for and MUST NOT be altered or reformatted:
   <body>
   Labels: 'token-offload' is required — create it on this repo if absent
   (colour #8b5cf6, 'A step paying for inference where a deterministic script
   would do'). Add the four-axis taxonomy labels and any milestone per your own
   rules; typically kind/chore or kind/capability-gap, area/agent-behavior,
   domain/agents, activity/automate.
   Return the issue reference in full owner/repo#num form."

  # for action == "comment"
  "Add this comment to <issue>, an existing inference-arbitrage finding being
   re-measured. Do not open a new issue. Post the body VERBATIM (the first line
   is an idempotency marker). If the issue is closed, use your judgement about
   reopening; the audit does not require it.
   <body>"

issuer owns repo, final title, labels beyond token-offload, and milestone. You own the body.

Collect the returned refs into issue-map.json — {"<candidate_id>": "owner/repo#N"}.

6. Output path B — the run summary

Path B runs after path A so the summary records real issue numbers, and it runs even when path A was skipped.

$IA/bin/audit-snapshot write --target <name> --from inv.json scan.json \
    --classified stable.json --issue-map issue-map.json \
    --repo <owner/repo> \
    --note "<anything the scripts cannot see — see below>"
$IA/bin/audit-snapshot pages --target <name> > pages.json

audit-snapshot renders Audits/<target>/<YYYY-MM-DD>, regenerates Audits/<target>/Latest, and appends the machine-readable record to Data/<target>/snapshots.jsonl. It does not write to Gitea either — delegate each page in pages.json to cluster:gitea-agent for the wiki of kotkan/claude-plugin-inference-arbitrage, respecting append_only on the JSONL.

--note is repeatable and is how a filing outcome reaches the page — the snapshot scripts make no Gitea calls and cannot see one. When filing was unavailable, that note is required (FR-6.4):

--note "filing: unavailable — <the reason from filing-plan, verbatim>"

Use it likewise for a search that failed, or a plan you could not fully execute. Say it in the report either way; the note is what makes it durable.

Publishing gotcha: wiki_write create/update take the page title, but delete takes the URL-decoded sub_url (Data/<target>/snapshots.jsonl.-) — deleting by title 404s.

7. Report to the user

In this order (FR-7.3): coverage and window → mechanical-share headline → ranked candidate table with position, confidence and value → diff against the previous snapshot → boundary questions, as questions → drift notes, with their documentation fix → issues filed or updated, in full owner/repo#num form → the wiki page path.

Rank by measured offload_value and say that you are doing so. Name skills and agents, never UUIDs or session ids. Emit only shapes and counts — no transcript content, file contents, command arguments, or user prose ever enters a report, an issue, or a wiki page (FR-3.5).

"Nothing to offload here" is a complete answer

If nothing clears the bar, say so plainly and stop. Do not pad the report to look thorough. token-budget is expected to produce exactly this result, and a run against it that yields confident candidates is a bug in this plugin, not a finding about token-budget. Report the pure-inference findings by name — they are the evidence that you looked rather than shrugged.

Never

• Never edit the plugin you audit, and never implement your own recommendation.
• Never call a Gitea tool directly from this skill.
• Never file a candidate the gate rejected, or one whose repo is unavailable.
• Never re-run an audit "to be safe" without the marker search. That is the one mistake that makes this plugin hated.