kotkan/claude-plugin-inference-arbitrage
3
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Add bin/candidate-digest: script the inv.json/scan.json join #28

Merged
oleks merged 2 commits from fix/issue-24-candidate-digest into main 2026-07-30 19:57:32 +03:00
Conversation 0 Commits 2 Files Changed 5
+434 -25
oleks commented 2026-07-30 19:27:37 +03:00
Owner
Copy Link
Copy Source

Closes kotkan/claude-plugin-inference-arbitrage#24.

Summary

The offload-analyst agent was hand-reshaping bin/plugin-inventory's
PluginInventory and bin/offload-scan's OffloadScan into a candidate
table on every audit run — 40.09% of this plugin's own audited spend
(849.5k weighted tokens over 26 invocations).

Adds bin/candidate-digest inv.json scan.json --json, implementing
candidate_digest(inv, scan) -> list[CandidateRow] exactly as proposed in
the issue. It's a digest, not an auto-filing script (position 3, never
position 1):

  • reuses boundary-classify's MIN_INVOCATIONS / MIN_ATTRIBUTED_TURNS and
    ia_store's MIN_SHARE_OF_SPEND — no new share/invocation thresholds
  • applies the judgment_density brake (new JUDGMENT_DENSITY_BRAKE = 0.3,
    chosen to sit clear of the issue's worked example: 0.034 → flagged vs
    0.55 → not flagged) so a row that crosses every mechanical threshold but
    carries real judgment density comes back with flags: []
  • has_bin_script suppresses flags entirely for a skill whose own
    command_blocks already invoke a bin/ script (the "verb_ratio=0.71
    backed by a 606-LOC script" example) — conservatively False for agent
    rows, since plugin-inventory's JSON carries no agent body/command-block
    text to check
  • flags is explicitly documented as advisory input to the analyst's
    reading of the actual n-gram/tool-call shape, never a verdict —
    bin/boundary-classify remains the only place a verdict is computed

Wired into skills/offload-audit/SKILL.md step 3 and
agents/offload-analyst.md §2 so future audit runs call the script instead
of reshaping by hand. Version bumped 0.7.0 → 0.8.0 (new script + agent/skill
instruction changes).

Test plan

  • tests/candidate-digest.test.sh (new, auto-discovered by
    tests/run-all.sh): the issue's three worked examples — scripted
    skill suppressed via has_bin_script, a flagged agent row
    (judgment_density=0.034), the EDGE case suppressed by the brake
    (judgment_density=0.55) — plus a below-MIN_SHARE_OF_SPEND case and
    a full CandidateRow schema check. All pass.
  • Full suite (tests/run-all.sh) passes, exit 0, no regressions.
  • Verified against a real matched inv.json/scan.json pair from a
    prior anxious audit (/tmp/anxious-inventory.json +
    /tmp/anxious-scan.json) — sensible, correctly-sorted output with
    flags firing only on rows that actually cross every gate.

Fix is complete and self-verified; I believe it's ready to arm for
merge-when-checks-succeed, but leaving that to whoever reviews.

Closes kotkan/claude-plugin-inference-arbitrage#24. ## Summary The `offload-analyst` agent was hand-reshaping `bin/plugin-inventory`'s `PluginInventory` and `bin/offload-scan`'s `OffloadScan` into a candidate table on every audit run — 40.09% of this plugin's own audited spend (849.5k weighted tokens over 26 invocations). Adds `bin/candidate-digest inv.json scan.json --json`, implementing `candidate_digest(inv, scan) -> list[CandidateRow]` exactly as proposed in the issue. It's a **digest, not an auto-filing script** (position 3, never position 1): - reuses `boundary-classify`'s `MIN_INVOCATIONS` / `MIN_ATTRIBUTED_TURNS` and `ia_store`'s `MIN_SHARE_OF_SPEND` — no new share/invocation thresholds - applies the `judgment_density` brake (new `JUDGMENT_DENSITY_BRAKE = 0.3`, chosen to sit clear of the issue's worked example: 0.034 → flagged vs 0.55 → not flagged) so a row that crosses every mechanical threshold but carries real judgment density comes back with `flags: []` - `has_bin_script` suppresses flags entirely for a skill whose own `command_blocks` already invoke a `bin/` script (the "verb_ratio=0.71 backed by a 606-LOC script" example) — conservatively `False` for agent rows, since `plugin-inventory`'s JSON carries no agent body/command-block text to check - `flags` is explicitly documented as advisory input to the analyst's reading of the actual n-gram/tool-call shape, never a verdict — `bin/boundary-classify` remains the only place a verdict is computed Wired into `skills/offload-audit/SKILL.md` step 3 and `agents/offload-analyst.md` §2 so future audit runs call the script instead of reshaping by hand. Version bumped 0.7.0 → 0.8.0 (new script + agent/skill instruction changes). ## Test plan - [x] `tests/candidate-digest.test.sh` (new, auto-discovered by `tests/run-all.sh`): the issue's three worked examples — scripted skill suppressed via `has_bin_script`, a flagged agent row (`judgment_density=0.034`), the EDGE case suppressed by the brake (`judgment_density=0.55`) — plus a below-`MIN_SHARE_OF_SPEND` case and a full `CandidateRow` schema check. All pass. - [x] Full suite (`tests/run-all.sh`) passes, exit 0, no regressions. - [x] Verified against a real matched `inv.json`/`scan.json` pair from a prior `anxious` audit (`/tmp/anxious-inventory.json` + `/tmp/anxious-scan.json`) — sensible, correctly-sorted output with flags firing only on rows that actually cross every gate. Fix is complete and self-verified; I believe it's ready to arm for merge-when-checks-succeed, but leaving that to whoever reviews.
oleks added 1 commit 2026-07-30 19:27:37 +03:00
Add bin/candidate-digest: script the inv.json/scan.json join
ci/woodpecker/push/test Pipeline was successful
Details
ci/woodpecker/pr/test Pipeline was successful
Details
c4389d8bb1 
The offload-analyst agent was hand-reshaping bin/plugin-inventory's
PluginInventory and bin/offload-scan's OffloadScan into a candidate
table on every audit run — 40.09% of this plugin's own audited spend
over 26 invocations (kotkan/claude-plugin-inference-arbitrage#24).

candidate_digest(inv, scan) -> list[CandidateRow] joins the two JSON
documents and computes shape: share_of_spend, mtr, judgment_density,
read_amplification, retry_density, fanout_multiplier,
dominant_ngram_recurrences, has_bin_script, and an advisory `flags`
list. It reuses boundary-classify's MIN_INVOCATIONS/
MIN_ATTRIBUTED_TURNS and ia_store's MIN_SHARE_OF_SPEND rather than
reinventing thresholds, and applies the judgment_density brake
(new JUDGMENT_DENSITY_BRAKE=0.3) so a row with real judgment density
never gets flagged even if every mechanical threshold is crossed.

This is a digest, not an auto-filing script (position 3, never
position 1): flags are advisory input to the analyst's own reading of
the actual n-gram/tool-call shape, never a verdict on their own.
bin/boundary-classify remains the only place a verdict is computed.

Wired into skills/offload-audit/SKILL.md step 3 and
agents/offload-analyst.md section 2 so future audit runs call the
script instead of reshaping by hand.

Tests: tests/candidate-digest.test.sh covers the issue's three worked
examples (scripted skill suppressed via has_bin_script, a flagged
agent row, and the judgment_density-brake-suppressed edge case) plus
a below-MIN_SHARE_OF_SPEND case and a full CandidateRow schema check.
Also verified against a real matched inv.json/scan.json pair from a
prior anxious audit in /tmp.
oleks scheduled this pull request to auto merge when all checks succeed 2026-07-30 19:30:36 +03:00
oleks added 1 commit 2026-07-30 19:56:21 +03:00
Merge remote-tracking branch 'origin/main' into fix/issue-24-candidate-digest
ci/woodpecker/push/test Pipeline was successful
Details
ci/woodpecker/pr/test Pipeline was successful
Details
230a44348c 
# Conflicts:
#	.claude-plugin/plugin.json
oleks merged commit 0482278270 into main 2026-07-30 19:57:32 +03:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
activity/automate
agent-wip
area/agent-behavior
Agent behavior and delegation logic
 bug
cost
domain/agents
Agent orchestration and multi-agent systems
 efficiency
infra-misconfig
kind/capability-gap
kind/defect
A defect or bug that needs fixing
 token-offload
A step paying for inference where a deterministic script would do (inference-arbitrage audit)
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
admin anton oleks
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: kotkan/claude-plugin-inference-arbitrage#28
Delete Branch "fix/issue-24-candidate-digest"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?