kotkan/claude-plugin-inference-arbitrage
3
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Make the wiki history check mandatory before stability-classify #19

Merged
oleks merged 2 commits from fix-17-stability-gate-wiki-check into main 2026-07-30 05:41:34 +03:00
Conversation 0 Commits 2 Files Changed 4
+129 -13
oleks commented 2026-07-30 05:41:27 +03:00
Owner
Copy Link
Copy Source

Fixes kotkan/claude-plugin-inference-arbitrage#17.

Root cause

bin/stability-classify's "unchecked — no prior window to compare" state is
correct FR-2.4 behavior for a genuine first audit, but the script only ever
reads the local filesystem snapshot store (never the wiki, by design — no
bin/ script holds wiki credentials). skills/offload-audit/SKILL.md step
4b documented seeding the local store from the wiki
(Data/<target>/snapshots.jsonl) as a conditional, manual aside ("if the
store is cold... seed it first") rather than an enforced step. Because the
gate's output is identical whether history is genuinely absent everywhere or
merely absent from a cold local cache, a cold-but-not-actually-empty store
silently degrades to "unchecked" and files unstable candidates — exactly as
observed on the 2026-07-30 anxious run.

Fix

  1. skills/offload-audit/SKILL.md step 4b: replaced the conditional
    "if cold, seed it" aside with an unconditional, mandatory pre-step. Every
    audit run now wiki_reads (via cluster:gitea-agent)
    Data/<target>/snapshots.jsonl and, if it exists, runs
    bin/audit-snapshot import --target <name> --snapshots <fetched file>
    before invoking stability-classify. import dedupes by run_id, so
    this is safe/idempotent to run unconditionally.
  2. bin/stability-classify now has a required --wiki-checked {empty,imported} flag: it hard-errors (rather than silently emitting
    "unchecked") when the local store is empty for the target and the flag is
    omitted. This turns the safeguard mechanical instead of doc-only.
  3. tests/stability.test.sh regression cases (g)/(h)/(i): (g) no flag on an
    empty store → error, (h) --wiki-checked=empty → unchecked exactly as
    before, (i) --wiki-checked=imported after a real audit-snapshot import
    → gate runs normally against the imported history.
  4. plugin.json bumped 0.6.2 → 0.6.3.

Test plan

Full suite run locally, every file passing:

$ for f in tests/*.test.sh; do bash "$f"; done

All of classify.test.sh, filing.test.sh, inventory.test.sh,
memory.test.sh, scan.test.sh, snapshot.test.sh, stability.test.sh
pass, including the three new regression cases (g)/(h)/(i) in
stability.test.sh.

Fixes kotkan/claude-plugin-inference-arbitrage#17. ## Root cause `bin/stability-classify`'s "unchecked — no prior window to compare" state is correct FR-2.4 behavior for a genuine first audit, but the script only ever reads the local filesystem snapshot store (never the wiki, by design — no `bin/` script holds wiki credentials). `skills/offload-audit/SKILL.md` step 4b documented seeding the local store from the wiki (`Data/<target>/snapshots.jsonl`) as a conditional, manual aside ("if the store is cold... seed it first") rather than an enforced step. Because the gate's output is identical whether history is genuinely absent everywhere or merely absent from a cold local cache, a cold-but-not-actually-empty store silently degrades to "unchecked" and files unstable candidates — exactly as observed on the 2026-07-30 anxious run. ## Fix 1. `skills/offload-audit/SKILL.md` step 4b: replaced the conditional "if cold, seed it" aside with an unconditional, mandatory pre-step. Every audit run now `wiki_read`s (via `cluster:gitea-agent`) `Data/<target>/snapshots.jsonl` and, if it exists, runs `bin/audit-snapshot import --target <name> --snapshots <fetched file>` before invoking `stability-classify`. `import` dedupes by `run_id`, so this is safe/idempotent to run unconditionally. 2. `bin/stability-classify` now has a required `--wiki-checked {empty,imported}` flag: it hard-errors (rather than silently emitting "unchecked") when the local store is empty for the target and the flag is omitted. This turns the safeguard mechanical instead of doc-only. 3. `tests/stability.test.sh` regression cases (g)/(h)/(i): (g) no flag on an empty store → error, (h) `--wiki-checked=empty` → unchecked exactly as before, (i) `--wiki-checked=imported` after a real `audit-snapshot import` → gate runs normally against the imported history. 4. `plugin.json` bumped 0.6.2 → 0.6.3. ## Test plan Full suite run locally, every file passing: ``` $ for f in tests/*.test.sh; do bash "$f"; done ``` All of `classify.test.sh`, `filing.test.sh`, `inventory.test.sh`, `memory.test.sh`, `scan.test.sh`, `snapshot.test.sh`, `stability.test.sh` pass, including the three new regression cases (g)/(h)/(i) in `stability.test.sh`.
oleks added 2 commits 2026-07-30 05:41:27 +03:00
Make the wiki history check mandatory before stability-classify (fix kotkan/claude-plugin-inference-arbitrage#17) 69ff443bd2 
bin/stability-classify only ever reads the local filesystem snapshot store
(by design, no bin/ script holds wiki credentials), but SKILL.md step 4b
documented seeding it from the wiki as a conditional "if the store is cold"
aside. That made a cold-but-not-actually-empty local cache indistinguishable
from a genuine first audit: both silently emit stability: unchecked and file
unstable candidates, as observed on the 2026-07-30 anxious run.

- bin/stability-classify: add a required --wiki-checked {empty,imported}
  flag; hard-error when the store is empty and the flag is omitted, instead
  of silently degrading to unchecked.
- tests/stability.test.sh: (g) no flag on an empty store errors, (h)
  --wiki-checked=empty behaves as before, (i) --wiki-checked=imported after a
  real `audit-snapshot import` runs the gate normally against the imported
  history.
- skills/offload-audit/SKILL.md: step 4b now fetches the wiki's
  Data/<target>/snapshots.jsonl and runs `audit-snapshot import`
  unconditionally (it dedupes by run_id, so this is safe every run) before
  invoking the gate.
- Bump plugin.json to 0.6.3.
Fix markdownlint line-length in the step 4b wiki-fetch snippet
ci/woodpecker/push/test Pipeline failed
Details
ci/woodpecker/pr/test Pipeline failed
Details
713469d0d9 
Follow-up to the kotkan/claude-plugin-inference-arbitrage#17 commit.
oleks scheduled this pull request to auto merge when all checks succeed 2026-07-30 05:41:32 +03:00
oleks merged commit 1a14456220 into main 2026-07-30 05:41:34 +03:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
activity/automate
agent-wip
area/agent-behavior
Agent behavior and delegation logic
 bug
cost
domain/agents
Agent orchestration and multi-agent systems
 efficiency
infra-misconfig
kind/capability-gap
kind/defect
A defect or bug that needs fixing
 token-offload
A step paying for inference where a deterministic script would do (inference-arbitrage audit)
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
admin anton oleks
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: kotkan/claude-plugin-inference-arbitrage#19
Delete Branch "fix-17-stability-gate-wiki-check"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?