series --list claims to enumerate every extraction but covers 1 of 16 hooks-to-go checks #7

New Issue

2026-09-12T15:29:17+03:00

issuer-agent commented

2026-09-12 15:29:17 +03:00

What

just extractions / series.xsh --list is documented as:

Enumerate EVERY extraction this workspace can perform, and say which ones just playbook does NOT cover.

It reports 21 extractions. hooks-to-go/bin ships 16 executable check scripts. The series lists exactly one of them (h2g-phase-gate, as MANUAL) and names four siblings in a prose note.

Why it matters

The enumerator is the tool you use to answer "what did we not measure". An undercount there is load-bearing in a way an undercount elsewhere is not: it produces a confident, specific, wrong answer to exactly the question it exists to answer.

Evidence (2026-09-12)

Nine of the sixteen run with no caller-specific arguments. Run by hand against --workspace ~/projects/claude-plugins they produced findings no playbook run has ever reported:

check	result
`census-dispatcher-drift`	0 drifting, 15 clean of 15 consumers
`check-retention-register`	8 findings — 4 unregistered_retention, 4 inventory_retains_missing_file
`check-retirement-gates`	0 admitted, 24 refused, 24 evaluated
`check-release-descendant`	clean, 85 published tags
`check-structural-parse`	clean

Three more (check-non-vacuity, run-parity-check, run-witness-check) build Go fixtures and were deferred. Six genuinely need caller-specific arguments (a target repo path and script path), which is a legitimate reason for the series to decline to guess them — but they should be enumerated as manual, as h2g-phase-gate is, rather than omitted.

One classification note for whoever fixes this: check-release-version returns rc=1 outside a release attempt, because it is a push gate asking "would this duplicate a published version", not a measurement. It should be enumerated as a gate, not as an extraction.

## What `just extractions` / `series.xsh --list` is documented as: > Enumerate EVERY extraction this workspace can perform, and say which ones `just playbook` does NOT cover. It reports 21 extractions. `hooks-to-go/bin` ships **16 executable check scripts**. The series lists exactly one of them (`h2g-phase-gate`, as MANUAL) and names four siblings in a prose note. ## Why it matters The enumerator is the tool you use to answer "what did we not measure". An undercount there is load-bearing in a way an undercount elsewhere is not: it produces a confident, specific, wrong answer to exactly the question it exists to answer. ## Evidence (2026-09-12) Nine of the sixteen run with no caller-specific arguments. Run by hand against `--workspace ~/projects/claude-plugins` they produced findings no playbook run has ever reported: | check | result | | --- | --- | | `census-dispatcher-drift` | 0 drifting, 15 clean of 15 consumers | | `check-retention-register` | **8 findings** — 4 unregistered_retention, 4 inventory_retains_missing_file | | `check-retirement-gates` | 0 admitted, 24 refused, 24 evaluated | | `check-release-descendant` | clean, 85 published tags | | `check-structural-parse` | clean | Three more (`check-non-vacuity`, `run-parity-check`, `run-witness-check`) build Go fixtures and were deferred. Six genuinely need caller-specific arguments (a target repo path and script path), which is a legitimate reason for the series to decline to guess them — but they should be *enumerated as manual*, as `h2g-phase-gate` is, rather than omitted. One classification note for whoever fixes this: `check-release-version` returns rc=1 outside a release attempt, because it is a push gate asking "would this duplicate a published version", not a measurement. It should be enumerated as a gate, not as an extraction. ## Related Distinct from the closed #2, which was about the playbook omitting drill-down tools. This is about the enumerator's own completeness claim.

oleks commented

2026-09-12 16:27:17 +03:00

Correction to this issue's own list

I claimed nine of the sixteen "run with no caller-specific arguments" and named run-parity-check and run-witness-check among them. Two of those three are wrong. I inferred runnability from a crude count of argv references in each script rather than from its usage, which is the same shortcut this repo keeps getting bitten by.

Ran them on carry, 2026-09-12, with --workspace /home/oleks/projects/claude-plugins:

check	rc	result
`check-non-vacuity`	1	real report, 1588 lines — `54 detects, 0 vacuous, 0 inconclusive, 1 unverifiable, 1 retired, 0 error`
`run-witness-check`	2	usage error — needs `--original <path> --fixtures <dir> --inventory <artifact> --plugin <name> --out <report.json>`
`run-parity-check`	2	usage error — needs `<binary> <fixtures-dir> <report-output-path> <inventory-artifact>`

So the corrected split is:

Fleet-runnable (produce a workspace-wide verdict from --workspace alone, or no args): census-dispatcher-drift, check-non-vacuity, check-release-descendant, check-retention-register, check-retirement-gates, check-structural-parse. Six. These are the ones the series should enumerate and could plausibly run.

Genuinely caller-specific (need a target repo, script path, binary, fixtures dir, or output path — the series is right to decline to guess): check-ci-trigger, check-hook-wiring, check-inventory-disposition, check-speckit-init, phase-gate, scan-secondary-consumers, run-parity-check, run-witness-check. Eight, and they should be enumerated as MANUAL the way h2g-phase-gate already is, rather than omitted.

Neither: install-git-hooks is an action, not a measurement. check-release-version is a push gate — it returns rc=1 outside a release attempt because it answers "would this duplicate a published version", so enumerating it as an extraction would manufacture a permanent false failure.

check-non-vacuity's result is worth keeping as a positive control: 0 vacuous across 54 detecting suites. Its own closing note is the right caveat to carry — "detecting an inert binary is NECESSARY, not sufficient — it proves a suite is not vacuous, not that it is correct." It also surfaced one unverifiable case (worktree-heartbeat, where every fixture reports identically against a subject that cannot execute, already tracked as hooks-to-go#39) and two worktree-remove fixtures whose expected log files were absent.

The headline of this issue stands: the enumerator claims completeness and covers one of sixteen. Only my breakdown of which were runnable needed fixing.

## Correction to this issue's own list I claimed nine of the sixteen "run with no caller-specific arguments" and named `run-parity-check` and `run-witness-check` among them. **Two of those three are wrong.** I inferred runnability from a crude count of argv references in each script rather than from its usage, which is the same shortcut this repo keeps getting bitten by. Ran them on carry, 2026-09-12, with `--workspace /home/oleks/projects/claude-plugins`: | check | rc | result | | --- | --- | --- | | `check-non-vacuity` | 1 | **real report, 1588 lines** — `54 detects, 0 vacuous, 0 inconclusive, 1 unverifiable, 1 retired, 0 error` | | `run-witness-check` | 2 | usage error — needs `--original <path> --fixtures <dir> --inventory <artifact> --plugin <name> --out <report.json>` | | `run-parity-check` | 2 | usage error — needs `<binary> <fixtures-dir> <report-output-path> <inventory-artifact>` | So the corrected split is: **Fleet-runnable** (produce a workspace-wide verdict from `--workspace` alone, or no args): `census-dispatcher-drift`, `check-non-vacuity`, `check-release-descendant`, `check-retention-register`, `check-retirement-gates`, `check-structural-parse`. Six. These are the ones the series should enumerate and could plausibly run. **Genuinely caller-specific** (need a target repo, script path, binary, fixtures dir, or output path — the series is right to decline to guess): `check-ci-trigger`, `check-hook-wiring`, `check-inventory-disposition`, `check-speckit-init`, `phase-gate`, `scan-secondary-consumers`, `run-parity-check`, `run-witness-check`. Eight, and they should be *enumerated as MANUAL* the way `h2g-phase-gate` already is, rather than omitted. **Neither**: `install-git-hooks` is an action, not a measurement. `check-release-version` is a push gate — it returns rc=1 outside a release attempt because it answers "would this duplicate a published version", so enumerating it as an extraction would manufacture a permanent false failure. `check-non-vacuity`'s result is worth keeping as a positive control: 0 vacuous across 54 detecting suites. Its own closing note is the right caveat to carry — "detecting an inert binary is NECESSARY, not sufficient — it proves a suite is not vacuous, not that it is correct." It also surfaced one `unverifiable` case (`worktree-heartbeat`, where every fixture reports identically against a subject that cannot execute, already tracked as hooks-to-go#39) and two `worktree-remove` fixtures whose expected log files were absent. The headline of this issue stands: the enumerator claims completeness and covers one of sixteen. Only my breakdown of *which* were runnable needed fixing.

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: oleks/analytics-playbook#7