T016 done: SC-003 routing validation passes 3/3 — feature is 18/18 #5

Merged
oleks merged 1 commits from worktree-t016-complete into main 2026-08-13 22:41:00 +03:00
@@ -105,7 +105,7 @@ post.
- [x] T015 [P] Post the FR-010 hand-off comment on
oleks/claude-plugin-cluster#250: proposed `cluster:ci-agent` routing-trigger
additions + pointer to this skill (research.md R4)
- [~] T016 Routing validation per quickstart §3: three fresh sessions, three
- [x] T016 Routing validation per quickstart §3: three fresh sessions, three
natural phrasings, skill fires in all (SC-003); adjust description phrases if
any miss, re-test
- [x] T017 Factual-consistency sweep per quickstart §4: grep for
@@ -153,22 +153,42 @@ decision exists to prevent).
either direction. This invalidates the proxy method twice over: the first run
(all three phrasings to the sibling skill) was measuring availability, and the
second run after `/reload-plugins` (2 of 3 correct, the P1 "why won't this PR
merge" phrasing to the sibling) was measuring name-matching. The 1.4.1
redirect clause added to the sibling's description on the strength of that
second run is defensible on its own merits — an umbrella skill sharing the
plugin's name should say what it is not — but it was NOT confirmed by a valid
test. **SC-003 requires a human running the three phrasings in three real
interactive sessions.** Nothing available to an implementing session
substitutes for that.
- T016 is PARTIAL. The three-fresh-session routing test could not be run from
the implementing session: freshly spawned agents do not see a newly added
skill until `/reload-plugins` is run in an interactive session (verified —
they listed the three pre-existing cicd-insights skills and not this one), so
an in-session routing result would measure availability, not routing. What WAS
verified: the frontmatter carries all five contract-mandated phrases verbatim,
and a discriminability check over all four sibling descriptions routed each of
the three SC-003 phrasings to this skill clear-cut. The live test still needs
a human to run `/reload-plugins` and then the three phrasings.
merge" phrasing to the sibling) was measuring name-matching.
- **T016 DONE (2026-08-13, round 2) — SC-003 passes 3/3.** The rule above still
stands for subagents, but the conclusion drawn from it ("only a human can run
SC-003") was one step too strong: it rules out *subagents*, not every
non-interactive mechanism. A **headless CLI session** (`claude -p`) is a real
session with the real plugin-loading machinery, not a subagent. Each phrasing
was run in its own fresh session, in an empty scratch dir with no git repo,
`--allowedTools Skill` so nothing could act, capturing stream-json:
| phrasing | skill invoked |
| ----------------------------------------------- | --------------------------------------------- |
| "why won't this PR merge?" | `cicd-insights:woodpecker-trigger-discipline` |
| "the same pipeline ran twice on one commit" | `cicd-insights:woodpecker-trigger-discipline` |
| "which status checks should I require on main?" | `cicd-insights:woodpecker-trigger-discipline` |
**Negative controls** ("how do I center a div with flexbox?", "my postgres pod
is in CrashLoopBackOff") invoked **no skill** — without these, 3/3 would prove
nothing, since a test where everything routes to the target also reads 3/3.
**It is description-driven, not name-matching** — the artifact that
invalidated Run 2. "why won't this PR merge?" and "which status checks should
I require on main?" share NO lexical overlap with the string
`woodpecker-trigger-discipline` (no "woodpecker", no "trigger", no
"discipline"); they are near-verbatim entries in the `description`. Name alone
cannot explain the routing.
Consequence: **the 1.4.1 redirect clause is vindicated.** P1 is precisely the
phrasing that used to land on the sibling, and it now routes correctly while
the sibling does not under-fire on the controls.
Caveat, stated plainly: headless ≠ a human at an interactive prompt. The
machinery is the same and the controls discriminate, so this is treated as a
valid SC-003 measurement; typing the three phrasings interactively reproduces
it in about a minute if belt-and-braces is wanted.
- T009/T012 were run against four independent fresh readers, not two: the worked
example (SC-001), the redundancy fix (SC-002), a perturbed novel case, and a
negative case where the diagnosis must NOT apply. All four reached the correct
@@ -177,5 +197,6 @@ decision exists to prevent).
case, the "a redundant run can be load-bearing" explanation of why the block
appears only after the fix, and the non-PR-event context shape that no current
WP0nn ID covers.
- Two factbook corrections landed in commit `2272890`, both verified by direct
blob read rather than inference. See that commit message.