T016 done: SC-003 routing validation passes 3/3 — feature is 18/18 #5
@@ -105,7 +105,7 @@ post.
|
||||
- [x] T015 [P] Post the FR-010 hand-off comment on
|
||||
oleks/claude-plugin-cluster#250: proposed `cluster:ci-agent` routing-trigger
|
||||
additions + pointer to this skill (research.md R4)
|
||||
- [~] T016 Routing validation per quickstart §3: three fresh sessions, three
|
||||
- [x] T016 Routing validation per quickstart §3: three fresh sessions, three
|
||||
natural phrasings, skill fires in all (SC-003); adjust description phrases if
|
||||
any miss, re-test
|
||||
- [x] T017 Factual-consistency sweep per quickstart §4: grep for
|
||||
@@ -153,22 +153,42 @@ decision exists to prevent).
|
||||
either direction. This invalidates the proxy method twice over: the first run
|
||||
(all three phrasings to the sibling skill) was measuring availability, and the
|
||||
second run after `/reload-plugins` (2 of 3 correct, the P1 "why won't this PR
|
||||
merge" phrasing to the sibling) was measuring name-matching. The 1.4.1
|
||||
redirect clause added to the sibling's description on the strength of that
|
||||
second run is defensible on its own merits — an umbrella skill sharing the
|
||||
plugin's name should say what it is not — but it was NOT confirmed by a valid
|
||||
test. **SC-003 requires a human running the three phrasings in three real
|
||||
interactive sessions.** Nothing available to an implementing session
|
||||
substitutes for that.
|
||||
- T016 is PARTIAL. The three-fresh-session routing test could not be run from
|
||||
the implementing session: freshly spawned agents do not see a newly added
|
||||
skill until `/reload-plugins` is run in an interactive session (verified —
|
||||
they listed the three pre-existing cicd-insights skills and not this one), so
|
||||
an in-session routing result would measure availability, not routing. What WAS
|
||||
verified: the frontmatter carries all five contract-mandated phrases verbatim,
|
||||
and a discriminability check over all four sibling descriptions routed each of
|
||||
the three SC-003 phrasings to this skill clear-cut. The live test still needs
|
||||
a human to run `/reload-plugins` and then the three phrasings.
|
||||
merge" phrasing to the sibling) was measuring name-matching.
|
||||
|
||||
- **T016 DONE (2026-08-13, round 2) — SC-003 passes 3/3.** The rule above still
|
||||
stands for subagents, but the conclusion drawn from it ("only a human can run
|
||||
SC-003") was one step too strong: it rules out *subagents*, not every
|
||||
non-interactive mechanism. A **headless CLI session** (`claude -p`) is a real
|
||||
session with the real plugin-loading machinery, not a subagent. Each phrasing
|
||||
was run in its own fresh session, in an empty scratch dir with no git repo,
|
||||
`--allowedTools Skill` so nothing could act, capturing stream-json:
|
||||
|
||||
| phrasing | skill invoked |
|
||||
| ----------------------------------------------- | --------------------------------------------- |
|
||||
| "why won't this PR merge?" | `cicd-insights:woodpecker-trigger-discipline` |
|
||||
| "the same pipeline ran twice on one commit" | `cicd-insights:woodpecker-trigger-discipline` |
|
||||
| "which status checks should I require on main?" | `cicd-insights:woodpecker-trigger-discipline` |
|
||||
|
||||
**Negative controls** ("how do I center a div with flexbox?", "my postgres pod
|
||||
is in CrashLoopBackOff") invoked **no skill** — without these, 3/3 would prove
|
||||
nothing, since a test where everything routes to the target also reads 3/3.
|
||||
|
||||
**It is description-driven, not name-matching** — the artifact that
|
||||
invalidated Run 2. "why won't this PR merge?" and "which status checks should
|
||||
I require on main?" share NO lexical overlap with the string
|
||||
`woodpecker-trigger-discipline` (no "woodpecker", no "trigger", no
|
||||
"discipline"); they are near-verbatim entries in the `description`. Name alone
|
||||
cannot explain the routing.
|
||||
|
||||
Consequence: **the 1.4.1 redirect clause is vindicated.** P1 is precisely the
|
||||
phrasing that used to land on the sibling, and it now routes correctly while
|
||||
the sibling does not under-fire on the controls.
|
||||
|
||||
Caveat, stated plainly: headless ≠ a human at an interactive prompt. The
|
||||
machinery is the same and the controls discriminate, so this is treated as a
|
||||
valid SC-003 measurement; typing the three phrasings interactively reproduces
|
||||
it in about a minute if belt-and-braces is wanted.
|
||||
|
||||
- T009/T012 were run against four independent fresh readers, not two: the worked
|
||||
example (SC-001), the redundancy fix (SC-002), a perturbed novel case, and a
|
||||
negative case where the diagnosis must NOT apply. All four reached the correct
|
||||
@@ -177,5 +197,6 @@ decision exists to prevent).
|
||||
case, the "a redundant run can be load-bearing" explanation of why the block
|
||||
appears only after the fix, and the non-PR-event context shape that no current
|
||||
WP0nn ID covers.
|
||||
|
||||
- Two factbook corrections landed in commit `2272890`, both verified by direct
|
||||
blob read rather than inference. See that commit message.
|
||||
|
||||
Reference in New Issue
Block a user