T016 done: SC-003 routing validation passes 3/3 — feature is 18/18 #5
@@ -105,7 +105,7 @@ post.
|
|||||||
- [x] T015 [P] Post the FR-010 hand-off comment on
|
- [x] T015 [P] Post the FR-010 hand-off comment on
|
||||||
oleks/claude-plugin-cluster#250: proposed `cluster:ci-agent` routing-trigger
|
oleks/claude-plugin-cluster#250: proposed `cluster:ci-agent` routing-trigger
|
||||||
additions + pointer to this skill (research.md R4)
|
additions + pointer to this skill (research.md R4)
|
||||||
- [~] T016 Routing validation per quickstart §3: three fresh sessions, three
|
- [x] T016 Routing validation per quickstart §3: three fresh sessions, three
|
||||||
natural phrasings, skill fires in all (SC-003); adjust description phrases if
|
natural phrasings, skill fires in all (SC-003); adjust description phrases if
|
||||||
any miss, re-test
|
any miss, re-test
|
||||||
- [x] T017 Factual-consistency sweep per quickstart §4: grep for
|
- [x] T017 Factual-consistency sweep per quickstart §4: grep for
|
||||||
@@ -153,22 +153,42 @@ decision exists to prevent).
|
|||||||
either direction. This invalidates the proxy method twice over: the first run
|
either direction. This invalidates the proxy method twice over: the first run
|
||||||
(all three phrasings to the sibling skill) was measuring availability, and the
|
(all three phrasings to the sibling skill) was measuring availability, and the
|
||||||
second run after `/reload-plugins` (2 of 3 correct, the P1 "why won't this PR
|
second run after `/reload-plugins` (2 of 3 correct, the P1 "why won't this PR
|
||||||
merge" phrasing to the sibling) was measuring name-matching. The 1.4.1
|
merge" phrasing to the sibling) was measuring name-matching.
|
||||||
redirect clause added to the sibling's description on the strength of that
|
|
||||||
second run is defensible on its own merits — an umbrella skill sharing the
|
- **T016 DONE (2026-08-13, round 2) — SC-003 passes 3/3.** The rule above still
|
||||||
plugin's name should say what it is not — but it was NOT confirmed by a valid
|
stands for subagents, but the conclusion drawn from it ("only a human can run
|
||||||
test. **SC-003 requires a human running the three phrasings in three real
|
SC-003") was one step too strong: it rules out *subagents*, not every
|
||||||
interactive sessions.** Nothing available to an implementing session
|
non-interactive mechanism. A **headless CLI session** (`claude -p`) is a real
|
||||||
substitutes for that.
|
session with the real plugin-loading machinery, not a subagent. Each phrasing
|
||||||
- T016 is PARTIAL. The three-fresh-session routing test could not be run from
|
was run in its own fresh session, in an empty scratch dir with no git repo,
|
||||||
the implementing session: freshly spawned agents do not see a newly added
|
`--allowedTools Skill` so nothing could act, capturing stream-json:
|
||||||
skill until `/reload-plugins` is run in an interactive session (verified —
|
|
||||||
they listed the three pre-existing cicd-insights skills and not this one), so
|
| phrasing | skill invoked |
|
||||||
an in-session routing result would measure availability, not routing. What WAS
|
| ----------------------------------------------- | --------------------------------------------- |
|
||||||
verified: the frontmatter carries all five contract-mandated phrases verbatim,
|
| "why won't this PR merge?" | `cicd-insights:woodpecker-trigger-discipline` |
|
||||||
and a discriminability check over all four sibling descriptions routed each of
|
| "the same pipeline ran twice on one commit" | `cicd-insights:woodpecker-trigger-discipline` |
|
||||||
the three SC-003 phrasings to this skill clear-cut. The live test still needs
|
| "which status checks should I require on main?" | `cicd-insights:woodpecker-trigger-discipline` |
|
||||||
a human to run `/reload-plugins` and then the three phrasings.
|
|
||||||
|
**Negative controls** ("how do I center a div with flexbox?", "my postgres pod
|
||||||
|
is in CrashLoopBackOff") invoked **no skill** — without these, 3/3 would prove
|
||||||
|
nothing, since a test where everything routes to the target also reads 3/3.
|
||||||
|
|
||||||
|
**It is description-driven, not name-matching** — the artifact that
|
||||||
|
invalidated Run 2. "why won't this PR merge?" and "which status checks should
|
||||||
|
I require on main?" share NO lexical overlap with the string
|
||||||
|
`woodpecker-trigger-discipline` (no "woodpecker", no "trigger", no
|
||||||
|
"discipline"); they are near-verbatim entries in the `description`. Name alone
|
||||||
|
cannot explain the routing.
|
||||||
|
|
||||||
|
Consequence: **the 1.4.1 redirect clause is vindicated.** P1 is precisely the
|
||||||
|
phrasing that used to land on the sibling, and it now routes correctly while
|
||||||
|
the sibling does not under-fire on the controls.
|
||||||
|
|
||||||
|
Caveat, stated plainly: headless ≠ a human at an interactive prompt. The
|
||||||
|
machinery is the same and the controls discriminate, so this is treated as a
|
||||||
|
valid SC-003 measurement; typing the three phrasings interactively reproduces
|
||||||
|
it in about a minute if belt-and-braces is wanted.
|
||||||
|
|
||||||
- T009/T012 were run against four independent fresh readers, not two: the worked
|
- T009/T012 were run against four independent fresh readers, not two: the worked
|
||||||
example (SC-001), the redundancy fix (SC-002), a perturbed novel case, and a
|
example (SC-001), the redundancy fix (SC-002), a perturbed novel case, and a
|
||||||
negative case where the diagnosis must NOT apply. All four reached the correct
|
negative case where the diagnosis must NOT apply. All four reached the correct
|
||||||
@@ -177,5 +197,6 @@ decision exists to prevent).
|
|||||||
case, the "a redundant run can be load-bearing" explanation of why the block
|
case, the "a redundant run can be load-bearing" explanation of why the block
|
||||||
appears only after the fix, and the non-PR-event context shape that no current
|
appears only after the fix, and the non-PR-event context shape that no current
|
||||||
WP0nn ID covers.
|
WP0nn ID covers.
|
||||||
|
|
||||||
- Two factbook corrections landed in commit `2272890`, both verified by direct
|
- Two factbook corrections landed in commit `2272890`, both verified by direct
|
||||||
blob read rather than inference. See that commit message.
|
blob read rather than inference. See that commit message.
|
||||||
|
|||||||
Reference in New Issue
Block a user