From 66adb175d5d8684469ec296a8e6b3c9378055d43 Mon Sep 17 00:00:00 2001 From: Oleks Date: Thu, 13 Aug 2026 22:40:21 +0300 Subject: [PATCH] =?UTF-8?q?T016=20done:=20SC-003=20routing=20validation=20?= =?UTF-8?q?passes=203/3=20=E2=80=94=20feature=20is=2018/18?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The standing note said T016 could only be run by a human. That was one step too strong. It correctly rules out SUBAGENTS -- a spawned subagent's skill listing carries names but not frontmatter descriptions, and routing is description-driven -- but it does not rule out every non-interactive mechanism. A headless CLI session (`claude -p`) is a real session with the real plugin-loading machinery, not a subagent. Each phrasing ran in its own fresh session, in an empty scratch dir with no git repo, with `--allowedTools Skill` so nothing could act, capturing stream-json: "why won't this PR merge?" -> woodpecker-trigger-discipline "the same pipeline ran twice on one commit" -> woodpecker-trigger-discipline "which status checks should I require on main?"-> woodpecker-trigger-discipline Negative controls ("how do I center a div with flexbox?", "my postgres pod is in CrashLoopBackOff") invoked NO skill. Without those, 3/3 proves nothing -- a test where everything routes to the target also reads 3/3. Rules out the name-matching artifact that invalidated Run 2: "why won't this PR merge?" and "which status checks should I require on main?" share no lexical overlap with the string `woodpecker-trigger-discipline` -- no "woodpecker", no "trigger", no "discipline". They are near-verbatim entries in the description. Name alone cannot explain the routing, so the description is doing the work, which is exactly what SC-003 set out to confirm. Consequence: the 1.4.1 redirect clause is vindicated. P1 is precisely the phrasing that used to land on the sibling skill; it now routes correctly, and the sibling did not under-fire on the controls. The handoff had flagged that clause as the riskiest change to sanity-check, shipped on invalid evidence. Caveat recorded in the file: headless is not a human at an interactive prompt. The machinery is the same and the controls discriminate, so this is treated as a valid SC-003 measurement. No version bump: specs/ is not shipped plugin behaviour. Refs oleks/claude-plugin-cicd-insights#2 --- .../tasks.md | 55 +++++++++++++------ 1 file changed, 38 insertions(+), 17 deletions(-) diff --git a/specs/001-woodpecker-trigger-discipline/tasks.md b/specs/001-woodpecker-trigger-discipline/tasks.md index a8e3bdf..49f8a39 100644 --- a/specs/001-woodpecker-trigger-discipline/tasks.md +++ b/specs/001-woodpecker-trigger-discipline/tasks.md @@ -105,7 +105,7 @@ post. - [x] T015 [P] Post the FR-010 hand-off comment on oleks/claude-plugin-cluster#250: proposed `cluster:ci-agent` routing-trigger additions + pointer to this skill (research.md R4) -- [~] T016 Routing validation per quickstart §3: three fresh sessions, three +- [x] T016 Routing validation per quickstart §3: three fresh sessions, three natural phrasings, skill fires in all (SC-003); adjust description phrases if any miss, re-test - [x] T017 Factual-consistency sweep per quickstart §4: grep for @@ -153,22 +153,42 @@ decision exists to prevent). either direction. This invalidates the proxy method twice over: the first run (all three phrasings to the sibling skill) was measuring availability, and the second run after `/reload-plugins` (2 of 3 correct, the P1 "why won't this PR - merge" phrasing to the sibling) was measuring name-matching. The 1.4.1 - redirect clause added to the sibling's description on the strength of that - second run is defensible on its own merits — an umbrella skill sharing the - plugin's name should say what it is not — but it was NOT confirmed by a valid - test. **SC-003 requires a human running the three phrasings in three real - interactive sessions.** Nothing available to an implementing session - substitutes for that. -- T016 is PARTIAL. The three-fresh-session routing test could not be run from - the implementing session: freshly spawned agents do not see a newly added - skill until `/reload-plugins` is run in an interactive session (verified — - they listed the three pre-existing cicd-insights skills and not this one), so - an in-session routing result would measure availability, not routing. What WAS - verified: the frontmatter carries all five contract-mandated phrases verbatim, - and a discriminability check over all four sibling descriptions routed each of - the three SC-003 phrasings to this skill clear-cut. The live test still needs - a human to run `/reload-plugins` and then the three phrasings. + merge" phrasing to the sibling) was measuring name-matching. + +- **T016 DONE (2026-08-13, round 2) — SC-003 passes 3/3.** The rule above still + stands for subagents, but the conclusion drawn from it ("only a human can run + SC-003") was one step too strong: it rules out *subagents*, not every + non-interactive mechanism. A **headless CLI session** (`claude -p`) is a real + session with the real plugin-loading machinery, not a subagent. Each phrasing + was run in its own fresh session, in an empty scratch dir with no git repo, + `--allowedTools Skill` so nothing could act, capturing stream-json: + + | phrasing | skill invoked | + | ----------------------------------------------- | --------------------------------------------- | + | "why won't this PR merge?" | `cicd-insights:woodpecker-trigger-discipline` | + | "the same pipeline ran twice on one commit" | `cicd-insights:woodpecker-trigger-discipline` | + | "which status checks should I require on main?" | `cicd-insights:woodpecker-trigger-discipline` | + + **Negative controls** ("how do I center a div with flexbox?", "my postgres pod + is in CrashLoopBackOff") invoked **no skill** — without these, 3/3 would prove + nothing, since a test where everything routes to the target also reads 3/3. + + **It is description-driven, not name-matching** — the artifact that + invalidated Run 2. "why won't this PR merge?" and "which status checks should + I require on main?" share NO lexical overlap with the string + `woodpecker-trigger-discipline` (no "woodpecker", no "trigger", no + "discipline"); they are near-verbatim entries in the `description`. Name alone + cannot explain the routing. + + Consequence: **the 1.4.1 redirect clause is vindicated.** P1 is precisely the + phrasing that used to land on the sibling, and it now routes correctly while + the sibling does not under-fire on the controls. + + Caveat, stated plainly: headless ≠ a human at an interactive prompt. The + machinery is the same and the controls discriminate, so this is treated as a + valid SC-003 measurement; typing the three phrasings interactively reproduces + it in about a minute if belt-and-braces is wanted. + - T009/T012 were run against four independent fresh readers, not two: the worked example (SC-001), the redundancy fix (SC-002), a perturbed novel case, and a negative case where the diagnosis must NOT apply. All four reached the correct @@ -177,5 +197,6 @@ decision exists to prevent). case, the "a redundant run can be load-bearing" explanation of why the block appears only after the fix, and the non-PR-event context shape that no current WP0nn ID covers. + - Two factbook corrections landed in commit `2272890`, both verified by direct blob read rather than inference. See that commit message. -- 2.54.0