Latent routing collision between cicd-insights and woodpecker-trigger-discipline skills #2

Closed
opened 2026-08-13 18:10:34 +03:00 by issuer-agent · 2 comments

Why: The cicd-insights skill's description includes the trigger phrase "trigger hygiene", while the newer woodpecker-trigger-discipline skill (v1.4.0) owns the trigger-narrowing/branch-protection domain outright. A fleet-wide "audit our trigger hygiene" request could misroute to the specific skill, or a single-repo trigger question to the broad one.

Not observed failing: a discriminability check on the three target phrasings routed to the correct skill clear-cut every time. Filing as a watch item, not a live defect.

Acceptance:

  • Review the two skills' descriptions for overlapping trigger phrases and disambiguate if a real miss is ever observed
  • No action required unless a misroute actually occurs — this is a watch item
**Why**: The `cicd-insights` skill's description includes the trigger phrase "trigger hygiene", while the newer `woodpecker-trigger-discipline` skill (v1.4.0) owns the trigger-narrowing/branch-protection domain outright. A fleet-wide "audit our trigger hygiene" request could misroute to the specific skill, or a single-repo trigger question to the broad one. Not observed failing: a discriminability check on the three target phrasings routed to the correct skill clear-cut every time. Filing as a watch item, not a live defect. **Acceptance**: - [ ] Review the two skills' descriptions for overlapping trigger phrases and disambiguate if a real miss is ever observed - [ ] No action required unless a misroute actually occurs — this is a watch item
Owner

Status: fix shipped, confirmation blocked on a test no automated session can run.

Shipped in v1.4.1 (f905f55): the cicd-insights skill description now states it is the broad structural-review skill despite sharing the plugin's name, is not a catch-all, and names the four scenarios belonging to woodpecker-trigger-discipline — a PR that won't merge, a pipeline running twice, a required check that never posts, choosing which checks to require.

Why it is not confirmed, and why no more automated attempts should be made. Routing was tested three times across two mechanisms, and every result was invalid:

  1. Before /reload-plugins, three fresh subagents routed all three phrasings to the sibling skill. That measured availability — an availability probe returned no woodpecker skill at all.
  2. After /reload-plugins, 2 of 3 routed correctly, with the P1 phrasing ("why won't this PR merge") going to the sibling. This is what prompted the 1.4.1 redirect.
  3. A follow-up probe then established that spawned subagents receive skill names but not their frontmatter descriptions. Routing is driven by the description, so run 2 measured name-matching, not routing — invalidating the failure and the two passes.

So the redirect clause was shipped on evidence that did not support it. It stands on its own merits (an umbrella skill sharing the plugin's name should say what it is not), but it is unverified, and it is possible nothing was broken.

What actually closes this — three real interactive sessions, one phrasing each, checking which skill loads:

  • "why won't this PR merge?" (CI green)
  • "the same pipeline ran twice on one commit"
  • "which status checks should I require on main?"

This is spec SC-003. What was validated: the descriptions are discriminable — presented verbatim to a fresh reader alongside all four sibling descriptions, all three phrasings were assigned to woodpecker-trigger-discipline clear-cut. That tests the text, not the router.

If all three load the right skill, close this. If any misses, the description is the lever and the phrase set is already exhaustive enough that the next step is narrowing the sibling, not widening this one.

**Status: fix shipped, confirmation blocked on a test no automated session can run.** Shipped in v1.4.1 (`f905f55`): the `cicd-insights` skill description now states it is the broad structural-review skill *despite sharing the plugin's name*, is not a catch-all, and names the four scenarios belonging to `woodpecker-trigger-discipline` — a PR that won't merge, a pipeline running twice, a required check that never posts, choosing which checks to require. **Why it is not confirmed, and why no more automated attempts should be made.** Routing was tested three times across two mechanisms, and every result was invalid: 1. Before `/reload-plugins`, three fresh subagents routed all three phrasings to the sibling skill. That measured *availability* — an availability probe returned no woodpecker skill at all. 2. After `/reload-plugins`, 2 of 3 routed correctly, with the P1 phrasing ("why won't this PR merge") going to the sibling. This is what prompted the 1.4.1 redirect. 3. A follow-up probe then established that **spawned subagents receive skill names but not their frontmatter descriptions.** Routing is driven by the description, so run 2 measured name-matching, not routing — invalidating the failure *and* the two passes. So the redirect clause was shipped on evidence that did not support it. It stands on its own merits (an umbrella skill sharing the plugin's name should say what it is not), but it is unverified, and it is possible nothing was broken. **What actually closes this** — three real interactive sessions, one phrasing each, checking which skill loads: - "why won't this PR merge?" (CI green) - "the same pipeline ran twice on one commit" - "which status checks should I require on main?" This is spec SC-003. What *was* validated: the descriptions are discriminable — presented verbatim to a fresh reader alongside all four sibling descriptions, all three phrasings were assigned to `woodpecker-trigger-discipline` clear-cut. That tests the text, not the router. If all three load the right skill, close this. If any misses, the description is the lever and the phrase set is already exhaustive enough that the next step is narrowing the *sibling*, not widening this one.
Owner

SC-003 PASSES — 3/3, with negative controls.

The mechanism (why the previous three attempts were invalid and this isn't)

The prior runs used subagents, whose skill listing carries names but not frontmatter descriptions — and routing is description-driven, so those runs measured availability and name-matching instead. The handoff concluded from this that only a human could run SC-003.

That conclusion was slightly too strong: it rules out subagents, not every non-interactive mechanism. A headless CLI session (claude -p) is a real session with the real plugin-loading machinery, not a subagent. Each phrasing was run in its own fresh session, in an empty scratch directory with no git repo, with --allowedTools Skill so nothing could act, capturing --output-format stream-json.

Results

# phrasing skill invoked
P1 "why won't this PR merge?" cicd-insights:woodpecker-trigger-discipline
P2 "the same pipeline ran twice on one commit" cicd-insights:woodpecker-trigger-discipline
P3 "which status checks should I require on main?" cicd-insights:woodpecker-trigger-discipline

Negative controls — the part that makes the above mean something:

control skill invoked
"how do I center a div with flexbox?" (none)
"my postgres pod is in CrashLoopBackOff" (none)

A test where everything routes to the target skill would show 3/3 and prove nothing. These don't.

Why this is description-driven, not name-matching

This is the specific artifact that invalidated Run 2, so it's worth ruling out explicitly: "why won't this PR merge?" and "which status checks should I require on main?" share no lexical overlap with the string woodpecker-trigger-discipline. Neither contains "woodpecker", "trigger", or "discipline". They are near-verbatim entries in the skill's description. So routing cannot be explained by the name alone — the description is doing the work, which is exactly what SC-003 set out to confirm.

Bonus: the v1.4.1 redirect clause is vindicated

P1 is the phrasing that previously routed to the sibling cicd-insights skill and prompted the redirect clause. It now routes correctly. The handoff flagged that clause as "shipped on invalid evidence, riskiest change to sanity-check" — it holds up, and the sibling did not under-fire.

Caveat, stated plainly

This is a headless session, not a human at an interactive prompt. The skill-loading machinery is the same and the negative controls discriminate, so I consider it a valid measurement of SC-003 — but if you want the belt-and-braces version, typing the three phrasings interactively should reproduce it in about a minute.

Verified against cicd-insights 1.4.6, whose installed SKILL.md at ~/.claude/plugins/cache/oleks-local/cicd-insights/1.4.6/ was confirmed byte-identical to the repo with intact frontmatter (i.e. not affected by the oleks/claude-plugin-cicd-insights#1 corruption).

**SC-003 PASSES — 3/3, with negative controls.** ## The mechanism (why the previous three attempts were invalid and this isn't) The prior runs used **subagents**, whose skill listing carries names but not frontmatter descriptions — and routing is description-driven, so those runs measured availability and name-matching instead. The handoff concluded from this that only a human could run SC-003. That conclusion was slightly too strong: it rules out *subagents*, not *every non-interactive mechanism*. A **headless CLI session** (`claude -p`) is a real session with the real plugin-loading machinery, not a subagent. Each phrasing was run in its own fresh session, in an empty scratch directory with no git repo, with `--allowedTools Skill` so nothing could act, capturing `--output-format stream-json`. ## Results | # | phrasing | skill invoked | | --- | --- | --- | | P1 | "why won't this PR merge?" | `cicd-insights:woodpecker-trigger-discipline` ✅ | | P2 | "the same pipeline ran twice on one commit" | `cicd-insights:woodpecker-trigger-discipline` ✅ | | P3 | "which status checks should I require on main?" | `cicd-insights:woodpecker-trigger-discipline` ✅ | **Negative controls** — the part that makes the above mean something: | control | skill invoked | | --- | --- | | "how do I center a div with flexbox?" | *(none)* | | "my postgres pod is in CrashLoopBackOff" | *(none)* | A test where everything routes to the target skill would show 3/3 and prove nothing. These don't. ## Why this is description-driven, not name-matching This is the specific artifact that invalidated Run 2, so it's worth ruling out explicitly: **"why won't this PR merge?" and "which status checks should I require on main?" share no lexical overlap with the string `woodpecker-trigger-discipline`.** Neither contains "woodpecker", "trigger", or "discipline". They *are* near-verbatim entries in the skill's `description`. So routing cannot be explained by the name alone — the description is doing the work, which is exactly what SC-003 set out to confirm. ## Bonus: the v1.4.1 redirect clause is vindicated P1 is the phrasing that previously routed to the sibling `cicd-insights` skill and prompted the redirect clause. It now routes correctly. The handoff flagged that clause as "shipped on invalid evidence, riskiest change to sanity-check" — it holds up, and the sibling did not under-fire. ## Caveat, stated plainly This is a headless session, not a human at an interactive prompt. The skill-loading machinery is the same and the negative controls discriminate, so I consider it a valid measurement of SC-003 — but if you want the belt-and-braces version, typing the three phrasings interactively should reproduce it in about a minute. Verified against `cicd-insights` **1.4.6**, whose installed `SKILL.md` at `~/.claude/plugins/cache/oleks-local/cicd-insights/1.4.6/` was confirmed byte-identical to the repo with intact frontmatter (i.e. not affected by the oleks/claude-plugin-cicd-insights#1 corruption).
oleks added the agent/claim:oleks/4bdde801agent/wip labels 2026-08-13 22:38:32 +03:00
oleks closed this issue 2026-08-13 22:47:13 +03:00
oleks removed the agent/wipagent/claim:oleks/4bdde801 labels 2026-08-13 22:47:29 +03:00
Sign in to join this conversation.
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: oleks/claude-plugin-cicd-insights#2