oleks/pipetree
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

Detect two Woodpecker footguns: apostrophe-in-single-quote and Woodpecker-eats-${VAR} #11

New Issue
Closed
opened 2026-08-03 19:41:55 +03:00 by oleks · 1 comment
oleks commented 2026-08-03 19:41:55 +03:00
Owner
Copy Link
Copy Source

Two Woodpecker CI footguns that pipetree should detect statically, from the YAML alone (no live API needed). Both cost a real pipeline failure on oleks/mempalace on 2026-08-03.

Footgun 1 — an apostrophe inside a single-quoted shell block ends the quote early

The common idiom across these pipelines is a step command like:

- |
  nix shell nixpkgs#foo -c bash -euxc '
    # ... many lines, including comments ...
  '

The whole thing from the opening ' to the closing ' is a single bash argument. ANY apostrophe inside it — including inside a # comment — closes that quote early from the outer shell's point of view, silently truncating/corrupting everything after it.

Real incident: oleks/mempalace pipeline 218 (and originally pipeline 145) failed with a baffling patch: command not found (exit 127) because an explanatory comment contained the word "Woodpecker's" / "test_mcp_http_transport.py's" — an apostrophe nobody thought of as shell-significant. Root-caused and fixed in oleks/mempalace commit e409e70 ("remove stray apostrophe that broke the outer single-quoted script", oleks/mempalace#68). The failure mode is maximally confusing because the error names a tool (patch) that IS installed — it just isn't on the PATH of the shell the script fell back to after the quote broke.

Footgun 2 — Woodpecker substitutes ${VAR} itself, before the shell ever sees it

Woodpecker performs its own ${VAR} substitution pass over step commands at pipeline-parse time (its own metadata vars like ${CI_COMMIT_TAG}), BEFORE the shell runs. So any ${SHELL_VAR} meant for the shell — e.g. ${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH} — gets mangled by Woodpecker first (undefined var -> silently becomes empty/mangled text), commonly producing an "unbound variable" failure under set -u.

The fix/convention (see oleks/mempalace commit 33b880d, referenced in a comment in oleks/mempalace's .woodpecker/test.yaml) is to escape as $$ when the shell should own the expansion: $${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}. This also bites inside comments — a comment merely mentioning ${VAR} gets substituted too, since Woodpecker's substitution runs over the raw text.

Ask

Add static checks to pipetree (probably folded into its existing -analyze mode, alongside the existing shared-image/broken-depends_on checks) that scan each step's commands: text for both patterns and flag them, with care to avoid false positives:

  • legitimate '\'' escaping should not be flagged as footgun 1
  • correctly-escaped $$ for shell-owned expansion should not be flagged as footgun 2
  • Woodpecker's own legitimate ${CI_*} builtins should not be flagged as footgun 2

Both footguns are static and will recur across the fleet, so catching them at scan time (before a pipeline run burns 10+ minutes discovering them) is worth the false-positive-avoidance effort.

A PR implementing static detection for both will reference this issue.

Two Woodpecker CI footguns that pipetree should detect statically, from the YAML alone (no live API needed). Both cost a real pipeline failure on oleks/mempalace on 2026-08-03. ## Footgun 1 — an apostrophe inside a single-quoted shell block ends the quote early The common idiom across these pipelines is a step command like: ```yaml - | nix shell nixpkgs#foo -c bash -euxc ' # ... many lines, including comments ... ' ``` The whole thing from the opening `'` to the closing `'` is a single bash argument. ANY apostrophe inside it — including inside a `#` comment — closes that quote early from the outer shell's point of view, silently truncating/corrupting everything after it. Real incident: oleks/mempalace pipeline 218 (and originally pipeline 145) failed with a baffling `patch: command not found` (exit 127) because an explanatory comment contained the word "Woodpecker's" / "test_mcp_http_transport.py's" — an apostrophe nobody thought of as shell-significant. Root-caused and fixed in oleks/mempalace commit `e409e70` ("remove stray apostrophe that broke the outer single-quoted script", oleks/mempalace#68). The failure mode is maximally confusing because the error names a tool (`patch`) that IS installed — it just isn't on the PATH of the shell the script fell back to after the quote broke. ## Footgun 2 — Woodpecker substitutes `${VAR}` itself, before the shell ever sees it Woodpecker performs its own `${VAR}` substitution pass over step commands at pipeline-parse time (its own metadata vars like `${CI_COMMIT_TAG}`), BEFORE the shell runs. So any `${SHELL_VAR}` meant for the shell — e.g. `${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}` — gets mangled by Woodpecker first (undefined var -> silently becomes empty/mangled text), commonly producing an "unbound variable" failure under `set -u`. The fix/convention (see oleks/mempalace commit `33b880d`, referenced in a comment in oleks/mempalace's `.woodpecker/test.yaml`) is to escape as `$$` when the shell should own the expansion: `$${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}`. This also bites inside comments — a comment merely mentioning `${VAR}` gets substituted too, since Woodpecker's substitution runs over the raw text. ## Ask Add static checks to pipetree (probably folded into its existing `-analyze` mode, alongside the existing shared-image/broken-depends_on checks) that scan each step's `commands:` text for both patterns and flag them, with care to avoid false positives: - legitimate `'\''` escaping should not be flagged as footgun 1 - correctly-escaped `$$` for shell-owned expansion should not be flagged as footgun 2 - Woodpecker's own legitimate `${CI_*}` builtins should not be flagged as footgun 2 Both footguns are static and will recur across the fleet, so catching them at scan time (before a pipeline run burns 10+ minutes discovering them) is worth the false-positive-avoidance effort. A PR implementing static detection for both will reference this issue.
oleks added the enhancement label 2026-08-03 19:41:55 +03:00
oleks closed this issue 2026-08-03 19:52:47 +03:00
oleks commented 2026-08-03 19:52:57 +03:00
Author
Owner
Copy Link
Copy Source

Merged via oleks/pipetree#12, merge commit 8ff6f3207e1531f8150b0c73d553b4e4cce4b20f. Issue auto-closed by the PR's Closes keyword.

Fleet-wide validation result: 0 stray apostrophes, 8 genuine unescaped-var findings in oleks/deals-site, oleks/element-web-patched (${BUILDKIT_ADDR}, ${HP}) and oleks/terminal-agent (${wait}) — all real shell variables that should be $${…}, none of which have bitten yet.

Merged via oleks/pipetree#12, merge commit `8ff6f3207e1531f8150b0c73d553b4e4cce4b20f`. Issue auto-closed by the PR's `Closes` keyword. Fleet-wide validation result: **0 stray apostrophes**, **8 genuine unescaped-var findings** in oleks/deals-site, oleks/element-web-patched (`${BUILDKIT_ADDR}`, `${HP}`) and oleks/terminal-agent (`${wait}`) — all real shell variables that should be `$${…}`, none of which have bitten yet.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
enhancement
New capability or improvement
No labels enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
oleks
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: oleks/pipetree#11
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?