diff --git a/README.md b/README.md index fb7cdee..83a8ead 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,20 @@ are excluded from scans — this keeps vendored upstream mirrors' own CI test fixtures (e.g. the `woodpecker` repo's own `test-dag`/`test-when`/... corpus) out of fleet-wide results. Pass `-all` to include everything. +`-analyze` also flags two Woodpecker footguns that are statically visible in +a step's `commands:` text, both of which have cost a real pipeline run: + +- a stray apostrophe inside a `-c bash -euxc '...'`-style single-quoted + shell block — including one inside a `#` comment — closes the quote + early and hands the rest of the script to the outer shell, usually + surfacing as a baffling "command not found" for a tool that IS + installed (oleks/mempalace pipeline 218). +- an unescaped `${VAR}` in a step command — Woodpecker substitutes braced + `${VAR}` in the raw pipeline text at parse time, before the shell runs, + so a shell variable meant for the shell needs to be escaped as `$${VAR}` + (Woodpecker's own `${CI_*}` builtins and a pipeline's own `matrix:` axis + names are the legitimate unescaped exceptions and aren't flagged). + ## Build ```sh diff --git a/SPEC.md b/SPEC.md index 3a7416a..173fb68 100644 --- a/SPEC.md +++ b/SPEC.md @@ -154,6 +154,67 @@ whoever ran this, not part of pipetree's own tracker): skipping" instead of an error. Any other status/network failure still surfaces as a real error, unchanged. +## Shipped (v0.4, footgun detection) + +`-analyze` now also flags two statically-detectable Woodpecker footguns +inside step `commands:` text (`internal/analyze/footguns.go`), on top of +the existing shared-image/broken-depends_on checks. Both were motivated by +a real pipeline failure (oleks/mempalace, 2026-08-03) and grounded in +reading actual incident commits, not guessed: + +- **StrayApostrophe**: the common `nix shell ... -c bash -euxc + '...multi-line script...'` idiom wraps the whole script in ONE + single-quoted shell argument. Bash gives single quotes no escape + mechanism other than the close-escape-reopen trick (`'`, `\`, `''`), so + ANY literal `'` inside the block — including inside a `#` comment — + closes the quote early and hands everything after it to the outer + shell to re-tokenize. Live incident: oleks/mempalace pipeline 218 + failed `exit 127` ("patch: command not found", though patch WAS + installed) because a comment added in `3fb2b02` contained the word + "Woodpecker's" — root-caused and fixed in commit `54899ec` + (oleks/mempalace#87/#90). The exact same idiom had already been bitten + once before (commit `e409e70`, oleks/mempalace#68). + + Detection: find the idiom's opening line (`-c '`/`-euxc '` etc. at the + end of a line) and its dedicated closing line (a line that trims to + exactly `'`); any additional unescaped `'` strictly between them is + flagged. Commands that don't match this precise shape (no dedicated + closing line) are left unchecked rather than guessed at, to avoid false + positives on differently-structured single-quoted commands. + +- **UnescapedWoodpeckerVar**: Woodpecker substitutes braced `${VAR}` in + the raw pipeline YAML text at parse time, before the shell ever runs — + including inside comments, since the substitution pass has no concept + of what's a comment. A shell variable meant for the shell (e.g. + `${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}`) gets mangled (undefined -> + empty/malformed text), typically surfacing as an "unbound variable" + failure under `set -u`. Fix: escape as `$${VAR}` so Woodpecker's + substitution collapses the `$$` to a literal `$`, leaving `${VAR}` + intact for the shell (commit `33b880d`, oleks/mempalace#44). + + Detection: any `${NAME...}` not immediately preceded by a second `$` + (the `$${...}` escape) and whose name isn't one of Woodpecker's own + `CI_*` metadata builtins or the pipeline's own `matrix:` axis names — + Woodpecker's matrix feature works BY substituting `${AXIS_NAME}` + throughout the YAML with each axis value before parsing, so that's a + second class of legitimate unescaped use alongside `CI_*`. + +**False positive found and fixed during fleet validation**: the first +fleet-wide run flagged `${TARGET_ARCH}` in `oleks/ii-researcher`, +`oleks/csi-s3`, and `oleks/common-chronicle` (18 hits total) — all three +declare `matrix: {TARGET_ARCH: [...]}` and reference `${TARGET_ARCH}` +unescaped in the step name and commands, which is Woodpecker's matrix +feature working exactly as documented, not a mistake. Excluding a +pipeline's own matrix axis names (alongside `CI_*`) dropped this to 0 +false positives; the check still correctly flags real footguns (fleet run +after the fix: 0 stray apostrophes, 8 genuine unescaped-var findings in +`oleks/deals-site`, `oleks/element-web-patched`, and +`oleks/terminal-agent` — shell variables set via step `environment:` or +local assignment, not Woodpecker metadata). Verified `oleks/mempalace` +(both the exact pipeline-218-broken commit and its fix) and +`oleks/oracle-adb-backend` (currently clean) as positive/negative +fixtures. + ## External validation: `woodpecker-cli` The official CLI (`woodpecker-cli`, v3.16.0, matches `ci.oleks.space`) is diff --git a/internal/analyze/analyze.go b/internal/analyze/analyze.go index 5ad4aab..914b051 100644 --- a/internal/analyze/analyze.go +++ b/internal/analyze/analyze.go @@ -1,7 +1,11 @@ // Package analyze looks across an already-scanned set of projects for // structural smells that don't show up looking at any one pipeline file // in isolation: step images worth consolidating, and depends_on -// references that name something that doesn't exist. +// references that name something that doesn't exist. It also runs two +// single-pipeline-local footgun checks (see footguns.go) alongside the +// cross-pipeline ones, since -analyze is already the closest thing this +// tool has to a lint mode and both footguns are things a human skims past +// without noticing. // // Both depends_on checks below are grounded in reading Woodpecker's own // source (git.oleks.space/oleks/woodpecker, pipeline/frontend/yaml/ @@ -68,6 +72,8 @@ type Report struct { SharedImages []SharedImage BrokenStepDependsOn []BrokenStepDependsOn BrokenPipelineDependsOn []BrokenPipelineDependsOn + StrayApostrophes []StrayApostrophe + UnescapedVars []UnescapedWoodpeckerVar } // Run analyzes projects and returns a Report, each section sorted for @@ -77,8 +83,12 @@ func Run(projects []*model.Project) Report { imageRefs := map[string][]PipelineRef{} var brokenStep []BrokenStepDependsOn var brokenPipeline []BrokenPipelineDependsOn + var strayApostrophes []StrayApostrophe + var unescapedVars []UnescapedWoodpeckerVar for _, proj := range projects { + runFootguns(proj, &strayApostrophes, &unescapedVars) + pipelineNames := make(map[string]bool, len(proj.Pipelines)) for _, pl := range proj.Pipelines { pipelineNames[pl.Name] = true @@ -147,5 +157,13 @@ func Run(projects []*model.Project) Report { return brokenPipeline[i].Pipeline < brokenPipeline[j].Pipeline }) - return Report{SharedImages: shared, BrokenStepDependsOn: brokenStep, BrokenPipelineDependsOn: brokenPipeline} + sortFootguns(strayApostrophes, unescapedVars) + + return Report{ + SharedImages: shared, + BrokenStepDependsOn: brokenStep, + BrokenPipelineDependsOn: brokenPipeline, + StrayApostrophes: strayApostrophes, + UnescapedVars: unescapedVars, + } } diff --git a/internal/analyze/footguns.go b/internal/analyze/footguns.go new file mode 100644 index 0000000..8bd4de8 --- /dev/null +++ b/internal/analyze/footguns.go @@ -0,0 +1,237 @@ +// Footgun detection for two Woodpecker CI gotchas that are statically +// visible in a step's `commands:` text and have both cost a real pipeline +// run (oleks/mempalace, 2026-08-03 and originally 2026-07-31): +// +// 1. StrayApostrophe: the common `nix shell ... -c bash -euxc '...multi- +// line script...'` idiom wraps the whole script in ONE single-quoted +// shell argument. Bash gives single quotes no escape mechanism at all +// (other than the close-escape-reopen trick, written as a single +// quote, backslash, then two single quotes) - so ANY literal +// `'` inside the block, including inside a `#` comment, closes the +// quote early and hands everything after it to the outer shell to +// re-tokenize, usually surfacing as a baffling "command not found" for +// a tool that IS installed. Live incident: oleks/mempalace pipeline +// 218 failed exit 127 ("patch: command not found", patch WAS +// installed) because a comment added in 3fb2b02 contained the word +// "Woodpecker's" - root-caused and fixed in commit 54899ec +// (oleks/mempalace#87/#90). The same class of bug had already bitten +// this exact idiom once before (commit e409e70, oleks/mempalace#68: +// "test_mcp_http_transport.py's" in a comment) - which is exactly why +// it's worth detecting statically rather than trusting people to +// remember not to use apostrophes in comments forever. +// +// 2. UnescapedWoodpeckerVar: Woodpecker substitutes braced `${VAR}` +// references in the raw pipeline YAML text itself, at parse time, +// before the shell ever runs - including inside comments, since the +// substitution pass has no idea what's a comment. A shell variable +// meant for the shell (e.g. `${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}`) +// gets mangled (undefined -> empty/malformed text), typically +// surfacing as an "unbound variable" failure under `set -u`. The fix +// is to escape it as `$$` so Woodpecker's substitution collapses that +// to a literal `$`, leaving `${VAR}` intact for the shell - see +// oleks/mempalace commit 33b880d (oleks/mempalace#44). Woodpecker's +// own builtin metadata vars (`${CI_*}`) are the one legitimate +// unescaped use and are not flagged. +package analyze + +import ( + "regexp" + "sort" + "strings" + + "git.oleks.space/oleks/pipetree/internal/model" +) + +// StrayApostrophe is one line inside a `-c '...'`-style single-quoted +// shell block whose apostrophe isn't the block's own close-escape-reopen +// idiom (a single quote, backslash, then two single quotes) - it closes +// the quote early, corrupting everything after it in the script. +type StrayApostrophe struct { + Project string + Pipeline string + Step string + Line int // 1-based line number within the command's text (not the file) + Context string // the offending line, trimmed +} + +// UnescapedWoodpeckerVar is a `${VAR}` in a step command that Woodpecker +// will substitute at parse time (before the shell runs) because it isn't +// escaped as `$${VAR}` and isn't one of Woodpecker's own `CI_*` builtins. +type UnescapedWoodpeckerVar struct { + Project string + Pipeline string + Step string + Line int + Var string + Context string +} + +// openQuoteRe matches a line (right-trimmed) ending in a `-c '`/`-euxc '` +// style flag that opens a single-quoted shell argument - the idiom is +// `... -c bash -euxc '` as the last thing on the line, with the script +// body starting on the next line. +var openQuoteRe = regexp.MustCompile(`-\S*c\s+'$`) + +// dollarBraceRe matches `${NAME...}` - NAME must start with a letter or +// underscore (excludes positional/special params like ${1} or ${@}, which +// Woodpecker's metadata substitution doesn't touch anyway) and may be +// followed by any parameter-expansion suffix (`:-default`, `:+alt`, ...). +var dollarBraceRe = regexp.MustCompile(`\$\{([A-Za-z_][A-Za-z0-9_]*)[^}]*\}`) + +// woodpeckerBuiltinVar reports whether name is one of Woodpecker's own +// substituted variables, for which an unescaped `${...}` is the correct, +// intended usage rather than a footgun: +// - namespaced CI_* metadata vars (pipeline/frontend/metadata package); +// - this pipeline's own `matrix:` axis names - Woodpecker's matrix +// feature works BY replacing `${AXIS_NAME}` throughout the YAML text +// with each axis value before parsing, so e.g. `matrix: {TARGET_ARCH: +// [...]}` + `${TARGET_ARCH}` in a step name/command is the documented, +// correct way to use it, not a mistake (confirmed against real fleet +// pipelines: oleks/ii-researcher, oleks/csi-s3, oleks/common-chronicle +// all use this pattern deliberately). +func woodpeckerBuiltinVar(name string, matrixVars map[string]bool) bool { + return strings.HasPrefix(name, "CI_") || matrixVars[name] +} + +// findStrayApostrophes scans one command string for the stray-apostrophe +// footgun. It looks for the `-c '` idiom's opening line, then the nearest +// following line that is - once trimmed - exactly `'` (the idiom's own +// dedicated closing line). Any additional, non-escaped `'` strictly +// between those two lines closes the shell's quote early. Commands with no +// such dedicated closing line are left alone rather than guessed at, to +// avoid false positives on differently-shaped single-quoted commands. +func findStrayApostrophes(cmd string) []struct { + Line int + Context string +} { + var hits []struct { + Line int + Context string + } + lines := strings.Split(cmd, "\n") + for i, raw := range lines { + if !openQuoteRe.MatchString(strings.TrimRight(raw, " \t\r")) { + continue + } + closeIdx := -1 + for j := i + 1; j < len(lines); j++ { + if strings.TrimSpace(lines[j]) == "'" { + closeIdx = j + break + } + } + if closeIdx == -1 { + // No dedicated closing line found for this opening - the + // script doesn't match the idiom precisely enough to check + // reliably, so skip rather than risk a false positive. + continue + } + for k := i + 1; k < closeIdx; k++ { + // Strip the legitimate `'\''` close-escape-reopen idiom before + // looking for a stray quote. + stripped := strings.ReplaceAll(lines[k], `'\''`, "") + if strings.Contains(stripped, "'") { + hits = append(hits, struct { + Line int + Context string + }{Line: k + 1, Context: strings.TrimSpace(lines[k])}) + } + } + } + return hits +} + +// findUnescapedVars scans one command string for `${VAR}` references that +// Woodpecker will substitute at parse time: any `${...}` not immediately +// preceded by a second `$` (the `$${...}` escape that hands the expansion +// to the shell) and whose var name isn't one of Woodpecker's own `CI_*` +// builtins or this pipeline's own matrix axis names (see +// woodpeckerBuiltinVar). matrixVars may be nil. +func findUnescapedVars(cmd string, matrixVars map[string]bool) []struct { + Line int + Var string + Context string +} { + var hits []struct { + Line int + Var string + Context string + } + lines := strings.Split(cmd, "\n") + for i, line := range lines { + trimmed := strings.TrimSpace(line) + for _, loc := range dollarBraceRe.FindAllStringSubmatchIndex(line, -1) { + start := loc[0] + name := line[loc[2]:loc[3]] + if start > 0 && line[start-1] == '$' { + continue // $${...} - correctly escaped for the shell + } + if woodpeckerBuiltinVar(name, matrixVars) { + continue // Woodpecker's own builtin metadata or matrix var + } + hits = append(hits, struct { + Line int + Var string + Context string + }{Line: i + 1, Var: name, Context: trimmed}) + } + } + return hits +} + +// runFootguns appends the two footgun checks' findings (for one project's +// pipelines) onto the accumulators passed in. +func runFootguns(proj *model.Project, apos *[]StrayApostrophe, vars *[]UnescapedWoodpeckerVar) { + for _, pl := range proj.Pipelines { + var matrixVars map[string]bool + if len(pl.Matrix) > 0 { + matrixVars = make(map[string]bool, len(pl.Matrix)) + for axis := range pl.Matrix { + matrixVars[axis] = true + } + } + for _, s := range pl.Steps { + for _, cmd := range s.Commands { + for _, h := range findStrayApostrophes(cmd) { + *apos = append(*apos, StrayApostrophe{ + Project: proj.Name, Pipeline: pl.Name, Step: s.Name, + Line: h.Line, Context: h.Context, + }) + } + for _, h := range findUnescapedVars(cmd, matrixVars) { + *vars = append(*vars, UnescapedWoodpeckerVar{ + Project: proj.Name, Pipeline: pl.Name, Step: s.Name, + Line: h.Line, Var: h.Var, Context: h.Context, + }) + } + } + } + } +} + +func sortFootguns(apos []StrayApostrophe, vars []UnescapedWoodpeckerVar) { + sort.Slice(apos, func(i, j int) bool { + if apos[i].Project != apos[j].Project { + return apos[i].Project < apos[j].Project + } + if apos[i].Pipeline != apos[j].Pipeline { + return apos[i].Pipeline < apos[j].Pipeline + } + if apos[i].Step != apos[j].Step { + return apos[i].Step < apos[j].Step + } + return apos[i].Line < apos[j].Line + }) + sort.Slice(vars, func(i, j int) bool { + if vars[i].Project != vars[j].Project { + return vars[i].Project < vars[j].Project + } + if vars[i].Pipeline != vars[j].Pipeline { + return vars[i].Pipeline < vars[j].Pipeline + } + if vars[i].Step != vars[j].Step { + return vars[i].Step < vars[j].Step + } + return vars[i].Line < vars[j].Line + }) +} diff --git a/internal/analyze/footguns_test.go b/internal/analyze/footguns_test.go new file mode 100644 index 0000000..b73e331 --- /dev/null +++ b/internal/analyze/footguns_test.go @@ -0,0 +1,235 @@ +package analyze + +import ( + "strings" + "testing" + + "git.oleks.space/oleks/pipetree/internal/model" + "git.oleks.space/oleks/pipetree/internal/parse" +) + +// loadProject parses one pipeline file (a testdata fixture) into a +// single-pipeline model.Project, the same shape buildProjects (main.go) +// produces for a real scan. +func loadProject(t *testing.T, name, file string) *model.Project { + t.Helper() + pl, err := parse.File(file) + if err != nil { + t.Fatalf("parse.File(%s): %v", file, err) + } + pl.Name = "default" + return &model.Project{Name: name, Path: file, Pipelines: []*model.Pipeline{pl}} +} + +// TestStrayApostrophe_PositiveCase reproduces the exact shape of the bug +// that broke oleks/mempalace pipeline 218 on 2026-08-03 (commit 54899ec^, +// fixed by 54899ec, oleks/mempalace#87/#90): a comment inside the +// `-c bash -euxc '...'` block containing the word "Woodpecker's" closes +// the quote early, dropping the rest of the script to the outer shell - +// the pipeline failed with a baffling "patch: command not found" (exit +// 127) even though patch was installed, just not on the fallback shell's +// PATH. Verifies pipetree catches it. +func TestStrayApostrophe_PositiveCase(t *testing.T) { + proj := loadProject(t, "mempalace", "testdata/mempalace-stray-apostrophe.yaml") + report := Run([]*model.Project{proj}) + + if len(report.StrayApostrophes) == 0 { + t.Fatal("expected at least one stray apostrophe finding, got none") + } + found := false + for _, a := range report.StrayApostrophes { + if a.Step != "pytest-ruff" { + t.Errorf("finding on unexpected step %q", a.Step) + } + found = found || strings.Contains(a.Context, "Woodpecker's") + } + if !found { + t.Errorf("expected a finding pointing at the offending line (\"...Woodpecker's...\"), got: %+v", report.StrayApostrophes) + } +} + +// TestUnescapedVar_PositiveCase reproduces the shape of the bug fixed in +// oleks/mempalace commit 33b880d (oleks/mempalace#44): an unescaped +// `${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}` that Woodpecker substitutes at +// parse time, before the shell runs. The same file also legitimately uses +// `${CI_COMMIT_TAG:-v3.5.0}` (a real Woodpecker builtin), which must NOT +// be flagged. +func TestUnescapedVar_PositiveCase(t *testing.T) { + proj := loadProject(t, "mempalace", "testdata/mempalace-unescaped-var.yaml") + report := Run([]*model.Project{proj}) + + var gotLDLibraryPath, gotCIBuiltin bool + for _, u := range report.UnescapedVars { + if u.Var == "LD_LIBRARY_PATH" { + gotLDLibraryPath = true + } + if u.Var == "CI_COMMIT_TAG" { + gotCIBuiltin = true + } + } + if !gotLDLibraryPath { + t.Errorf("expected LD_LIBRARY_PATH to be flagged as an unescaped shell var, got: %+v", report.UnescapedVars) + } + if gotCIBuiltin { + t.Errorf("CI_COMMIT_TAG is a legitimate Woodpecker builtin and must not be flagged, got: %+v", report.UnescapedVars) + } + // This fixture's single-quoted shell block has no apostrophe footgun. + if len(report.StrayApostrophes) != 0 { + t.Errorf("expected no stray-apostrophe findings in this fixture, got: %+v", report.StrayApostrophes) + } +} + +// TestFootguns_NegativeCase_KnownGoodFixture is the current (post-fix), +// live oleks/mempalace .woodpecker/test.yaml: correctly-escaped $${...} +// var references, a legitimate ${CI_COMMIT_TAG} builtin use outside any +// single-quoted block, and an apostrophe-free -c bash -euxc '...' block. +// Regression guard for false positives - this exact file was flagged as +// "known-good post-fix" and a hit here would need fixing before shipping. +func TestFootguns_NegativeCase_KnownGoodFixture(t *testing.T) { + proj := loadProject(t, "mempalace", "testdata/mempalace-fixed.yaml") + report := Run([]*model.Project{proj}) + + if len(report.StrayApostrophes) != 0 { + t.Errorf("false positive: expected no stray-apostrophe findings on the known-good fixture, got: %+v", report.StrayApostrophes) + } + if len(report.UnescapedVars) != 0 { + t.Errorf("false positive: expected no unescaped-var findings on the known-good fixture, got: %+v", report.UnescapedVars) + } +} + +// TestFootguns_NegativeCase_MatrixVar guards against the false positive +// found while validating against the real fleet: oleks/ii-researcher (and +// equivalently oleks/csi-s3, oleks/common-chronicle) declares `matrix: +// {TARGET_ARCH: [...]}` and legitimately references `${TARGET_ARCH}` +// unescaped in the step name and commands - that's Woodpecker's own +// matrix-substitution feature working as intended, not a footgun, and +// must not be flagged. +func TestFootguns_NegativeCase_MatrixVar(t *testing.T) { + proj := loadProject(t, "ii-researcher", "testdata/matrix-fixture.yaml") + report := Run([]*model.Project{proj}) + + for _, u := range report.UnescapedVars { + if u.Var == "TARGET_ARCH" { + t.Errorf("false positive: TARGET_ARCH is this pipeline's own matrix axis, must not be flagged: %+v", u) + } + } +} + +// TestFindStrayApostrophes_Unit exercises the line-level detector directly +// against small synthetic commands, independent of YAML parsing. +func TestFindStrayApostrophes_Unit(t *testing.T) { + cases := []struct { + name string + cmd string + want int + }{ + { + name: "clean block, no apostrophe", + cmd: "nix shell nixpkgs#bash -c bash -euxc '\n" + + " echo hello\n" + + " echo world\n" + + "'", + want: 0, + }, + { + name: "apostrophe in a comment ends the quote early", + cmd: "nix shell nixpkgs#bash -c bash -euxc '\n" + + " # this is Woodpecker's escape hatch\n" + + " echo hello\n" + + "'", + want: 1, + }, + { + name: "legitimate close-escape-reopen idiom is not flagged", + cmd: "nix shell nixpkgs#bash -c bash -euxc '\n" + + " echo '\\''it'\\''s fine'\\''\n" + + "'", + want: 0, + }, + { + name: "no dedicated closing line - not checked (avoid false positive)", + cmd: "nix shell nixpkgs#bash -c bash -euxc 'echo hi'", + want: 0, + }, + { + name: "no single-quoted -c block at all", + cmd: "echo \"it's a plain command, no -c '...' idiom here\"", + want: 0, + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := findStrayApostrophes(tc.cmd) + if len(got) != tc.want { + t.Errorf("findStrayApostrophes(%q) = %d hits %+v, want %d", tc.cmd, len(got), got, tc.want) + } + }) + } +} + +// TestFindUnescapedVars_Unit exercises the ${VAR} detector directly. +func TestFindUnescapedVars_Unit(t *testing.T) { + cases := []struct { + name string + cmd string + matrixVars map[string]bool + want []string + }{ + { + name: "correctly escaped shell var", + cmd: `export LD_LIBRARY_PATH="$CCLIB/lib$${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"`, + want: nil, + }, + { + name: "unescaped shell var", + cmd: `export LD_LIBRARY_PATH="$CCLIB/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"`, + want: []string{"LD_LIBRARY_PATH"}, + }, + { + name: "legitimate unescaped Woodpecker builtin", + cmd: `VERSION=$(printf "%s" "${CI_COMMIT_TAG:-v3.5.0}")`, + want: nil, + }, + { + name: "unescaped var mentioned inside a comment still bites", + cmd: "# don't forget ${MY_SECRET} needs escaping\necho hi", + want: []string{"MY_SECRET"}, + }, + { + name: "bare $VAR and $(...) are unaffected", + cmd: `echo $HOME && VER=$(date +%s)`, + want: nil, + }, + { + // Woodpecker's matrix feature works BY substituting ${AXIS} + // throughout the YAML with each axis value before parsing - + // confirmed live in oleks/ii-researcher, oleks/csi-s3, + // oleks/common-chronicle. This is the correct, intended use, + // not a footgun, so a pipeline's own matrix axis names must + // not be flagged. + name: "matrix axis var is not a footgun", + cmd: `ci/local.sh --arch "${TARGET_ARCH}"`, + matrixVars: map[string]bool{"TARGET_ARCH": true}, + want: nil, + }, + { + name: "non-matrix var still flagged even with a matrix in scope", + cmd: `echo "${OTHER_VAR}"`, + matrixVars: map[string]bool{"TARGET_ARCH": true}, + want: []string{"OTHER_VAR"}, + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := findUnescapedVars(tc.cmd, tc.matrixVars) + if len(got) != len(tc.want) { + t.Fatalf("findUnescapedVars(%q) = %+v, want vars %v", tc.cmd, got, tc.want) + } + for i, w := range tc.want { + if got[i].Var != w { + t.Errorf("hit %d: var = %q, want %q", i, got[i].Var, w) + } + } + }) + } +} diff --git a/internal/analyze/testdata/matrix-fixture.yaml b/internal/analyze/testdata/matrix-fixture.yaml new file mode 100644 index 0000000..fd65726 --- /dev/null +++ b/internal/analyze/testdata/matrix-fixture.yaml @@ -0,0 +1,78 @@ +labels: + arch: amd64 + +clone: + - name: clone + image: woodpeckerci/plugin-git + environment: + CI_NETRC_MACHINE: git.oleks.space + CI_NETRC_USERNAME: oleks + CI_NETRC_PASSWORD: + from_secret: gitea_clone_token + PLUGIN_TAGS: "false" + PLUGIN_DEPTH: "1" + +# DISABLED: replaced by .woodpecker/arm64.yaml and .woodpecker/amd64.yaml +# which run on native arch nodes. +when: + - event: manual + evaluate: "false" + +# Matrix: uncomment additional arches to enable multi-arch builds +# Remote buildkit workers do the native-arch build per target. +matrix: + TARGET_ARCH: + # - s390x + - arm64 + # - amd64 + +steps: + - name: build-api-${TARGET_ARCH} + image: git.oleks.space/oleks/nix-ci:latest + environment: + REGISTRY_TOKEN: + from_secret: registry_token + commands: + - echo "▸ arch=$(uname -m)" + - | + TAG=$(echo "$CI_COMMIT_TAG" | sed 's/^v//') + CHECK=$(curl -sf -H "Authorization: token $REGISTRY_TOKEN" \ + "https://git.oleks.space/api/v1/packages/oleks?type=container&q=ii-researcher/api&limit=50" || echo "") + if echo "$CHECK" | grep -q "\"version\":\"$TAG-${TARGET_ARCH}\""; then + echo "Image ii-researcher/api:$TAG-${TARGET_ARCH} already exists, skipping build" + exit 0 + fi + - echo "$REGISTRY_TOKEN" | docker login git.oleks.space -u oleks --password-stdin + - | + BUILDER_HOST="buildkit-rootless-${TARGET_ARCH}.infra.svc.cluster.local" + BUILDER_PORT="1234" + echo "Waiting for builder..." + for i in $(seq 1 30); do + if echo >/dev/tcp/$BUILDER_HOST/$BUILDER_PORT 2>/dev/null; then + echo "Builder ready"; break + fi + [ "$i" -eq 30 ] && echo "Builder not available" && exit 1 + sleep 10 + done + docker buildx create --name ${TARGET_ARCH}-api --driver remote "tcp://$BUILDER_HOST:$BUILDER_PORT" + TAG=$(echo "$CI_COMMIT_TAG" | sed 's/^v//') + docker buildx build \ + --builder ${TARGET_ARCH}-api \ + --platform linux/${TARGET_ARCH} \ + --build-arg REGISTRY_TOKEN="$REGISTRY_TOKEN" \ + -f Dockerfile.api \ + -t "git.oleks.space/oleks/ii-researcher/api:$TAG-${TARGET_ARCH}" \ + --push . + backend_options: + kubernetes: + nodeSelector: + provider: digitalocean + resources: + requests: + memory: 6Gi + limits: + memory: 6Gi + labels: + commit-tag: "${CI_COMMIT_TAG}" + commit-branch: "${CI_COMMIT_BRANCH}" + pipeline-number: "${CI_PIPELINE_NUMBER}" diff --git a/internal/analyze/testdata/mempalace-fixed.yaml b/internal/analyze/testdata/mempalace-fixed.yaml new file mode 100644 index 0000000..45da6ff --- /dev/null +++ b/internal/analyze/testdata/mempalace-fixed.yaml @@ -0,0 +1,135 @@ +# CI gate for the patched mempalace tree (oleks/mempalace#44). +# +# The image build applies an ordered stack of downstream patches to the upstream +# v3.5.0 tarball (see flake.nix `patches = [ ... ]`), but the build itself does +# NOT run the test suite — so a behavior patch that breaks an upstream test, or +# pushes a function over ruff's C901 budget, would ship silently. This job +# reconstructs the exact patched tree and runs pytest + ruff so patch drift +# fails CI on push/PR, BEFORE a release tag builds an image. +# +# DRY: the patch list + order are read straight out of flake.nix (the single +# source of truth), so this can never fall out of sync with what the image builds. +# arch: arm64 — mempalace is arm64-only; an amd64 label mis-scheduled this onto +# agents that report labels:null and the long clone got context-canceled +# (oleks/mempalace#41 CI clone-cancel root cause). +labels: + arch: arm64 + +# push+pull_request together double-run every commit on a branch with an open +# PR (oleks/cluster#364's trigger case) -- push is scoped to main (post-merge +# validation) so pull_request alone covers feature branches. +when: + - event: pull_request + - event: push + branch: main + - event: manual + +steps: + - name: pytest-ruff + image: git.oleks.space/oleks/nix-ci:latest + environment: + # ci/setup.sh writes a git.oleks.space netrc from this — without it, + # evaluating this flake's private git+https inputs (fleet-pins, + # heatwave-backend, parity-lib) to resolve `.#devShells..test` + # fails the same way flake-check.yaml's GITEA_CLONE_TOKEN comment + # describes (oleks/mempalace#57). + GITEA_CLONE_TOKEN: + from_secret: gitea_clone_token + commands: + - echo "▸ arch=$(uname -m)" + - sh ci/setup.sh + # `nix develop .#test` now evaluates this repo's own flake.nix (unlike the + # old bare `nix shell nixpkgs#...`, which never touched it), and flake.nix + # unconditionally `import`s the gitignored version.nix — so it has to exist + # (and be `git add -f`'d, matching flake-check.yaml/arm64.yaml: a local git + # flake's pure eval only sees git-tracked/staged files) before any output + # can be evaluated, even one that doesn't otherwise care about VERSION. + - VERSION=$(echo "${CI_COMMIT_TAG:-v3.5.0}" | sed 's/^v//; s/-[0-9]*$//') + - printf '"%s"\n' "$VERSION" > version.nix && git add -f version.nix + # oleks/mempalace#68/#73/#325 fix: narrow the test shell to lightweight direct + # deps only (uv, ruff, python3, curl, tar, gzip, sed, grep, patch, coreutils), + # pinned to THIS repo's nixpkgs (fleet/nixpkgs-ci) via flake.nix's input pins. + # This avoids evaluating the full flake.nix (which pulls in image-build + # infrastructure + deep transitive closures), and restores the fast, + # reproducible cache locality from pre-#73. The #68 PATH fix (pinned uv/ruff) + # is preserved: nix shell resolves them against the fleet pin, not live + # nixpkgs-unstable. oleks/cluster#325 (slow attic cache) no longer dominates + # the setup phase because we're not pulling full image-build closures anymore. + # Resolved out here, where `nix` is still on PATH (inside `nix shell` it is + # not). Exported so the apostrophe-free block below can consume it. + - export CXX_LIB_DIR="$(nix eval --raw nixpkgs#stdenv.cc.cc.lib.outPath)/lib" + - | + # Read nixpkgs from flake.lock to ensure we use the fleet-pinned version, + # matching what the full flake would use. Avoids nixpkgs-unstable drift. + nix shell \ + "nixpkgs#uv" \ + "nixpkgs#ruff" \ + "nixpkgs#python312" \ + "nixpkgs#stdenv.cc.cc.lib" \ + "nixpkgs#curl" \ + "nixpkgs#gnutar" \ + "nixpkgs#gzip" \ + "nixpkgs#gnused" \ + "nixpkgs#gnugrep" \ + "nixpkgs#patch" \ + "nixpkgs#coreutils" \ + -c bash -euxc ' + # Explicit bash, not the base images bare /bin/sh (dash/busybox, + # unknown which) — see oleks/mempalace#68 for the debugging this + # replaced. All tools above come from the pinned nixpkgs, so + # they are guaranteed present on PATH without a retry/poll loop. + UV_BIN=$(command -v uv) + RUFF_BIN=$(command -v ruff) + echo "resolved: uv=$UV_BIN ruff=$RUFF_BIN" + # Use the nix-provided CPython, NOT a uv-downloaded standalone one: the + # nix-ci image has no nix-ld, so uv-fetched interpreters cannot find their + # dynamic loader ("Python interpreter not found"). only-system + never + # makes uv build the venv from python3.12 on PATH. + export UV_PYTHON_DOWNLOADS=never UV_PYTHON_PREFERENCE=only-system + + # numpy/chromadb arrive as manylinux wheels that dlopen libstdc++.so.6, + # which the nix-ci image does not provide (no nix-ld either). devShells.test + # sets this, but a6f9b5c stopped CI from using that shell when it narrowed + # the step to a bare `nix shell` for cache locality (#73/#325) -- and dropped + # stdenv.cc.cc.lib with it, regressing the #190/PR#77 fix (88cfa1b) and + # leaving main red (oleks/mempalace#87/#90). Re-added here rather than + # reverting to `nix develop .#test`, which would pull back the image-build + # closures a6f9b5c deliberately removed. + # + # NOTE: this whole block is a single-quoted `bash -euxc ...` argument, so + # an apostrophe anywhere in it -- including in a comment -- terminates the + # quote early and silently drops the rest of the script back to /bin/sh + # (seen live: pipeline 218 failed with "patch: command not found"). + # Keep this block apostrophe-free. + # + # LIB_DIR is resolved OUTSIDE this shell and passed in via the environment: + # `nix shell` puts only the listed packages on PATH, and nix itself is not + # one of them, so `nix eval` in here would be command-not-found. + export LD_LIBRARY_PATH="$${CXX_LIB_DIR}$${LD_LIBRARY_PATH:+:$$LD_LIBRARY_PATH}" + + # version.nix is gitignored; take it from the release tag when present + # (v3.5.0-7 -> 3.5.0), else the current default (bump with version.nix). + VERSION=$(printf "%s" "$${CI_COMMIT_TAG:-v3.5.0}" | sed "s/^v//; s/-[0-9]*$//") + echo "mempalace version=$VERSION" + + curl -fsSL "https://github.com/milla-jovovich/mempalace/archive/refs/tags/v$VERSION.tar.gz" | tar xz + cd "mempalace-$VERSION" + + # Apply the patch stack in the EXACT order flake.nix declares it, + # parsed from the `patches = [ ... ];` block (no hand-maintained copy). + PATCHES=$(sed -n "/patches = \[/,/\];/p" ../flake.nix | grep -oE "mempalace-[a-z0-9-]+\.patch") + echo "applying:"; echo "$PATCHES" + for p in $PATCHES; do echo "+ $p"; patch -p1 < "../$p"; done + + # uv pulls the test deps (real chromadb, prometheus_client, pyyaml, + # ruff) into an ephemeral env; the package imports from the patched + # tree via PYTHONPATH — same invocation a developer runs locally. + # mcp/starlette/uvicorn/httpx (oleks/mempalace#27): the streamable-HTTP + # transport test file exercises the real SDK client/server over a real + # loopback socket, matching the real-socket approach the legacy HTTP + # transport test file (test_mcp_http_transport.py) already uses. + PYTHONPATH=. "$UV_BIN" run --python python3.12 --with pytest --with chromadb --with pyyaml --with prometheus_client \ + --with "mcp>=1.29,<2" --with starlette --with uvicorn --with httpx \ + pytest tests/test_mcp_server.py tests/test_mcp_streamable_http.py -q -p no:cacheprovider + "$RUFF_BIN" check mempalace/mcp_server.py mempalace/mcp_streamable_http.py + ' diff --git a/internal/analyze/testdata/mempalace-stray-apostrophe.yaml b/internal/analyze/testdata/mempalace-stray-apostrophe.yaml new file mode 100644 index 0000000..48c9402 --- /dev/null +++ b/internal/analyze/testdata/mempalace-stray-apostrophe.yaml @@ -0,0 +1,123 @@ +# CI gate for the patched mempalace tree (oleks/mempalace#44). +# +# The image build applies an ordered stack of downstream patches to the upstream +# v3.5.0 tarball (see flake.nix `patches = [ ... ]`), but the build itself does +# NOT run the test suite — so a behavior patch that breaks an upstream test, or +# pushes a function over ruff's C901 budget, would ship silently. This job +# reconstructs the exact patched tree and runs pytest + ruff so patch drift +# fails CI on push/PR, BEFORE a release tag builds an image. +# +# DRY: the patch list + order are read straight out of flake.nix (the single +# source of truth), so this can never fall out of sync with what the image builds. +# arch: arm64 — mempalace is arm64-only; an amd64 label mis-scheduled this onto +# agents that report labels:null and the long clone got context-canceled +# (oleks/mempalace#41 CI clone-cancel root cause). +labels: + arch: arm64 + +# push+pull_request together double-run every commit on a branch with an open +# PR (oleks/cluster#364's trigger case) -- push is scoped to main (post-merge +# validation) so pull_request alone covers feature branches. +when: + - event: pull_request + - event: push + branch: main + - event: manual + +steps: + - name: pytest-ruff + image: git.oleks.space/oleks/nix-ci:latest + environment: + # ci/setup.sh writes a git.oleks.space netrc from this — without it, + # evaluating this flake's private git+https inputs (fleet-pins, + # heatwave-backend, parity-lib) to resolve `.#devShells..test` + # fails the same way flake-check.yaml's GITEA_CLONE_TOKEN comment + # describes (oleks/mempalace#57). + GITEA_CLONE_TOKEN: + from_secret: gitea_clone_token + commands: + - echo "▸ arch=$(uname -m)" + - sh ci/setup.sh + # `nix develop .#test` now evaluates this repo's own flake.nix (unlike the + # old bare `nix shell nixpkgs#...`, which never touched it), and flake.nix + # unconditionally `import`s the gitignored version.nix — so it has to exist + # (and be `git add -f`'d, matching flake-check.yaml/arm64.yaml: a local git + # flake's pure eval only sees git-tracked/staged files) before any output + # can be evaluated, even one that doesn't otherwise care about VERSION. + - VERSION=$(echo "${CI_COMMIT_TAG:-v3.5.0}" | sed 's/^v//; s/-[0-9]*$//') + - printf '"%s"\n' "$VERSION" > version.nix && git add -f version.nix + # oleks/mempalace#68/#73/#325 fix: narrow the test shell to lightweight direct + # deps only (uv, ruff, python3, curl, tar, gzip, sed, grep, patch, coreutils), + # pinned to THIS repo's nixpkgs (fleet/nixpkgs-ci) via flake.nix's input pins. + # This avoids evaluating the full flake.nix (which pulls in image-build + # infrastructure + deep transitive closures), and restores the fast, + # reproducible cache locality from pre-#73. The #68 PATH fix (pinned uv/ruff) + # is preserved: nix shell resolves them against the fleet pin, not live + # nixpkgs-unstable. oleks/cluster#325 (slow attic cache) no longer dominates + # the setup phase because we're not pulling full image-build closures anymore. + - | + # Read nixpkgs from flake.lock to ensure we use the fleet-pinned version, + # matching what the full flake would use. Avoids nixpkgs-unstable drift. + nix shell \ + "nixpkgs#uv" \ + "nixpkgs#ruff" \ + "nixpkgs#python312" \ + "nixpkgs#stdenv.cc.cc.lib" \ + "nixpkgs#curl" \ + "nixpkgs#gnutar" \ + "nixpkgs#gzip" \ + "nixpkgs#gnused" \ + "nixpkgs#gnugrep" \ + "nixpkgs#patch" \ + "nixpkgs#coreutils" \ + -c bash -euxc ' + # Explicit bash, not the base images bare /bin/sh (dash/busybox, + # unknown which) — see oleks/mempalace#68 for the debugging this + # replaced. All tools above come from the pinned nixpkgs, so + # they are guaranteed present on PATH without a retry/poll loop. + UV_BIN=$(command -v uv) + RUFF_BIN=$(command -v ruff) + echo "resolved: uv=$UV_BIN ruff=$RUFF_BIN" + # Use the nix-provided CPython, NOT a uv-downloaded standalone one: the + # nix-ci image has no nix-ld, so uv-fetched interpreters cannot find their + # dynamic loader ("Python interpreter not found"). only-system + never + # makes uv build the venv from python3.12 on PATH. + export UV_PYTHON_DOWNLOADS=never UV_PYTHON_PREFERENCE=only-system + + # numpy/chromadb arrive as manylinux wheels that dlopen libstdc++.so.6, + # which the nix-ci image does not provide (no nix-ld either). devShells.test + # sets this, but a6f9b5c stopped CI from using that shell when it narrowed + # the step to a bare `nix shell` for cache locality (#73/#325) -- and dropped + # stdenv.cc.cc.lib with it, regressing the #190/PR#77 fix (88cfa1b) and + # leaving main red (oleks/mempalace#87/#90). Re-add it here rather than + # reverting to `nix develop .#test`, which would pull the image-build + # closures a6f9b5c deliberately removed. $$ escapes Woodpecker's own + # substitution so the shell owns the expansion (same as 33b880d). + export LD_LIBRARY_PATH="$$(nix eval --raw nixpkgs#stdenv.cc.cc.lib.outPath)/lib$${LD_LIBRARY_PATH:+:$$LD_LIBRARY_PATH}" + + # version.nix is gitignored; take it from the release tag when present + # (v3.5.0-7 -> 3.5.0), else the current default (bump with version.nix). + VERSION=$(printf "%s" "$${CI_COMMIT_TAG:-v3.5.0}" | sed "s/^v//; s/-[0-9]*$//") + echo "mempalace version=$VERSION" + + curl -fsSL "https://github.com/milla-jovovich/mempalace/archive/refs/tags/v$VERSION.tar.gz" | tar xz + cd "mempalace-$VERSION" + + # Apply the patch stack in the EXACT order flake.nix declares it, + # parsed from the `patches = [ ... ];` block (no hand-maintained copy). + PATCHES=$(sed -n "/patches = \[/,/\];/p" ../flake.nix | grep -oE "mempalace-[a-z0-9-]+\.patch") + echo "applying:"; echo "$PATCHES" + for p in $PATCHES; do echo "+ $p"; patch -p1 < "../$p"; done + + # uv pulls the test deps (real chromadb, prometheus_client, pyyaml, + # ruff) into an ephemeral env; the package imports from the patched + # tree via PYTHONPATH — same invocation a developer runs locally. + # mcp/starlette/uvicorn/httpx (oleks/mempalace#27): the streamable-HTTP + # transport test file exercises the real SDK client/server over a real + # loopback socket, matching the real-socket approach the legacy HTTP + # transport test file (test_mcp_http_transport.py) already uses. + PYTHONPATH=. "$UV_BIN" run --python python3.12 --with pytest --with chromadb --with pyyaml --with prometheus_client \ + --with "mcp>=1.29,<2" --with starlette --with uvicorn --with httpx \ + pytest tests/test_mcp_server.py tests/test_mcp_streamable_http.py -q -p no:cacheprovider + "$RUFF_BIN" check mempalace/mcp_server.py mempalace/mcp_streamable_http.py + ' diff --git a/internal/analyze/testdata/mempalace-unescaped-var.yaml b/internal/analyze/testdata/mempalace-unescaped-var.yaml new file mode 100644 index 0000000..42726dd --- /dev/null +++ b/internal/analyze/testdata/mempalace-unescaped-var.yaml @@ -0,0 +1,63 @@ +# CI gate for the patched mempalace tree (oleks/mempalace#44). +# +# The image build applies an ordered stack of downstream patches to the upstream +# v3.5.0 tarball (see flake.nix `patches = [ ... ]`), but the build itself does +# NOT run the test suite — so a behavior patch that breaks an upstream test, or +# pushes a function over ruff's C901 budget, would ship silently. This job +# reconstructs the exact patched tree and runs pytest + ruff so patch drift +# fails CI on push/PR, BEFORE a release tag builds an image. +# +# DRY: the patch list + order are read straight out of flake.nix (the single +# source of truth), so this can never fall out of sync with what the image builds. +# arch: arm64 — mempalace is arm64-only; an amd64 label mis-scheduled this onto +# agents that report labels:null and the long clone got context-canceled +# (oleks/mempalace#41 CI clone-cancel root cause). +labels: + arch: arm64 + +when: + - event: [push, pull_request, manual] + +steps: + - name: pytest-ruff + image: git.oleks.space/oleks/nix-ci:latest + commands: + - echo "▸ arch=$(uname -m)" + # Everything runs inside one nix shell so the exact tool set is guaranteed + # regardless of the base image contents. + - | + nix shell nixpkgs#uv nixpkgs#python312 nixpkgs#curl nixpkgs#gnutar nixpkgs#gzip \ + nixpkgs#gnused nixpkgs#gnugrep nixpkgs#patch nixpkgs#coreutils \ + -c sh -euxc ' + # Use the nix-provided CPython, NOT a uv-downloaded standalone one: the + # nix-ci image has no nix-ld, so uv-fetched interpreters cannot find their + # dynamic loader ("Python interpreter not found"). only-system + never + # makes uv build the venv from python3.12 on PATH. + export UV_PYTHON_DOWNLOADS=never UV_PYTHON_PREFERENCE=only-system + # manylinux wheel .so files (pydantic-core, chromadb deps) are dlopened by + # the interpreter and need libstdc++/libgcc on the loader path (no /usr/lib + # on NixOS). + CCLIB=$(nix build --no-link --print-out-paths nixpkgs#stdenv.cc.cc.lib) + export LD_LIBRARY_PATH="$CCLIB/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" + + # version.nix is gitignored; take it from the release tag when present + # (v3.5.0-7 -> 3.5.0), else the current default (bump with version.nix). + VERSION=$(printf "%s" "${CI_COMMIT_TAG:-v3.5.0}" | sed "s/^v//; s/-[0-9]*$//") + echo "mempalace version=$VERSION" + + curl -fsSL "https://github.com/milla-jovovich/mempalace/archive/refs/tags/v$VERSION.tar.gz" | tar xz + cd "mempalace-$VERSION" + + # Apply the patch stack in the EXACT order flake.nix declares it, + # parsed from the `patches = [ ... ];` block (no hand-maintained copy). + PATCHES=$(sed -n "/patches = \[/,/\];/p" ../flake.nix | grep -oE "mempalace-[a-z0-9-]+\.patch") + echo "applying:"; echo "$PATCHES" + for p in $PATCHES; do echo "+ $p"; patch -p1 < "../$p"; done + + # uv pulls the test deps (real chromadb, prometheus_client, pyyaml, + # ruff) into an ephemeral env; the package imports from the patched + # tree via PYTHONPATH — same invocation a developer runs locally. + PYTHONPATH=. uv run --python python3.12 --with pytest --with chromadb --with pyyaml --with prometheus_client \ + pytest tests/test_mcp_server.py -q -p no:cacheprovider + uv run --python python3.12 --with ruff ruff check mempalace/mcp_server.py + ' diff --git a/internal/render/analyze.go b/internal/render/analyze.go index 66139ce..bd6c987 100644 --- a/internal/render/analyze.go +++ b/internal/render/analyze.go @@ -26,4 +26,14 @@ func Analyze(w io.Writer, report analyze.Report) { for _, b := range report.BrokenPipelineDependsOn { _, _ = fmt.Fprintf(w, " %s / %s depends_on %q (not found)\n", b.Project, b.Pipeline, b.MissingName) } + + _, _ = fmt.Fprintf(w, "\n%d stray apostrophe(s) inside a single-quoted shell block (closes the quote early, drops the rest of the script to the outer shell):\n", len(report.StrayApostrophes)) + for _, a := range report.StrayApostrophes { + _, _ = fmt.Fprintf(w, " %s / %s / step %s, line %d: %s\n", a.Project, a.Pipeline, a.Step, a.Line, a.Context) + } + + _, _ = fmt.Fprintf(w, "\n%d unescaped ${VAR} in step commands (Woodpecker substitutes these at parse time, before the shell runs - escape as $${VAR} to hand it to the shell):\n", len(report.UnescapedVars)) + for _, u := range report.UnescapedVars { + _, _ = fmt.Fprintf(w, " %s / %s / step %s, line %d: ${%s} in %s\n", u.Project, u.Pipeline, u.Step, u.Line, u.Var, u.Context) + } }