ci-scripts

Shared Woodpecker CI templates, plugins, and centralized pipeline conventions for the Oleks ecosystem. This repository hosts reusable CI components (clone steps, build templates, arch-specific runners) and tracks the cross-repo CI optimization effort to reduce duplication across ~50 projects in ~/projects.

pypi-wheel-attic template (oleks/ci-scripts#1)

manifest.yaml + generate.py render .woodpecker.yaml for the fleet of s390x cross-build repos that share the "single build-and-publish step, inline attic watch-store accelerator, tag-triggered PyPI wheel publish" shape. Generated files carry a GENERATED by oleks/ci-scripts — do not hand-edit header; edit manifest.yaml and re-run python3 generate.py instead.

geesefs-s390x, sentry-cli-s390x, attic-client-s390x, angie-arm64, devpi-arm64, and mempalace are bespoke pipelines (OCI image publish, binary builds, multi-file workflows) and are out of scope for either template.

Verify with:

python3 generate.py --check   # fails if any rendered file is stale
woodpecker-cli lint <repo>/.woodpecker.yaml

thin-parity-lib-publish template (oleks/ci-scripts#10)

The same generate.py also renders a second, genuinely different shape via render_thin() + the thin_repos: list in manifest.yaml: no inline attic accelerator, publish is a pure nix run .#publish[-s390x] front door, optionally gated by a pipeline-doctor --strict step first. Covers asyncpg, cryptography, lightningcss, nextjs-swc, rollup-s390x.

Real per-repo variance parameterized (not papered over): doctor gate presence, token_direct (REGISTRY_TOKEN straight from secret vs. CI_REGISTRY_TOKEN + explicit remap), publish_style (PUBLISH=1 nix run .#publish vs. nix run .#publish-s390x -- --publish), tags_full (full clone with tags vs. shallow), and dead_secrets (some repos declare ATTIC_TOKEN/REDIS_PASSWORD but never reference them — preserved as-is, not this issue's scope to clean up).

Excluded after re-audit, not forced:

• fastuuid-s390x — declares CI_REGISTRY_TOKEN but has no export REGISTRY_TOKEN=... remap before PUBLISH=1 nix run .#publish (cryptography-s390x, the only other CI_REGISTRY_TOKEN user, does have the remap). This looks like a real latent bug — publish would run with REGISTRY_TOKEN unset. Flagged, not silently fixed via templating.
• onnxruntime-s390x — same CI_REGISTRY_TOKEN-without-remap pattern as fastuuid-s390x, and it's actually attic+redis shaped (single step) like the pypi-wheel-attic family above, not this one. Left hand-maintained.
• scikit-learn-s390x, scipy-s390x — attic+redis shaped like the pypi-wheel-attic family, but their Redis setup writes exports to a file via a heredoc that's never sourced elsewhere in the step, unlike tiktoken-s390x's direct-export style. Whether that heredoc form actually does anything meaningful isn't confirmed; normalizing it to direct exports would be a real behavior change, not a safe refactor. Left hand-maintained.

Node-pinning convention (oleks/ci-scripts#5)

Woodpecker agents advertise custom_labels of {arch, worker} (e.g. arch: arm64, worker: kotkan). Pipelines pin work to an architecture with the top-level pipeline labels: key, matched against the agent's arch label:

labels:
  arch: amd64   # or arm64

This is the one fleet-wide convention. Two other mechanisms that used to coexist are retired:

• labels: {platform: mermaid} — an older alias for arch: amd64 from before there were multiple amd64 agents. Migrated to arch: amd64.
• backend_options.kubernetes.nodeSelector: {provider: digitalocean} — a literal Kubernetes node-label selector (a different mechanism than Woodpecker agent labels entirely), left over from a single-provider cluster. Dropped: agent-level labels: already routes the pipeline to the right architecture, so pinning to a specific cloud provider's nodes on top of that is redundant and brittle if node labels change.

backend_options.kubernetes.nodeSelector with kubernetes.io/arch or kubernetes.io/hostname is still valid where a pipeline needs an actual k8s node-level constraint (e.g. sentry-cli-s390x pinning to a specific host) — that's a different, legitimate use of the mechanism and untouched here.

|| true ban (oleks/ci-scripts#6)

Per standing convention, || true is banned repo-wide — it silently swallows any failure, not just the one it was written for. The fleet's attic watch-store teardown (kill $ATTIC_PID && wait $ATTIC_PID || true) is converted to explicit exit-code handling that still tolerates the benign case (the backgrounded attic watch-store process exiting non-zero after being killed) without masking anything else:

- if ! kill $ATTIC_PID 2>/dev/null; then
    echo "attic watch-store already exited"
  fi
- if ! wait $ATTIC_PID 2>/dev/null; then
    echo "attic watch-store exited non-zero after kill (expected)"
  fi