ci-scripts/README.md

ci-scripts

Shared Woodpecker CI templates, plugins, and centralized pipeline conventions for the Oleks ecosystem. This repository hosts reusable CI components (clone steps, build templates, arch-specific runners) and tracks the cross-repo CI optimization effort to reduce duplication across ~50 projects in ~/projects.

pypi-wheel-attic template (oleks/ci-scripts#1)

manifest.yaml + generate.py render .woodpecker.yaml for the fleet of s390x cross-build repos that share the "single build-and-publish step, inline attic watch-store accelerator, tag-triggered PyPI wheel publish" shape. Generated files carry a GENERATED by oleks/ci-scripts — do not hand-edit header; edit manifest.yaml and re-run python3 generate.py instead.

Not every s390x/arm64 repo fits this template — some (asyncpg, cryptography, fastuuid, lightningcss, nextjs-swc, rollup, onnxruntime, scikit-learn, scipy) run a genuinely different "thin parity-lib publish" pipeline with per-repo Redis sccache wiring and a pipeline-doctor gate — those differences are load-bearing, not copy-paste drift, and were deliberately left un-templated (see the scope note atop manifest.yaml). geesefs-s390x, sentry-cli-s390x, attic-client-s390x, angie-arm64, devpi-arm64, and mempalace are bespoke pipelines (OCI image publish, binary builds, multi-file workflows) and are also out of scope.

Verify with:

python3 generate.py --check   # fails if any rendered file is stale
woodpecker-cli lint <repo>/.woodpecker.yaml

Node-pinning convention (oleks/ci-scripts#5)

Woodpecker agents advertise custom_labels of {arch, worker} (e.g. arch: arm64, worker: kotkan). Pipelines pin work to an architecture with the top-level pipeline labels: key, matched against the agent's arch label:

labels:
  arch: amd64   # or arm64

This is the one fleet-wide convention. Two other mechanisms that used to coexist are retired:

• labels: {platform: mermaid} — an older alias for arch: amd64 from before there were multiple amd64 agents. Migrated to arch: amd64.
• backend_options.kubernetes.nodeSelector: {provider: digitalocean} — a literal Kubernetes node-label selector (a different mechanism than Woodpecker agent labels entirely), left over from a single-provider cluster. Dropped: agent-level labels: already routes the pipeline to the right architecture, so pinning to a specific cloud provider's nodes on top of that is redundant and brittle if node labels change.

backend_options.kubernetes.nodeSelector with kubernetes.io/arch or kubernetes.io/hostname is still valid where a pipeline needs an actual k8s node-level constraint (e.g. sentry-cli-s390x pinning to a specific host) — that's a different, legitimate use of the mechanism and untouched here.

|| true ban (oleks/ci-scripts#6)

Per standing convention, || true is banned repo-wide — it silently swallows any failure, not just the one it was written for. The fleet's attic watch-store teardown (kill $ATTIC_PID && wait $ATTIC_PID || true) is converted to explicit exit-code handling that still tolerates the benign case (the backgrounded attic watch-store process exiting non-zero after being killed) without masking anything else:

- if ! kill $ATTIC_PID 2>/dev/null; then echo "attic watch-store already exited"; fi
- if ! wait $ATTIC_PID 2>/dev/null; then echo "attic watch-store exited non-zero after kill (expected)"; fi