71 lines
3.4 KiB
Markdown
71 lines
3.4 KiB
Markdown
# ci-scripts
|
|
|
|
Shared Woodpecker CI templates, plugins, and centralized pipeline conventions for the Oleks ecosystem. This repository hosts reusable CI components (clone steps, build templates, arch-specific runners) and tracks the cross-repo CI optimization effort to reduce duplication across ~50 projects in ~/projects.
|
|
|
|
## pypi-wheel-attic template (oleks/ci-scripts#1)
|
|
|
|
`manifest.yaml` + `generate.py` render `.woodpecker.yaml` for the fleet of
|
|
s390x cross-build repos that share the "single build-and-publish step, inline
|
|
`attic watch-store` accelerator, tag-triggered PyPI wheel publish" shape.
|
|
Generated files carry a `GENERATED by oleks/ci-scripts — do not hand-edit`
|
|
header; edit `manifest.yaml` and re-run `python3 generate.py` instead.
|
|
|
|
Not every s390x/arm64 repo fits this template — some (`asyncpg`,
|
|
`cryptography`, `fastuuid`, `lightningcss`, `nextjs-swc`, `rollup`,
|
|
`onnxruntime`, `scikit-learn`, `scipy`) run a genuinely different "thin
|
|
parity-lib publish" pipeline with per-repo Redis sccache wiring and a
|
|
pipeline-doctor gate — those differences are load-bearing, not copy-paste
|
|
drift, and were deliberately left un-templated (see the scope note atop
|
|
`manifest.yaml`). `geesefs-s390x`, `sentry-cli-s390x`, `attic-client-s390x`,
|
|
`angie-arm64`, `devpi-arm64`, and `mempalace` are bespoke pipelines (OCI image
|
|
publish, binary builds, multi-file workflows) and are also out of scope.
|
|
|
|
Verify with:
|
|
|
|
```
|
|
python3 generate.py --check # fails if any rendered file is stale
|
|
woodpecker-cli lint <repo>/.woodpecker.yaml
|
|
```
|
|
|
|
## Node-pinning convention (oleks/ci-scripts#5)
|
|
|
|
Woodpecker agents advertise `custom_labels` of `{arch, worker}` (e.g.
|
|
`arch: arm64, worker: kotkan`). Pipelines pin work to an architecture with the
|
|
top-level pipeline `labels:` key, matched against the agent's `arch` label:
|
|
|
|
```yaml
|
|
labels:
|
|
arch: amd64 # or arm64
|
|
```
|
|
|
|
This is the one fleet-wide convention. Two other mechanisms that used to
|
|
coexist are retired:
|
|
|
|
- `labels: {platform: mermaid}` — an older alias for `arch: amd64` from before
|
|
there were multiple amd64 agents. Migrated to `arch: amd64`.
|
|
- `backend_options.kubernetes.nodeSelector: {provider: digitalocean}` — a
|
|
literal Kubernetes node-label selector (a different mechanism than
|
|
Woodpecker agent labels entirely), left over from a single-provider
|
|
cluster. Dropped: agent-level `labels:` already routes the pipeline to the
|
|
right architecture, so pinning to a specific cloud provider's nodes on top
|
|
of that is redundant and brittle if node labels change.
|
|
|
|
`backend_options.kubernetes.nodeSelector` with `kubernetes.io/arch` or
|
|
`kubernetes.io/hostname` is still valid where a pipeline needs an actual k8s
|
|
node-level constraint (e.g. `sentry-cli-s390x` pinning to a specific host) —
|
|
that's a different, legitimate use of the mechanism and untouched here.
|
|
|
|
## `|| true` ban (oleks/ci-scripts#6)
|
|
|
|
Per standing convention, `|| true` is banned repo-wide — it silently swallows
|
|
any failure, not just the one it was written for. The fleet's `attic
|
|
watch-store` teardown (`kill $ATTIC_PID && wait $ATTIC_PID || true`) is
|
|
converted to explicit exit-code handling that still tolerates the benign case
|
|
(the backgrounded `attic watch-store` process exiting non-zero after being
|
|
killed) without masking anything else:
|
|
|
|
```yaml
|
|
- if ! kill $ATTIC_PID 2>/dev/null; then echo "attic watch-store already exited"; fi
|
|
- if ! wait $ATTIC_PID 2>/dev/null; then echo "attic watch-store exited non-zero after kill (expected)"; fi
|
|
```
|