# ci-scripts

Shared Woodpecker CI templates, plugins, and centralized pipeline conventions
for the Oleks ecosystem. This repository hosts reusable CI components (clone
steps, build templates, arch-specific runners) and tracks the cross-repo CI
optimization effort to reduce duplication across ~50 projects in ~/projects.

## pypi-wheel-attic template (oleks/ci-scripts#1)

`manifest.yaml` + `generate.py` render `.woodpecker.yaml` for the fleet of
s390x cross-build repos that share the "single build-and-publish step, inline
`attic watch-store` accelerator, tag-triggered PyPI wheel publish" shape.
Generated files carry a `GENERATED by oleks/ci-scripts — do not hand-edit`
header; edit `manifest.yaml` and re-run `python3 generate.py` instead.

`geesefs-s390x`, `sentry-cli-s390x`, `attic-client-s390x`, `angie-arm64`,
`devpi-arm64`, and `mempalace` are bespoke pipelines (OCI image publish,
binary builds, multi-file workflows) and are out of scope for either
template.

Verify with:

```sh
python3 generate.py --check   # fails if any rendered file is stale
woodpecker-cli lint <repo>/.woodpecker.yaml
```

## thin-parity-lib-publish template (oleks/ci-scripts#10)

The same `generate.py` also renders a second, genuinely different shape via
`render_thin()` + the `thin_repos:` list in `manifest.yaml`: no inline attic
accelerator, publish is a pure `nix run .#publish[-s390x]` front door,
optionally gated by a `pipeline-doctor --strict` step first. Covers
`asyncpg`, `cryptography`, `lightningcss`, `nextjs-swc`, `rollup`-s390x.

Real per-repo variance parameterized (not papered over): `doctor` gate
presence, `token_direct` (`REGISTRY_TOKEN` straight from secret vs.
`CI_REGISTRY_TOKEN` + explicit remap), `publish_style` (`PUBLISH=1 nix run
.#publish` vs. `nix run .#publish-s390x -- --publish`), `tags_full` (full
clone with tags vs. shallow), and `dead_secrets` (some repos declare
`ATTIC_TOKEN`/`REDIS_PASSWORD` but never reference them — preserved as-is,
not this issue's scope to clean up).

Excluded after re-audit, not forced:

- `fastuuid-s390x` — declares `CI_REGISTRY_TOKEN` but has **no** `export
  REGISTRY_TOKEN=...` remap before `PUBLISH=1 nix run .#publish`
  (`cryptography-s390x`, the only other `CI_REGISTRY_TOKEN` user, does have
  the remap). This looks like a real latent bug — publish would run with
  `REGISTRY_TOKEN` unset. Flagged, not silently fixed via templating.
- `onnxruntime-s390x` — same `CI_REGISTRY_TOKEN`-without-remap pattern as
  `fastuuid-s390x`, and it's actually attic+redis shaped (single step) like
  the `pypi-wheel-attic` family above, not this one. Left hand-maintained.
- `scikit-learn-s390x`, `scipy-s390x` — attic+redis shaped like the
  `pypi-wheel-attic` family, but their Redis setup writes exports to a file
  via a heredoc that's never sourced elsewhere in the step, unlike
  `tiktoken-s390x`'s direct-export style. Whether that heredoc form actually
  does anything meaningful isn't confirmed; normalizing it to direct exports
  would be a real behavior change, not a safe refactor. Left hand-maintained.

## Node-pinning convention (oleks/ci-scripts#5)

Woodpecker agents advertise `custom_labels` of `{arch, worker}` (e.g.
`arch: arm64, worker: kotkan`). Pipelines pin work to an architecture with the
top-level pipeline `labels:` key, matched against the agent's `arch` label:

```yaml
labels:
  arch: amd64   # or arm64
```

This is the one fleet-wide convention. Two other mechanisms that used to
coexist are retired:

- `labels: {platform: mermaid}` — an older alias for `arch: amd64` from before
  there were multiple amd64 agents. Migrated to `arch: amd64`.
- `backend_options.kubernetes.nodeSelector: {provider: digitalocean}` — a
  literal Kubernetes node-label selector (a different mechanism than
  Woodpecker agent labels entirely), left over from a single-provider
  cluster. Dropped: agent-level `labels:` already routes the pipeline to the
  right architecture, so pinning to a specific cloud provider's nodes on top
  of that is redundant and brittle if node labels change.

`backend_options.kubernetes.nodeSelector` with `kubernetes.io/arch` or
`kubernetes.io/hostname` is still valid where a pipeline needs an actual k8s
node-level constraint (e.g. `sentry-cli-s390x` pinning to a specific host) —
that's a different, legitimate use of the mechanism and untouched here.

## `|| true` ban (oleks/ci-scripts#6)

Per standing convention, `|| true` is banned repo-wide — it silently swallows
any failure, not just the one it was written for. The fleet's `attic
watch-store` teardown (`kill $ATTIC_PID && wait $ATTIC_PID || true`) is
converted to explicit exit-code handling that still tolerates the benign case
(the backgrounded `attic watch-store` process exiting non-zero after being
killed) without masking anything else:

```yaml
- if ! kill $ATTIC_PID 2>/dev/null; then
    echo "attic watch-store already exited"
  fi
- if ! wait $ATTIC_PID 2>/dev/null; then
    echo "attic watch-store exited non-zero after kill (expected)"
  fi
```