oleks/ci-scripts
1
0
Fork 0
Code Issues 8 Pull Requests Actions Packages Projects Releases Wiki Activity
Page: Spec/Modern-CI-Target
Pages
Spec/Current-CI-Ecosystem Spec/Fleet Publish Plugin System Spec/Modern-CI-Target
Clone
2
Spec/Modern-CI-Target
oleks edited this page 2026-07-08 21:11:12 +03:00
Table of Contents

Goals

  1. One source of truth enforced (not manual): every repo's CI registration and pipeline shape live in a canonical registry, backed by a verification job that catches drift.

  2. Fewer pipeline shapes ← converged DSL: reduce 6+ custom YAML patterns down to ~2 (one for Nix+Attic builds, one for container multi-arch).

  3. No copy-paste ← secrets standardized, one token name per kind, every repo audited for registration + secret presence.

  4. Reproducible transitions ← phases are sequential; each phase closes a GitHub issue and is verifiable before the next begins.

Target Architecture (By Phase)

Phase 1: EPIC #8 (DONE 2026-07-04)

Baseline generator + fleet conventions established. Closed issues:

  • #1 Consolidate s390x/arm64 cross-build fleet into one shared template
  • #2 Extract shared buildx multi-arch quartet (amd64/arm64/manifest/chart)
  • #3 Shared clone step definition
  • #4 Centralize registry/host constants
  • #5 Unify node-pinning label keys
  • #6 Audit and remove copied || true failure-masking
  • #7 Split monolithic step scripts + banner gaps
  • #9 Manifest step must not be skipped on partial arch success
  • #10 Second fleet template for the parity-lib publish family

Artifacts: manifest.yaml + generate.py + render.py, 15 templated repos, arch-banner + no || true + node-pinning conventions.

Phase 2: Enforcement & Hygiene

Deliverables:

  1. Drift detection pipeline ← scheduled job or per-push step in oleks/ci-scripts:

    • Re-render all manifests locally.
    • Diff against each repo's HEAD:.woodpecker.yaml.
    • Open/update issues on drift (title: "ci-scripts drift detected", body: diff). Auto-close when diff clears.
    • Prevents silent hand-edits and regeneration skips.

  2. Org-level Woodpecker secrets ← migrate per-repo copies of registry_token, attic_token, gitea_clone_token to org scope. Verify every repo that references a secret actually has it (no 404s from undefined secrets at runtime).

  3. Token naming fix ← standardize on one token name. Currently:

    • registry_token (correct, org secret).
    • CI_REGISTRY_TOKEN (per-repo env) ↦ REGISTRY_TOKEN (shell export, latent bug on fastuuid/onnyxruntime).
    • Decision required: rename all to registry_token org secret, or rename the secret to ci_registry_token and update all templates?

  4. CI registration audit ← verify every repo that should have a Woodpecker pipeline is actually registered (no 404s like oleks/woodpecker-peek #13). Link repos in manifest to Woodpecker API /api/v1/repos/<owner>/<name> and confirm at least one pipeline is present.

  5. s390x repo shape migration note ← s390x repos will migrate their .woodpecker.yaml pipeline SHAPE for consistency with the fleet (via #8 Phase 1), but they remain no-build: the generator renders neutralized pipelines (when: event: manual, alpine echo) so tag pushes never trigger builds. The #16 fleet migration gate must not auto-enable s390x builds while hardware is unavailable. Policy enforced in generator commit bf36991.

Success criteria: All registered repos have matching drift-free .woodpecker.yaml files, all secrets resolve, zero undefined-secret runtime errors.

Phase 3: Shape Convergence

Goal: Replace generated YAML bulk with thin parameterized pipelines calling versioned shared code.

Strategy:

  1. OCI image bundle ← package ci-scripts releases as a versioned Woodpecker plugin image or standalone script bundle (nix flake app):

    • Encapsulates build ↦ push ↦ manifest-consolidate logic (buildx matrix/split/single, registry probe, digest pinning).
    • Per-repo pipeline becomes ~10 lines: clone, banner, nix run git.oleks.space/oleks/ci-scripts#build-and-publish -- --config <per-repo-json>.
    • Centralizes registry-probe + manifest logic (no per-repo duplication).

  2. Parity-lib convergence ← fold flat nix builds onto shared oleks/parity-lib helpers (precedent: foldImageLayers, shared flake-module for per-arch image builds). Reduce buildx templates to a single pluggable shape.

  3. Target result: 3 pipeline shapes → 1 parameterized shape for Nix+container builds. Bespoke repos (quoted in "Not Covered") remain exempt but documented.

Success criteria: Every templated repo's .woodpecker.yaml is <100 lines, most logic is version-pinned calls to published scripts/images, generator still verifies correctness but doesn't emit bulk YAML.

Phase 4: Fold Bespoke Tail

Goal: Migrate or explicitly declare permanent-bespoke repos.

Case-by-case audit:

  • iia-researcher, commonground-legacy (amd64-only Dockerfile) ← add arm64 legs via buildx plugin or declare amd64-only in manifest + skip arm64 CI entirely?
  • fastuuid-s390x, onnyxruntime-s390x (token remap bug) ← template them into thin-publish once token bug is understood?
  • scikit-learn-s390x, scipy-s390x (heredoc Redis) ← normalize Redis setup to direct export and fold into thin-publish?
  • builder-arbitrage (cosign + digest-pinned) ← keep as bespoke (true multi-step orchestration), document rationale.
  • nixos-ci, iia-agent, terminal-agent, mermaid-gpu-dashboard ← stay bespoke (architectural uniqueness), link to justification docs.

Result: A final "permanent bespoke" list in the manifest with reasons for each (and links to the issues/docs justifying them).

Non-Goals

  • No move off Woodpecker ← this is a Woodpecker optimization, not a migration to GitHub Actions / other CI.
  • No forcing bespoke toolchains prematurely ← nix2container, cosign, moby-buildkit stay in repos that own them; don't fold them into shared templates until stable.

Invariants (Must Survive Transition)

  • Arch banner ← every step's first line echo "▸ arch=$(uname -m)".
  • No || true ← explicit exit-code handling everywhere.
  • Arch-only labels ← labels: {arch: amd64|arm64|s390x-native} is the sole node-pinning knob; no nodeSelector or platform aliases.
  • Parse-time ${} ← braced variables are parse-phase (Woodpecker substitution); shell uses bare $VAR or $${ (escaped brace). Discipline enforced by linter or guide.
  • Manifest-on-partial-success ← multi-arch workflows must not skip manifest consolidation if one arch fails (registry probe gates it).
  • GENERATED headers ← wherever .woodpecker.yaml files are still rendered (even in Phase 3, if thin pipelines call generated sub-pipelines), mark them.

Open Questions

  1. Plugin image vs script bundle ← should Phase 3 wrap shared logic as a Woodpecker plugin (docker image, image: entry) or a Nix script/flake app (lower footprint, less duplication with the parent flake)?

  2. Drift-detection venue ← scheduled job in Woodpecker? Per-push hook in oleks/ci-scripts? Separate repo (oleks/ci-drift-detector)? Trade-offs: visibility, CPU budget, GitHub issue spam.

  3. Org secrets rollout order ← Phase 2 migration: do all tokens at once, or registry_token first (highest impact, lowest risk)?

  4. Bespoke redeclaration ← Phase 4: in manifest.yaml as a bespoke: section, or a separate .bespoke.md file in the repo?

See also