From de9800f6113c997472a12ea199ca243255f0f1bd Mon Sep 17 00:00:00 2001 From: repo-worker Date: Sun, 5 Jul 2026 16:49:35 +0300 Subject: [PATCH] Split buildx-multiarch into buildx-build/buildx-manifest flows (oleks/ci-scripts#15 implementation) --- Spec%2FFleet-Publish-Plugin-System.md | 133 +++++++++++++++++++++++++- 1 file changed, 132 insertions(+), 1 deletion(-) diff --git a/Spec%2FFleet-Publish-Plugin-System.md b/Spec%2FFleet-Publish-Plugin-System.md index 8747b1a..40a5af4 100644 --- a/Spec%2FFleet-Publish-Plugin-System.md +++ b/Spec%2FFleet-Publish-Plugin-System.md @@ -1 +1,132 @@ -"$(cat /home/oleks/.claude/jobs/55b2c067/tmp/spec.md)" \ No newline at end of file +# Fleet Publish Plugin System + +## 1. Purpose + +One versioned plugin image + org secrets + thin per-repo pipelines replace generator-rendered 60-line files. The generator manifest remains source of truth for rendering the thin files; drift detection guards them. + +## 2. Plugin image `git.oleks.space/oleks/ci-fleet-publish` + +Semver tags; repos pin major (`:1`). Base image = nix-ci toolchain image. + +Responsibilities baked in: +- Arch banner first (output to start each run) +- resolv.conf shim (nameserver 169.254.20.10, ndots:1) +- nix.conf shim (max-jobs, cores, sandbox settings) +- nixos-ci-entrypoint +- attic login + watch-store lifecycle with explicit non-masking shutdown (no `|| true`) +- error surfacing + +## 3. Settings interface + +Woodpecker `settings:` maps to `PLUGIN_*` environment variables. Available settings: + +| Setting | Type | Default | Purpose | +|---------|------|---------|---------| +| `run` | string (required) | — | Repo-specific command, e.g. `"PUBLISH=1 nix run .#publish"` | +| `memory` | raw YAML | — | Backend k8s resources; stays in thin file (plugins cannot set backend_options) | +| `cache` | bool | `true` | Enable attic caching | +| `flow` | string | `publish` | Pipeline flow: `publish`, `buildx-build`, or `buildx-manifest` | +| `image_name` | string | — | OCI image name (`buildx-build`/`buildx-manifest` only) | +| `platforms` | string | — | Comma-separated platforms to probe (`buildx-manifest` only) | +| `tag_scheme` | string | `{version}-{arch}` | Tag scheme, e.g. `{version}-{arch}` (`buildx-build`/`buildx-manifest` only) | + +**Deviation from the original one-value `buildx-multiarch` flow (oleks/ci-scripts#15):** a single flow value can't tell the plugin whether this Woodpecker step is building the CURRENT node's per-arch tag or consolidating the final manifest — those are two different steps (a `matrix:` build step + a `depends_on: build, runs_on: [success, failure]` manifest step), same plugin image. Split into two explicit flow values instead: +- `buildx-build` — buildx build+push a single per-arch tag for the current node's arch (one matrix leg per arch). +- `buildx-manifest` — probe the registry for per-arch tags and create a manifest from whatever exists. + +**Multi-arch resilience (#9):** After per-arch builds complete, the `buildx-manifest` step probes the registry for which per-arch tags landed and creates a manifest from whatever exists. Fails only if no per-arch tags are found. + +## 4. Thin pipeline contract + +Repos may contain ~14 lines total: + +```yaml +labels: + arch: amd64 # or arm64, as needed + +when: + event: tag + # ... other conditions ... + +steps: + - name: publish + image: git.oleks.space/oleks/ci-fleet-publish:1 + settings: + run: "PUBLISH=1 nix run .#publish" + cache: true + # ... other settings ... + backend_options: # raw YAML for k8s resources + limits: + memory: "16Gi" +``` + +Everything else (additional steps, matrix, secrets redeclaration, etc.) is **forbidden** and will be flagged by drift detection. + +**Example: lxml-s390x** + +```yaml +labels: + arch: s390x + +when: + event: tag + +steps: + - name: publish + image: git.oleks.space/oleks/ci-fleet-publish:1 + settings: + run: "PUBLISH=1 nix run .#publish" + cache: true + flow: publish +``` + +**Example: buildx-multiarch quartet (xonsh-image style)** + +```yaml +# build.yaml (matrix over ARCH) +matrix: + ARCH: [amd64, arm64] +steps: + - name: build-and-push + image: git.oleks.space/oleks/ci-fleet-publish:1 + settings: + flow: buildx-build + image_name: git.oleks.space/oleks/xonsh-image + tag_scheme: "{version}-{arch}" + +# manifest.yaml (depends_on: build, runs_on: [success, failure]) +steps: + - name: manifest + image: git.oleks.space/oleks/ci-fleet-publish:1 + settings: + flow: buildx-manifest + image_name: git.oleks.space/oleks/xonsh-image + platforms: "amd64,arm64" + tag_scheme: "{version}-{arch}" +``` + +## 5. Secrets + +Org-level secrets for `oleks/*` repos: + +- `gitea_clone_token` — Gitea token with repo read scope +- `registry_token` — OCI registry token (canonical env name: `REGISTRY_TOKEN`) +- `attic_token` — Attic authentication token + +Per-repo secrets are **deprecated**. If existing secret values cannot be recovered (Woodpecker secrets are write-only), mint fresh Gitea tokens with equivalent scopes (see issue #11). + +## 6. Versioning & rollout + +Plugin changes ship as new semver tags. Breaking changes to the settings interface bump major version. Repos pinned to `:1` pick up patch updates automatically. + +Rollout order: +1. Fleet repos (15 repos) +2. Quartet repos (6 repos) + +## 7. Enforcement + +The drift-detection pipeline (issue #13) re-renders manifests from the generator, diffs against repo HEAD, and opens issues on any drift from the thin pipeline contract. + +## See also + +- [Spec/Modern-CI-Target](wiki/Spec/Modern-CI-Target) — Overall Modern CI target design