Fleet Publish Plugin System
1. Purpose
One versioned plugin image + org secrets + thin per-repo pipelines replace generator-rendered 60-line files. The generator manifest remains source of truth for rendering the thin files; drift detection guards them.
2. Plugin image git.oleks.space/oleks/ci-fleet-publish
Semver tags; repos pin major (:1). Base image = nix-ci toolchain.
Responsibilities baked in:
- Arch banner first (output to start each run)
- resolve.conf shim (nameserver 169.254.20.10, ndots:1)
- nix.conf shim (max-jobs, cores, sandbox settings)
- nixos-ci-entrypoint
- attic login + watch-store lifecycle with explicit non-masking shutdown (no
|| true) - error surfacing
3. Settings interface
Woodpecker settings: maps to PLUGIN_* environment variables. Available settings:
| Setting | Type | Default | Purpose |
|---|---|---|---|
run |
string (required) | – | Repo-specific command, e.g. "PUBLISH=1 nix run .#publish" |
memory |
raw YAML | – | Backend k8s resources; stays in thin file (plugins cannot set backend_options) |
cache |
bool | true |
Enable attic caching |
flow |
string | publish |
Pipeline flow: publish, buildx-build, or buildx-manifest |
image_name |
string (required) | – | OCI image name (buildx-build/buildx-manifest only). Must be explicit — never derive it from the repo/dir name in the manifest generator; a repo's actually-published image can genuinely differ (oleks/ci-scripts#22: xonsh-image's repo field vs. its live image oleks/xonsh) |
platforms |
string | – | Comma-separated platforms to probe (buildx-manifest only) |
tag_scheme |
string | {version}-{arch} |
Tag scheme, e.g. {version}-{arch} (buildx-build/buildx-manifest only) |
version |
string | – | Explicit version override. Wins over CI_COMMIT_TAG when set (nothing currently sets this, so it's a no-op for every existing consumer); falls back to the pipeline's CI_COMMIT_TAG, else "latest" (buildx-build/buildx-manifest only) |
tag_strip_v |
bool | false |
Strip a leading v and a trailing -<build-number> from the resolved version before tagging, e.g. v0.22.7-4 → 0.22.7. Opt-in per repo to reproduce a ci/local.sh that already does this normalization (oleks/ci-scripts#21); default preserves the raw tag verbatim — this plugin's own v1.0.x self-build tags rely on that default (buildx-build/buildx-manifest only) |
Deviation from the original one-value buildx-multiarch flow (oleks/ci-scripts#15): a single flow value can't tell the plugin whether this Woodpecker step is building the CURRENT node's per-arch tag or consolidating the final manifest — those are two different steps (a matrix: build step + a depends_on: build, runs_on: [success, failure] manifest step), same plugin image. Split into two explicit flow values instead:
buildx-build← buildx build+push a single per-arch tag for the current node's arch (one matrix leg per arch).buildx-manifest← probe the registry for per-arch tags and create a manifest from whatever exists.
Multi-arch resilience (#9): After per-arch builds complete, the buildx-manifest step probes the registry for which per-arch tags landed and creates a manifest from whatever exists. Fails only if no per-arch tags are found.
Mixing environment: with settings: on the same step (intentional, verified safe): real thin steps also declare environment: for token secrets (REGISTRY_TOKEN, ATTIC_TOKEN, GITEA_CLONE_TOKEN) alongside the settings: table above — see render_thin_fleet.py/render_thin_quartet.py. Woodpecker's own linter flags this combination ("Should not configure both 'environment' and 'settings'", plus a schema warning "Additional property settings is not allowed") because a step carrying both is no longer classified as a pure plugin. This is not a defect — confirmed by reading Woodpecker 3.15's own source (pipeline/frontend/yaml/compiler/convert.go's createProcess()), not just the docs: it calls settings.ParamsToEnv on container.Settings (→ PLUGIN_*) and on container.Environment (→ raw env vars) unconditionally, with no IsPlugin() gate on either call — both secrets and PLUGIN_* settings reliably reach the container regardless of the mix. The two real side effects of losing plugin classification (losing "plugin secret filter" scoping, losing privileged-mode auto-escalation) don't apply here: the DB column backing plugin-filtered secrets was dropped years ago (server/store/datastore/migration/004_remove_secrets_plugin_only_col.go), and none of these repos use privileged mode. If a future reader sees the linter warnings and assumes something's broken — it isn't, on 3.15. Tracked forward-looking cleanup (not urgent, since the linter's schema pass is only a soft warning today by explicit upstream design — // TODO: let pipelines fail if the schema is invalid): oleks/ci-scripts#24.
4. Thin pipeline contract
Repos may contain ~14 lines total:
labels:
arch: amd64 # or arm64, as needed
when:
event: tag
# ... other conditions ...
steps:
- name: publish
image: git.oleks.space/oleks/ci-fleet-publish:1
settings:
run: "PUBLISH=1 nix run .#publish"
cache: true
# ... other settings ...
backend_options: # raw YAML for k8s resources
limits:
memory: "16Gi"
Everything else (additional steps, matrix, secrets redeclaration, etc.) is forbidden and will be flagged by drift detection.
Example: lxml-s390x
labels:
arch: s390x
when:
event: tag
steps:
- name: publish
image: git.oleks.space/oleks/ci-fleet-publish:1
settings:
run: "PUBLISH=1 nix run .#publish"
cache: true
flow: publish
Example: buildx-multiarch quartet (xonsh-image style)
image_name is git.oleks.space/oleks/xonsh here, NOT .../xonsh-image —
the repo is xonsh-image but the image it has always published is xonsh
(confirmed against the live package registry; oleks/ci-scripts#22). Always
verify a repo's actual published image name before wiring image_name —
never assume it matches the repo/dir name. tag_strip_v: "true" reproduces
xonsh-image's ci/local.sh, which strips the tag's leading v before
tagging (live tags are 0.22.7-<arch>, not v0.22.7-<arch>;
oleks/ci-scripts#21).
# build.yaml (matrix over ARCH)
matrix:
ARCH: [amd64, arm64]
steps:
- name: build-and-push
image: git.oleks.space/oleks/ci-fleet-publish:1
settings:
flow: buildx-build
image_name: git.oleks.space/oleks/xonsh
tag_scheme: "{version}-{arch}"
tag_strip_v: "true"
# manifest.yaml (depends_on: build, runs_on: [success, failure])
steps:
- name: manifest
image: git.oleks.space/oleks/ci-fleet-publish:1
settings:
flow: buildx-manifest
image_name: git.oleks.space/oleks/xonsh
platforms: "amd64,arm64"
tag_scheme: "{version}-{arch}"
tag_strip_v: "true"
5. Secrets
Org-level secrets for oleks/* repos:
gitea_clone_token← Gitea token with repo read scoperegistry_token← OCI registry token (canonical env name:REGISTRY_TOKEN)attic_token← Attic authentication token
Per-repo secrets are deprecated. If existing secret values cannot be recovered (Woodpecker secrets are write-only), mint fresh Gitea tokens with equivalent scopes (see issue #11).
6. Versioning & rollout
Plugin changes ship as new semver tags. Breaking changes to the settings interface bump major version. Repos pinned to :1 pick up patch updates automatically.
v1.0.4 added the version/tag_strip_v settings (oleks/ci-scripts#21) and
tightened image_name to be required with no inference (oleks/ci-scripts#22)
— both additive, default-preserving changes; no consumer needs to change
anything to stay on :1.
Rollout order:
- Fleet repos (15 repos)
- Quartet repos (6 repos)
7. Enforcement & s390x No-Build Policy
The drift-detection pipeline (issue #13) re-renders manifests from the generator, diffs against repo HEAD, and opens issues on any drift from the thin pipeline contract.
s390x repos are no-build: repos in the building/s390x/* manifest entries remain no-build while s390x hardware is unavailable. The generator renders neutralized .woodpecker.yaml files (when: event: manual, alpine echo step explaining why) so tag pushes never trigger builds. The plugin will not be called for s390x repos until no_build: false is set in manifest.yaml. arm64 repos are unaffected and continue to build normally.
See also
- Spec/Modern-CI-Target ← Overall Modern CI target design