Audit ci/local.sh before migrating matrix quartet repos (xonsh-image, csi-s3) to the plugin buildx flow #20

New Issue

2026-07-09T19:12:49+03:00

oleks commented

2026-07-09 19:12:49 +03:00

Child of oleks/ci-scripts#17. Blocking safety check before the matrix-kind migration ships.

The risk (flagged in generator/render_thin_quartet.py's own header): migrating xonsh-image and csi-s3 to the plugin's buildx-build / buildx-manifest flows means the plugin's generic docker buildx build --push <context> (building the repo's Dockerfile directly) replaces whatever ci/local.sh --arch / --manifest currently does internally.

If ci/local.sh does anything beyond a plain Dockerfile build+push — custom build args, extra tags, pre/post steps, non-default context or Dockerfile path, cache wiring, digest pinning, cosign — the thin pipeline would silently drop it and break working CI.

Unlike the fleet family (whose migration just moves inline shell into settings.run:), this is a real behavioral swap.

Work:

Read ci/local.sh in xonsh-image and csi-s3 end to end. Enumerate everything each does in --arch and --manifest modes.
Compare against what the plugin's buildx-build and buildx-manifest flows actually do (see entrypoint.sh in this repo; note --provenance=false --sbom=false, and that buildx-manifest probes the registry for landed per-arch tags per oleks/ci-scripts#9).
Verdict per repo: equivalent (safe to migrate as-is), needs plugin settings (name which: image_name / platforms / tag_scheme / build args), or too bespoke (exclude from #17 with reason).
If a gap exists, either extend the plugin's settings interface (and bump ci-fleet-publish) or exclude the repo — do not paper over it.

Acceptance: a written per-repo verdict comment on this issue with evidence (quoted ci/local.sh behavior vs plugin behavior). No repo pushes and no plugin changes in this issue — audit + recommendation only.

Do not touch s390x repos (no-build).

**Child of oleks/ci-scripts#17.** Blocking safety check before the `matrix`-kind migration ships. **The risk** (flagged in `generator/render_thin_quartet.py`'s own header): migrating xonsh-image and csi-s3 to the plugin's `buildx-build` / `buildx-manifest` flows means the plugin's **generic** `docker buildx build --push <context>` (building the repo's Dockerfile directly) **replaces** whatever `ci/local.sh --arch` / `--manifest` currently does internally. If `ci/local.sh` does anything beyond a plain Dockerfile build+push — custom build args, extra tags, pre/post steps, non-default context or Dockerfile path, cache wiring, digest pinning, cosign — the thin pipeline would **silently drop it** and break working CI. Unlike the fleet family (whose migration just moves inline shell into `settings.run:`), this is a real behavioral swap. **Work:** - [ ] Read `ci/local.sh` in `xonsh-image` and `csi-s3` end to end. Enumerate everything each does in `--arch` and `--manifest` modes. - [ ] Compare against what the plugin's `buildx-build` and `buildx-manifest` flows actually do (see `entrypoint.sh` in this repo; note `--provenance=false --sbom=false`, and that `buildx-manifest` probes the registry for landed per-arch tags per oleks/ci-scripts#9). - [ ] Verdict per repo: **equivalent** (safe to migrate as-is), **needs plugin settings** (name which: image_name / platforms / tag_scheme / build args), or **too bespoke** (exclude from #17 with reason). - [ ] If a gap exists, either extend the plugin's settings interface (and bump `ci-fleet-publish`) or exclude the repo — do not paper over it. **Acceptance:** a written per-repo verdict comment on this issue with evidence (quoted `ci/local.sh` behavior vs plugin behavior). **No repo pushes and no plugin changes in this issue** — audit + recommendation only. **Do not** touch s390x repos (no-build).

oleks added this to the Modern CI Phase 2+3 milestone 2026-07-09 19:12:49 +03:00

oleks referenced this issue

2026-07-09 19:13:07 +03:00

Migrate fleet repos (15) to thin pipelines #16

repo-worker commented

2026-07-09 19:18:41 +03:00

Audit result: `ci/local.sh` vs plugin `buildx-build`/`buildx-manifest`

Read end-to-end: xonsh-image/ci/local.sh, csi-s3/ci/local.sh, both repos' .woodpecker/*.yaml, entrypoint.sh, generator/render.py (current thick generator), generator/render_thin_quartet.py (thin generator), and manifests/{xonsh-image,csi-s3}.yaml. Cross-checked against the live Gitea package registry and Woodpecker pipeline history.

xonsh-image (gitea repo `oleks/xonsh`) — needs plugin settings + one plugin gap that must close first

image_name mismatch (confirmed live). render_thin_quartet.py derives image_name = f"git.oleks.space/oleks/{repo}" from the manifest's repo: xonsh-image field → oleks/xonsh-image. But ci/local.sh's default is:

IMAGE_REPO="${IMAGE_REPO:-xonsh}"
...
IMAGE="${REGISTRY_HOST}/${REGISTRY_OWNER}/${IMAGE_REPO}"

The live gitea remote is git.oleks.space/oleks/xonsh.git, and the package registry confirms the actually-published image is oleks/xonsh (tags 0.22.7, 0.22.7-amd64, 0.22.7-arm64, latest — verified via package_read list_versions, id 4310-4315, published by pipeline #9 which succeeded). Migrating with the generator's current derivation would silently build+push to a different, currently-empty image path (oleks/xonsh-image) while nothing updates the real one. Fix: the manifest/generator must set image_name: git.oleks.space/oleks/xonsh explicitly — the plugin's settings.image_name already supports an override, this is a generator bug, not a plugin gap.

Version/tag-format gap the plugin cannot currently express. ci/local.sh's derive_version() strips the leading v (and a trailing -N build suffix) from $CI_COMMIT_TAG:

t="${t#v}"; t="${t%-[0-9]*}"

Live tags on this repo are v0.22.7 (confirmed via list_tags), and the live published tags are 0.22.7-amd64 / 0.22.7-arm64 / 0.22.7 — without the v. The plugin's buildx_build_flow/buildx_manifest_flow in entrypoint.sh do:

local version="${CI_COMMIT_TAG:-${PLUGIN_VERSION:-latest}}"

CI_COMMIT_TAG always wins over PLUGIN_VERSION when the pipeline is tag-triggered (which it always is here — when: event: tag), so there is no existing setting to reproduce the strip. Migrating as-is would rename every future release tag from 0.22.7-amd64 to v0.22.7-amd64, a live tag-format break, not a cosmetic one. This needs an entrypoint.sh change (e.g. strip v + trailing -[0-9]+ by default, or a PLUGIN_VERSION-wins toggle) before migrating — out of scope for this read-only audit per the issue's constraints.

Everything else matches: no build-args, no clone override, no custom timeout, no skip-if-published logic; context is the repo root in both (matches plugin default context: .); tag_scheme: "{version}-{arch}" matches; manifest dest_tags default ($version,latest) matches once the version gap above is closed.

Verdict: needs plugin settings (image_name) AND a version-stripping capability the plugin doesn't have yet — do not migrate until entrypoint.sh gains it.

csi-s3 (gitea repo `oleks/csi-s3`) — too bespoke, exclude from #17

Multiple behaviors in ci/local.sh have no expression at all in the current buildx-build/buildx-manifest flows (not settings gaps — missing mechanisms):

Build-args: build_arch() does
```
--build-arg "GEESEFS_VERSION=${GEESEFS_VERSION}"
--build-arg "REGISTRY_TOKEN=${REGISTRY_TOKEN}"
```
(Dockerfile uses REGISTRY_TOKEN to fetch the s390x geesefs binary from the Gitea generic registry). entrypoint.sh's buildx_build_flow has zero --build-arg passthrough. GEESEFS_VERSION happens to equal the Dockerfile's ARG default (0.43.5) today so this is silently harmless right now, but it's a real, principled gap (and s390x re-enabling would break outright — dormant per the repo's no-s390x-build policy, not absent).
Dual per-arch tags in one build: build_arch() tags both ${IMAGE}:${TAG}-${arch} and ${IMAGE}:latest-${arch} in the same docker buildx build invocation. The plugin's buildx-build flow renders exactly one tag per invocation (tag_scheme). The final combined :TAG/:latest manifests come out equivalent via buildx-manifest's default dest_tags, but the intermediate :latest-${arch} tags would stop being published.
Skip-if-already-published idempotency: image_tag_exists() queries the Gitea package API and skips the build if the per-arch tag already exists — "Mirrors the original pipeline's 'skip if the per-arch tag exists' check." The plugin always rebuilds/re-pushes unconditionally; no equivalent setting exists.
Explicit clone override + custom timeout, dropped by the generator itself, not just the plugin. manifests/csi-s3.yaml sets explicit_clone: true (→ custom clone step: PLUGIN_TAGS: "false", PLUGIN_DEPTH: "1", a dedicated gitea_clone_token secret) and timeout: 240. These are live, wired fields consumed by generator/render.py's render_matrix_explicit_clone() — but render_thin_quartet.py's render_build/render_manifest emit no clone: block and no timeout: field at all. Migrating via the thin generator as currently written would silently drop both, reverting to Woodpecker's default clone (fetches tags, no depth limit) and default timeout.
Fixed vs per-arch label: the current build.yaml pins labels: arch: amd64 for the whole pipeline file even though it matrices over amd64/arm64 (fixed_label: amd64 in the manifest) — Woodpecker agent selection is fixed, only the in-pod target arch varies via TARGET_ARCH/nodeSelector. render_thin_quartet.py's render_build emits labels: arch: {arch} per leg (varies with the matrix), which is a different Woodpecker agent-selection input, not just cosmetic.
No CI run history exists for this repo yet (pipeline_list returns empty) and no csi-s3-driver/csi-s3 container package exists in the registry — the --publish path is unproven in production, unlike xonsh-image where the live registry state confirms the exact drift.

Verdict: too bespoke for #17 as currently scoped. Closing items 1-4 would require several new plugin settings (build_args, clone overrides, timeout, an extra-tag scheme) plus generator changes to actually emit them — at that point it's a feature-request queue against ci-fleet-publish, not a like-for-like migration. Recommend excluding csi-s3 from #17 and tracking the needed plugin extensions separately if the migration is still wanted later.

No repo pushes, no plugin changes made — audit + recommendation only, per the acceptance criteria.

## Audit result: `ci/local.sh` vs plugin `buildx-build`/`buildx-manifest` Read end-to-end: `xonsh-image/ci/local.sh`, `csi-s3/ci/local.sh`, both repos' `.woodpecker/*.yaml`, `entrypoint.sh`, `generator/render.py` (current thick generator), `generator/render_thin_quartet.py` (thin generator), and `manifests/{xonsh-image,csi-s3}.yaml`. Cross-checked against the live Gitea package registry and Woodpecker pipeline history. --- ### xonsh-image (gitea repo `oleks/xonsh`) — **needs plugin settings + one plugin gap that must close first** **image_name mismatch (confirmed live).** `render_thin_quartet.py` derives `image_name = f"git.oleks.space/oleks/{repo}"` from the manifest's `repo: xonsh-image` field → `oleks/xonsh-image`. But `ci/local.sh`'s default is: ``` IMAGE_REPO="${IMAGE_REPO:-xonsh}" ... IMAGE="${REGISTRY_HOST}/${REGISTRY_OWNER}/${IMAGE_REPO}" ``` The live gitea remote is `git.oleks.space/oleks/xonsh.git`, and the package registry confirms the actually-published image is `oleks/xonsh` (tags `0.22.7`, `0.22.7-amd64`, `0.22.7-arm64`, `latest` — verified via `package_read list_versions`, id 4310-4315, published by pipeline #9 which succeeded). Migrating with the generator's current derivation would silently build+push to a *different, currently-empty* image path (`oleks/xonsh-image`) while nothing updates the real one. **Fix:** the manifest/generator must set `image_name: git.oleks.space/oleks/xonsh` explicitly — the plugin's `settings.image_name` already supports an override, this is a generator bug, not a plugin gap. **Version/tag-format gap the plugin cannot currently express.** `ci/local.sh`'s `derive_version()` strips the leading `v` (and a trailing `-N` build suffix) from `$CI_COMMIT_TAG`: ``` t="${t#v}"; t="${t%-[0-9]*}" ``` Live tags on this repo are `v0.22.7` (confirmed via `list_tags`), and the live published tags are `0.22.7-amd64` / `0.22.7-arm64` / `0.22.7` — **without** the `v`. The plugin's `buildx_build_flow`/`buildx_manifest_flow` in `entrypoint.sh` do: ``` local version="${CI_COMMIT_TAG:-${PLUGIN_VERSION:-latest}}" ``` `CI_COMMIT_TAG` always wins over `PLUGIN_VERSION` when the pipeline is tag-triggered (which it always is here — `when: event: tag`), so there is **no existing setting** to reproduce the strip. Migrating as-is would rename every future release tag from `0.22.7-amd64` to `v0.22.7-amd64`, a live tag-format break, not a cosmetic one. This needs an `entrypoint.sh` change (e.g. strip `v` + trailing `-[0-9]+` by default, or a `PLUGIN_VERSION`-wins toggle) before migrating — out of scope for this read-only audit per the issue's constraints. Everything else matches: no build-args, no clone override, no custom timeout, no skip-if-published logic; context is the repo root in both (matches plugin default `context: .`); `tag_scheme: "{version}-{arch}"` matches; manifest `dest_tags` default (`$version,latest`) matches once the version gap above is closed. **Verdict: needs plugin settings (image_name) AND a version-stripping capability the plugin doesn't have yet — do not migrate until entrypoint.sh gains it.** --- ### csi-s3 (gitea repo `oleks/csi-s3`) — **too bespoke, exclude from #17** Multiple behaviors in `ci/local.sh` have no expression at all in the current `buildx-build`/`buildx-manifest` flows (not settings gaps — missing mechanisms): 1. **Build-args**: `build_arch()` does ``` --build-arg "GEESEFS_VERSION=${GEESEFS_VERSION}" --build-arg "REGISTRY_TOKEN=${REGISTRY_TOKEN}" ``` (Dockerfile uses `REGISTRY_TOKEN` to fetch the s390x geesefs binary from the Gitea generic registry). `entrypoint.sh`'s `buildx_build_flow` has **zero** `--build-arg` passthrough. `GEESEFS_VERSION` happens to equal the Dockerfile's `ARG` default (`0.43.5`) today so this is silently harmless *right now*, but it's a real, principled gap (and s390x re-enabling would break outright — dormant per the repo's no-s390x-build policy, not absent). 2. **Dual per-arch tags in one build**: `build_arch()` tags both `${IMAGE}:${TAG}-${arch}` and `${IMAGE}:latest-${arch}` in the *same* `docker buildx build` invocation. The plugin's `buildx-build` flow renders exactly one tag per invocation (`tag_scheme`). The final combined `:TAG`/`:latest` manifests come out equivalent via `buildx-manifest`'s default `dest_tags`, but the intermediate `:latest-${arch}` tags would stop being published. 3. **Skip-if-already-published idempotency**: `image_tag_exists()` queries the Gitea package API and skips the build if the per-arch tag already exists — "Mirrors the original pipeline's 'skip if the per-arch tag exists' check." The plugin always rebuilds/re-pushes unconditionally; no equivalent setting exists. 4. **Explicit clone override + custom timeout, dropped by the generator itself, not just the plugin.** `manifests/csi-s3.yaml` sets `explicit_clone: true` (→ custom clone step: `PLUGIN_TAGS: "false"`, `PLUGIN_DEPTH: "1"`, a dedicated `gitea_clone_token` secret) and `timeout: 240`. These are live, wired fields consumed by `generator/render.py`'s `render_matrix_explicit_clone()` — but `render_thin_quartet.py`'s `render_build`/`render_manifest` emit no `clone:` block and no `timeout:` field at all. Migrating via the thin generator as currently written would silently drop both, reverting to Woodpecker's default clone (fetches tags, no depth limit) and default timeout. 5. **Fixed vs per-arch label**: the current `build.yaml` pins `labels: arch: amd64` for the *whole* pipeline file even though it matrices over `amd64`/`arm64` (`fixed_label: amd64` in the manifest) — Woodpecker agent selection is fixed, only the in-pod target arch varies via `TARGET_ARCH`/`nodeSelector`. `render_thin_quartet.py`'s `render_build` emits `labels: arch: {arch}` per leg (varies with the matrix), which is a different Woodpecker agent-selection input, not just cosmetic. 6. No CI run history exists for this repo yet (`pipeline_list` returns empty) and no `csi-s3-driver`/`csi-s3` container package exists in the registry — the `--publish` path is unproven in production, unlike xonsh-image where the live registry state confirms the exact drift. **Verdict: too bespoke for #17 as currently scoped.** Closing items 1-4 would require several new plugin settings (`build_args`, clone overrides, `timeout`, an extra-tag scheme) plus generator changes to actually emit them — at that point it's a feature-request queue against `ci-fleet-publish`, not a like-for-like migration. Recommend excluding csi-s3 from #17 and tracking the needed plugin extensions separately if the migration is still wanted later. --- No repo pushes, no plugin changes made — audit + recommendation only, per the acceptance criteria.

issuer-agent referenced this issue

2026-07-09 19:20:47 +03:00

ci-fleet-publish: buildx flows can't reproduce ci/local.sh's tag normalization #21

oleks referenced this issue

2026-07-09 19:21:24 +03:00

Migrate quartet repos (6) to thin multi-arch pipelines #17

admin referenced this issue

2026-07-09 19:21:26 +03:00

render_thin_quartet.py derives wrong image_name (uses manifest repo field) #22

repo-worker referenced this issue

2026-07-09 19:34:30 +03:00

Thin-render logic for single/split quartet repos (the-eye, emdash, flake-hub, woodpecker-peek) #19