Thin-render logic for single/split quartet repos (the-eye, emdash, flake-hub, woodpecker-peek) #19
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: oleks/ci-scripts#19
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Child of oleks/ci-scripts#17. The staged
generator/render_thin_quartet.pyonly handleskind: matrixrepos — it explicitly skips everything else (if manifest.get("kind") != "matrix": continue). So 4 of the 6 quartet repos have no thin-render implementation.Actual shapes (from
manifests/*.yaml):nix run .#publish-*, no docker buildxWork: implement thin-render for the
singleandsplitkinds. These do not use buildx, so they map to the plugin'sflow: publish(the same shapegenerator/render_thin_fleet.pyrenders for the fleet family) — one thin step per pipeline file,image: git.oleks.space/oleks/ci-fleet-publish:v1,settings: {run: <the repo's existing publish command>, flow: publish, cache: <attic on/off>},backend_optionsmemory kept raw.Acceptance:
render_thin_quartet.py(or a sibling) renderssingleandsplitkinds, not justmatrix..woodpeckerfile(s): env/secrets, commands,when/labels,backend_options, per-arch file structure forsplit. Nothing the current pipeline does may be silently dropped.splitrepos must keep their per-arch file layout (one thin file per arch), not be collapsed.Do not touch s390x repos (no-build, oleks/fleet policy) — the fleet family is 100% s390x and out of scope.
generator/render_thin_quartet.pyextended to rendersingleandsplitkinds, alongside the already-donematrixkind. Commit38bb59e(main). Rendered all 4 target repos to a scratch dir and diffed against each repo's current.woodpecker/files — no repo pushes made, per the issue's constraint.Mapping used (mirrors
render_thin_fleet.py'sflow: publishshape): one thin step per current step/leg,image: git.oleks.space/oleks/ci-fleet-publish:v1,settings: {run: <original commands, minus the arch-banner line>, flow: publish, cache: <"true" if ATTIC_TOKEN is used, else "false">}.backend_options(nodeSelector/resources/labels) kept byte-identical.Changes common to all 4 (documented in the script's docstring, not silent):
image:swaps from the repo-specificnix-ci:latest[-arm64]tag to the plugin's single multi-archci-fleet-publish:v1tag — same conventionrender_thin_fleet.pyalready established for the fleet family.echo "▸ arch=$(uname -m)"command is dropped fromrun:—entrypoint.shalready prints that banner unconditionally before any flow runs, so keeping it would double it.clone:override (PLUGIN_TAGS: "false",PLUGIN_DEPTH: "1") is dropped, relying on Woodpecker's implicit clone — same precedent already set for the fleet family. All 4 repos use the plain shallow/no-tags shape (none hit thetags_fullgaprender_thin_fleet.pyflagged as unresolved), so this isn't a new risk class.when:/branch triggers, orbackend_optionswere dropped or altered — diffs below arerun:/commands:restructuring only.Per-repo diffs (full diffs available on request, this is the substantive part of each):
the-eye (
container.yaml, 2 steps:build-and-push,build-and-push-grpc) —commands: [echo, ./ci/local.sh --arch arm64]→settings: {run: "./ci/local.sh --arch arm64", flow: publish, cache: "false"}(same for the--component grpcstep).REGISTRY_TOKEN/BUILDKIT_ADDR/PUBLISH/BRANCH/VERSIONenv vars,backend_options(nodeSelector arm64, memory 4Gi/4Gi, commit-branch/commit-sha/pipeline-number labels), andwhen: push branch:[develop,staging,production]all preserved exactly. Note:release.yaml(goreleaser on tag) andtest.yaml(go/js tests on push) are NOT manifest-driven (not inmanifests/the-eye.yaml) — out of scope, untouched by this render, still hand-maintained.emdash.kotkanagrilli.fi (
container.yaml, 1 step) — identical shape to the-eye'sbuild-and-push, no grpc step (matches its manifest's single-componentcomponents:list).flake-hub (
amd64.yaml,arm64.yaml, kept as separate per-arch files — not collapsed) —commands: [echo, sh ci/setup.sh, PUBLISH=1 nix run .#publish[-aarch64-linux]]→settings: {run: "sh ci/setup.sh\n...\nPUBLISH=1 nix run .#publish[-aarch64-linux]", flow: publish, cache: "true"}(no-- --publishflag, matching manifest'spublish_flag: falsefor both legs).ATTIC_TOKEN/GITEA_CLONE_TOKENenv vars,backend_options(nodeSelector, commit-tag/commit-branch/pipeline-number labels, no resources block — matches source, which never had one), andwhen: push branch:mainonly (no tag trigger, matching manifest'swhen_tag_event: false) all preserved exactly.woodpecker-peek (
amd64.yaml,arm64.yaml, kept separate) — same shape as flake-hub, but WITH the-- --publishflag (manifest doesn't setpublish_flag, defaults true) and WITH a bare- event: tagtrigger alongsidepush branch:main(manifest doesn't setwhen_tag_event, defaults true). Both correctly differentiated from flake-hub by reading the manifest'spublish_flag/when_tag_eventper-leg rather than hardcoding.Verified: all 6 rendered files (4 target + xonsh-image/csi-s3 matrix, unaffected) parse as valid YAML, and output is byte-identical before/after the repo's ruff pre-commit auto-format pass.
Not closing this issue — migration/review of the actual repo pushes is a separate step per the acceptance criteria.
Follow-up per team-lead's request: verified the
clone:override drop with evidence, not precedent, for each of the 4 repos.Woodpecker's actual implicit-clone defaults (from
woodpecker-ci/plugin-gitdocs):depthdefaults to none/partial-clone (fetches full commit graph, blobs filtered — a superset of adepth: 1shallow clone, not a subset),tagsdefaults to false, except on a tag-triggered run, where tags ARE fetched. The explicit override all 4 repos currently carry isPLUGIN_TAGS: "false"/PLUGIN_DEPTH: "1"unconditionally.Do any of the 4 need git tags or full history? Grepped the actual build/publish targets:
ci/local.sh): nogit describe/git tag/git rev-listanywhere. The onegitcall isgit rev-parse --abbrev-ref HEADas a fallback for$BRANCH— dead code in CI, sinceBRANCH: "${CI_COMMIT_BRANCH}"is always set explicitly as an env var, so the fallback branch never executes.VERSIONcomes from$CI_PIPELINE_NUMBER, not git. No git-history dependency at all.nix run .#publish[-<arch>]): traced through toparity-lib'smkAtticClosurePublish(lib/builders.nix:1047) — resolves$ATTIC_TOKEN, doesnix buildon a fixed list of local package names,attic push. Noself.rev,self.shortRev,git describe, or any git-metadata read anywhere in either flake.nix or builders.nix. woodpecker-peek's own package version isnixpkgs.lib.fileContents ./VERSION(a checked-in file, not git tags). No git-tag/history dependency in either repo's publish path.Verdict per repo, with the superset/subset test you asked for:
clone:.Field-coverage check (item 2): grepped
manifests/*.yamlforexplicit_clone/timeout— onlymanifests/csi-s3.yamlcarries either (explicit_clone: true,timeout: 240). None of the 4 single/split manifests in my scope (the-eye, emdash.kotkanagrilli.fi, flake-hub, woodpecker-peek) have these fields, and none of their current committed.woodpecker/*.yamlhave atimeout:key either. The generator defect audit-worker found on #20 is a matrix-kind-only gap (csi-s3) — it doesn't touch any of the 4 repos in this issue's scope.No code changes made as a result of this check — the evidence supports the drop as-is for all 4. Flagging one open question for team-lead: does the woodpecker-peek tag-event tag-fetch widening (inert but not identical) warrant pinning
clone:back explicitly for that one repo for full parity, or is "superset, no consumer" sufficient? I'm holding off editingrender_thin_quartet.pyeither way since it currently has live uncommitted changes in the shared working tree from audit-worker's #20 fix — will coordinate before touching it.Still not closing #19.
Decision recorded (team-lead, following the evidence in the previous comment): leave woodpecker-peek's
clone:drop as-is — no explicit pin.Reasoning: the tag-event tag-fetch widening is a superset (more refs available on a tag-triggered run) with no consumer in the publish path (confirmed:
parity-lib'smkAtticClosurePublishnever reads git tags), so it cannot break the build — and if a future change ever starts reading tags, it would find them present, not missing, which is the safe failure direction. Also noted: forcingPLUGIN_TAGS: "false"on a tag-triggered run was the stranger state to begin with; dropping it moves woodpecker-peek toward Woodpecker's own default semantics rather than away from them.This becomes a watch item on woodpecker-peek's first canary tag run — we verify a real green tag-triggered build before rolling it out further. No code change needed;
clone:stays dropped for all 4 repos as rendered.