oleks/parity-lib
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: shared per-archetype parity publish-app builders (v0.1.0)

Browse Source 
Implements the shared parity flake-module library so the ~51 parity repos
consume one source of truth instead of hand-inlined publish shells.

- lib.mk{PyPiWheel,S390xNpm,GenericBinary,Nix2Container,GoBinary,Helm}Publish
  builders returning stage-<arch>/publish-<arch>/publish-index/publish/
  push-staged apps per the corrected emmett#44 standard (build-parity stages to
  ./.parity-stage with no registry contact; publish dry-runs by default;
  publish-index is build-free + fail-closed; :latest is the last digest copy).
- Shared ci/parity-lib.sh: token resolution ($REGISTRY_TOKEN + pass fallback,
  never printed), dev-tag guard, version derivation, dry-run gate, preflight.
- pipeline-doctor package/app asserting the parity contract (cluster #193).

Refs cluster #192, #193, #194, emmett#44.
This commit is contained in:
Oleks
2026-06-02 04:15:48 +03:00
parent 310848ecb6
commit 2201257e89
8 changed files with 1443 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+3
View File
@@ -0,0 +1,3 @@
.parity-stage/
result
result-*
CHANGELOG.md
+31
View File
@@ -0,0 +1,31 @@
# Changelog
<!-- markdownlint-disable MD013 -->
All notable changes to parity-lib are documented here. This project follows
semantic versioning; the version is a conceptual tag (no git tag is created).
## v0.1.0
Initial release (cluster #192/#193/#194, emmett#44).
- `lib.mkParityBuilders pkgs` plus per-builder wrappers exposing the six
archetype publish-app builders:
- `mkPyPiWheelPublish` — single-arch Gitea PyPI wheel.
- `mkS390xNpmPublish` — single-arch Gitea npm native addon.
- `mkGenericBinaryPublish` — single-arch Gitea generic-registry binary.
- `mkGoBinaryPublish` — alias of `mkGenericBinaryPublish` (explicit archetype).
- `mkNix2ContainerPublish` — multi-arch OCI image with `publish-index` and
`:latest` digest copy.
- `mkHelmPublish` — Helm chart to an OCI registry.
- Each builder returns flake apps following the corrected parity standard:
`stage-<arch>` (build-parity, no registry), `publish-<arch>` (dry-run by
default), `publish-index` (build-free, fail-closed multi-arch assembly via
regctl), `publish` (all local arches + index + `:latest` last), and
`push-staged` (replay `./.parity-stage`).
- Shared shell library `ci/parity-lib.sh` (token resolution with
`$REGISTRY_TOKEN` + `pass` fallback and never printed, dev-tag guard, version
derivation, the dry-run gate, registry preflight, stage-dir helpers).
- `packages.pipeline-doctor` / `apps.pipeline-doctor` (cluster #193): static
parity-contract checker that prints local-equivalent commands.
- `flake.lock` fully pinned; nixpkgs follows the shared `fleet-pins` `nixpkgs-ci`.
README.md
+180 -1
View File
@@ -1,3 +1,182 @@
# parity-lib # parity-lib
Shared versioned flake-module for local pipeline parity (emmett#44, cluster #192): mkPyPiWheelPublish / mkS390xNpmPublish / mkGenericBinaryPublish / mkNix2ContainerPublish + pipeline-doctor. Single source of truth so per-repo publish apps cannot drift. <!-- markdownlint-disable MD013 MD040 -->
Shared per-archetype **publish-app builders** for the ~51 "parity" repos
(cluster #192/#193/#194, emmett#44). Instead of each repo hand-inlining its own
`stage` / `publish` / `push-staged` shell, every repo consumes ONE versioned
source of truth: this flake's `lib.mk*Publish` builders generate the flake apps
that implement the corrected parity standard.
## The corrected parity standard (emmett#44)
Parity has two halves, and only one of them can be cluster-independent:
- **BUILD/STAGE parity is cluster-independent.** `nix run .#stage-<arch>`
cross-builds the artifact into a local on-disk store (`./.parity-stage/<arch>`
by convention) and makes **NO registry contact**, so it runs identically on
emmett even when armer / the registry is down.
- **PUBLISH parity shares fate with the cluster** (the only registry is
co-located with it). The publish apps name the blocker (`registry-down`) up
front and never half-push.
Every push entrypoint **DRY-RUNS by default** — it stages and prints what it
WOULD push. You must pass `--publish` (or set `PUBLISH=1`) to mutate the
registry, so an accidental local run can never push.
## App shape each builder emits
| app | parity half | registry | mutates? |
| --- | --- | --- | --- |
| `stage-<arch>` | BUILD | none | no |
| `publish-<arch>` | BUILD + PUBLISH | one arch | only with `--publish` |
| `publish-index` | PUBLISH | multi-arch assembly (regctl) | only with `--publish` |
| `publish` | BUILD + PUBLISH | all local arches + index + `:latest` | only with `--publish` |
| `push-staged` | PUBLISH | replay `.parity-stage` | only with `--publish` |
- `publish-index` is **build-free**: it assembles `<image>:<TAG>` from the
per-arch digest-pinned tags via `regctl index create`, and is **fail-closed** —
if a required arch was not pushed this run, it refuses to assemble a partial
index.
- `publish` runs all locally-buildable arches, then the index, then copies
`:<TAG>` → `:latest` as the **LAST** (single, idempotent) mutation.
- `push-staged` replays artifacts from `./.parity-stage` to the registry, for
when the cluster was down at build time.
Single-arch archetypes (PyPI wheel, npm addon, generic/Go binary, Helm chart)
have no multi-arch index, so they expose `stage-<arch>` / `publish-<arch>` /
`publish` / `push-staged` only. The nix2container (OCI) builder is the one that
yields the full `publish-index` / `:latest` set.
## Shared building blocks (also exposed)
All builders source one shell library (`ci/parity-lib.sh`, materialized in the
Nix store) so behavior cannot drift between repos:
- **Token resolution** — `$REGISTRY_TOKEN` → `pass infra/gitea/...` fallback →
named hard-fail. The token is **never printed** and scripts run under
`set -euo pipefail` only (never `set -x`).
- **Dev-tag guard** — refuses a real (`:latest`/release) publish unless
`$VERSION` is set or `$CI_COMMIT_TAG` is a `v*` tag.
- **Version derivation** — `$VERSION` → strip leading `v` + trailing `-N` from
`$CI_COMMIT_TAG` → the flake's pinned default. Identical for CI and local.
## API — `lib.*`
`lib` is system-independent. Two ways to consume it:
```nix
# (a) one call, all builders, with your own pkgs:
parity.lib.mkParityBuilders pkgs # -> { mkPyPiWheelPublish, ... }
# (b) per-builder wrapper that takes pkgs as the first argument:
parity.lib.mkPyPiWheelPublish pkgs { pname = "..."; version = "..."; ... }
```
Exposed attrs:
- `lib.mkParityBuilders` — `pkgs -> { the six builders + mkApp + shellLib }`
- `lib.mkPyPiWheelPublish` — `pkgs -> args -> { apps }`
- `lib.mkS390xNpmPublish` — `pkgs -> args -> { apps }`
- `lib.mkGenericBinaryPublish` — `pkgs -> args -> { apps }`
- `lib.mkNix2ContainerPublish` — `pkgs -> args -> { apps }`
- `lib.mkGoBinaryPublish` — `pkgs -> args -> { apps }` (alias of generic-binary)
- `lib.mkHelmPublish` — `pkgs -> args -> { apps }`
### Builder arguments
```nix
mkPyPiWheelPublish {
pname = "asyncpg";
version = "0.31.0"; # default; overridden by $VERSION / $CI_COMMIT_TAG
wheel = self.packages.x86_64-linux.default; # drv/dir to glob *.whl, or a .whl path
arch = "s390x"; # default
}
mkS390xNpmPublish {
pname = "@rollup/rollup-linux-s390x-gnu";
version = "4.0.0";
nodeFile = self.packages.x86_64-linux.addon; # the *.node
nodeFileName = "rollup.linux-s390x-gnu.node";
packageJson = ''{ "name": "@rollup/...", "version": "$VERSION", ... }'';
}
mkGenericBinaryPublish { # mkGoBinaryPublish is an alias
pname = "geesefs";
version = "0.43.5";
binary = "${self.packages.x86_64-linux.default}/bin/geesefs";
assetName = "geesefs-linux-s390x";
arch = "s390x";
}
mkNix2ContainerPublish {
imageName = "git.oleks.space/oleks/nix-ci";
version = "1.2.3";
images = { # only arches buildable on THIS host
amd64 = { copyTo = self.packages.x86_64-linux.nix-ci.copyTo; };
};
arches = [ "amd64" "arm64" ]; # the full set the index must cover (fail-closed)
}
mkHelmPublish {
pname = "firecrawl";
version = "0.1.3";
chartSrc = ./charts/firecrawl;
ociRepo = "oci://git.oleks.space/oleks"; # default
}
```
Common optional args on every builder: `registryHost` (`git.oleks.space`),
`registryOwner` (`oleks`), `passEntry`
(`infra/gitea/personal_access_token_packages_rw`).
### Consuming it in a parity repo's flake
```nix
{
inputs.parity.url = "git+https://git.oleks.space/oleks/parity-lib";
outputs = { self, nixpkgs, parity, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let pkgs = import nixpkgs { inherit system; };
in {
apps = parity.lib.mkGenericBinaryPublish pkgs {
pname = "geesefs";
version = "0.43.5";
binary = "${self.packages.${system}.default}/bin/geesefs";
assetName = "geesefs-linux-s390x";
};
});
}
```
`.woodpecker.yaml` stays thin — it runs the identical app, so CI and local
cannot drift:
```yaml
commands:
- PUBLISH=1 nix run .#publish
```
## pipeline-doctor (cluster #193)
`nix run .#pipeline-doctor -- <repo-path>` asserts the parity contract on a
repo: archetype declared, consumes parity-lib, token =
`$REGISTRY_TOKEN` + `pass` fallback and never printed, dev-tag guard present, a
`--dry-run` default exists, apps carry `meta.description`, and an enumerable
`publish-*` / `stage-*` naming. It prints the local-equivalent commands. It is
read-only (no token, no registry contact) and exits non-zero if any required
check fails.
## Verifying changes locally
```bash
nix eval .#lib --apply builtins.attrNames --json
nix flake show
nix build .#pipeline-doctor
shellcheck ci/*.sh
statix check .
markdownlint-cli2 README.md
```
The git.oleks.space server runs a pre-receive linter; the commands above mirror
it.
ci/parity-lib.sh
+149
View File
@@ -0,0 +1,149 @@
# shellcheck shell=bash
# shellcheck disable=SC2034 # vars/functions here are consumed by sourcing apps
#
# parity-lib shared shell helpers (cluster #192, emmett#44).
#
# This file is the single source of truth for the cross-cutting concerns every
# parity publish app shares: token resolution, version derivation, the dev-tag
# guard, argument parsing (the dry-run gate), registry preflight and the
# on-disk stage directory convention. Each archetype builder in lib/builders.nix
# generates a small writeShellApplication that sources THIS file and then does
# only its archetype-specific build + push.
#
# TOKEN HYGIENE: nothing here ever echoes the token. Scripts that source this
# MUST run under `set -euo pipefail` only and MUST NOT enable `set -x`.
# ---------------------------------------------------------------------------
# Conventions (overridable by the sourcing app before calling the helpers).
# ---------------------------------------------------------------------------
PARITY_PASS_ENTRY="${PARITY_PASS_ENTRY:-infra/gitea/personal_access_token_packages_rw}"
PARITY_REGISTRY_HOST="${PARITY_REGISTRY_HOST:-git.oleks.space}"
PARITY_REGISTRY_OWNER="${PARITY_REGISTRY_OWNER:-oleks}"
# Local on-disk stage. BUILD-parity artifacts land here; push-staged replays them.
PARITY_STAGE_DIR="${PARITY_STAGE_DIR:-${PWD}/.parity-stage}"
# ---------------------------------------------------------------------------
# Token resolution: $REGISTRY_TOKEN -> `pass` fallback -> named hard fail.
# Prints the token on stdout so a caller can `tok="$(parity_resolve_token)"`.
# The CALLER must capture it and never echo it. This function itself is silent.
# ---------------------------------------------------------------------------
parity_resolve_token() {
if [ -n "${REGISTRY_TOKEN:-}" ]; then
printf '%s' "$REGISTRY_TOKEN"
return 0
fi
if command -v pass >/dev/null 2>&1; then
local t
if t="$(pass show "$PARITY_PASS_ENTRY" 2>/dev/null || true)" && [ -n "$t" ]; then
printf '%s' "$t"
return 0
fi
fi
echo "BLOCKER(empty-token): set \$REGISTRY_TOKEN (CI from_secret) or store '$PARITY_PASS_ENTRY' in pass; refusing to publish without credentials." >&2
return 1
}
# ---------------------------------------------------------------------------
# Version derivation, identical for CI and local.
# Precedence: $VERSION -> strip-v + strip-trailing-increment of $CI_COMMIT_TAG
# -> the default passed as $1 (the flake's pinned version).
# Tag shape accepted: v<x.y.z> optionally with a -<N> build increment suffix.
# Prints the derived version on stdout.
# ---------------------------------------------------------------------------
parity_derive_version() {
local default_version="${1:-}"
if [ -n "${VERSION:-}" ]; then
printf '%s' "$VERSION"
return 0
fi
if [ -n "${CI_COMMIT_TAG:-}" ]; then
printf '%s' "$CI_COMMIT_TAG" | sed 's/^v//; s/-[0-9]*$//'
return 0
fi
if [ -n "$default_version" ]; then
printf '%s' "$default_version"
return 0
fi
echo "BLOCKER(no-version): no \$VERSION, no \$CI_COMMIT_TAG and no default version." >&2
return 1
}
# ---------------------------------------------------------------------------
# Dev-tag guard. Refuse a real (registry-mutating, :latest/release) publish
# unless an explicit $VERSION is set OR $CI_COMMIT_TAG looks like a v* tag.
# This stops an accidental local `--publish` from clobbering the registry with
# the flake's default development version.
# ---------------------------------------------------------------------------
parity_devtag_guard() {
if [ -n "${VERSION:-}" ]; then
return 0
fi
if printf '%s' "${CI_COMMIT_TAG:-}" | grep -Eq '^v[0-9]'; then
return 0
fi
echo "BLOCKER(dev-tag): refusing to publish without an explicit version." >&2
echo " Set VERSION=<x.y.z> or push a v* tag (CI_COMMIT_TAG) before publishing." >&2
return 1
}
# ---------------------------------------------------------------------------
# Argument gate. Sets the global PARITY_PUBLISH (0 = dry-run default, 1 = push).
# Honors $PUBLISH=1, --publish and --dry-run. --help prints usage and exits 0.
# Usage: parity_parse_args "<app-description>" "$@"
# ---------------------------------------------------------------------------
parity_parse_args() {
local desc="$1"
shift
PARITY_PUBLISH="${PUBLISH:-0}"
local a
for a in "$@"; do
case "$a" in
--publish) PARITY_PUBLISH=1 ;;
--dry-run) PARITY_PUBLISH=0 ;;
-h | --help)
echo "$desc"
echo ""
echo "Dry-run by default: builds/stages and prints what WOULD be pushed."
echo " --publish | PUBLISH=1 actually mutate the registry"
echo " --dry-run force dry-run"
echo "Env: VERSION=<x.y.z> override version; REGISTRY_TOKEN registry token"
echo " (fallback: pass $PARITY_PASS_ENTRY)."
exit 0
;;
*)
echo "error: unknown argument '$a' (try --help)" >&2
exit 2
;;
esac
done
}
# ---------------------------------------------------------------------------
# Registry preflight: name the blocker if the registry is unreachable BEFORE
# attempting any push (CI shares fate with the cluster the registry lives on).
# ---------------------------------------------------------------------------
parity_registry_preflight() {
if ! curl -fsS -o /dev/null --max-time 10 "https://${PARITY_REGISTRY_HOST}/api/v1/version" 2>/dev/null; then
echo "BLOCKER(registry-down): https://${PARITY_REGISTRY_HOST} unreachable — re-run when the registry is back; staged artifacts in ${PARITY_STAGE_DIR} are unchanged." >&2
return 1
fi
}
# ---------------------------------------------------------------------------
# Stage directory helpers. The convention: each staged artifact is written to
# ${PARITY_STAGE_DIR}/<arch>/ alongside a one-line meta file recording the
# version + intended registry coordinates, so push-staged can replay later.
# ---------------------------------------------------------------------------
parity_stage_path() {
# $1 = arch
printf '%s/%s' "$PARITY_STAGE_DIR" "$1"
}
parity_stage_reset() {
# $1 = arch ; wipes + recreates that arch's stage dir, prints the path.
local d
d="$(parity_stage_path "$1")"
rm -rf "$d"
mkdir -p "$d"
printf '%s' "$d"
}
ci/pipeline-doctor.sh
Executable
+131
View File
@@ -0,0 +1,131 @@
#!/usr/bin/env bash
# pipeline-doctor (cluster #193): assert the parity contract for a repo.
#
# Given a repo path, it statically checks that the repo follows the corrected
# emmett#44 parity standard, and prints the local-equivalent commands a dev can
# run. It is read-only: it never touches the registry and never needs a token.
#
# Usage: pipeline-doctor [<repo-path>] (default: .)
# Exit: 0 = all required checks pass; 1 = one or more required checks failed.
set -euo pipefail
REPO="${1:-.}"
REPO="$(cd "$REPO" && pwd)"
FLAKE="$REPO/flake.nix"
fail=0
pass_n=0
note() { printf ' %s %s\n' "$1" "$2"; }
ok() {
pass_n=$((pass_n + 1))
note "PASS" "$1"
}
bad() {
fail=$((fail + 1))
note "FAIL" "$1"
}
warn() { note "WARN" "$1"; }
echo "pipeline-doctor: $REPO"
if [ ! -f "$FLAKE" ]; then
echo "FAIL: no flake.nix at $REPO" >&2
exit 1
fi
flake_txt="$(cat "$FLAKE")"
# 1. archetype declared (a publish/stage app whose name encodes the arch, or an
# explicit archetype marker comment).
if printf '%s' "$flake_txt" | grep -Eq 'archetype|stage-[a-z0-9_]+|publish-[a-z0-9_]+'; then
ok "archetype declared (stage-*/publish-* app or archetype marker present)"
else
bad "no archetype: expected a stage-<arch>/publish-<arch> app or 'archetype' marker"
fi
# 2. consumes parity-lib (the shared module).
if printf '%s' "$flake_txt" | grep -Eq 'parity-lib|parity\.lib|mk(PyPiWheel|S390xNpm|GenericBinary|Nix2Container|GoBinary|Helm)Publish'; then
ok "consumes parity-lib (input or one of the mk*Publish builders)"
else
bad "does not consume parity-lib: inline the input and use a mk*Publish builder"
fi
# 3. token = $REGISTRY_TOKEN with pass fallback, and never printed.
token_src="$flake_txt"
if [ -d "$REPO/ci" ]; then
token_src="$token_src
$(cat "$REPO"/ci/*.sh 2>/dev/null || true)"
fi
if printf '%s' "$token_src" | grep -Eq 'REGISTRY_TOKEN' &&
printf '%s' "$token_src" | grep -Eq 'parity_resolve_token|pass (show )?infra/gitea/personal_access_token_packages_rw'; then
ok "token = \$REGISTRY_TOKEN with pass fallback"
else
bad "token contract missing: need \$REGISTRY_TOKEN + pass fallback (parity_resolve_token)"
fi
# token never printed: flag an obvious leak. A redacting pipe (sed s/$tok/.../)
# is fine, so only flag a bare echo/printf of the token NOT piped to a redactor,
# or an enabled `set -x` (which would trace the token). The leak pattern uses a
# literal '$' built via a variable so this lib stays single-quote clean (SC2016).
dollar='[$]'
# Anchor on echo/printf used as a COMMAND (line-leading, after `;`, `|`, `&&`,
# `(` or `then/do/else`) so the doctor doesn't flag the regex literals it
# carries inside string assignments.
leak_pat="(^|[;|&(]|then |do |else )[[:space:]]*(echo|printf)[^|]*${dollar}(REGISTRY_TOKEN|TOKEN|tok|token)([^A-Za-z0-9_]|$)"
leak=0
if printf '%s' "$token_src" | grep -Eq "^[[:space:]]*set -x([[:space:]]|$)"; then leak=1; fi
# Exclude: redactions, stdin-feeds, the doctor's own $token_src var, an escaped
# '\$' (a token NAME in a message, not its value), and the sanctioned
# `printf '%s' "$TOKEN"` capture idiom (the resolver returns the token on stdout
# for a caller to capture — that is the one blessed place a token is emitted).
if printf '%s' "$token_src" |
grep -E "$leak_pat" |
grep -vE "REDACTED|sed |stdin|token_src" |
grep -vqE 'printf .%s. "[$](REGISTRY_TOKEN|TOKEN|tok|token)"|\\[$]'; then
leak=1
fi
if [ "$leak" -eq 0 ]; then
ok "no obvious token leak (token never bare-echoed; no set -x)"
else
bad "possible token leak: a token var is echo/printf'd un-redacted, or set -x is enabled"
fi
# 4. dev-tag guard present.
if printf '%s' "$token_src" | grep -Eq 'parity_devtag_guard|refusing to publish without an explicit'; then
ok "dev-tag guard present"
else
bad "no dev-tag guard: a default-version --publish could clobber the registry"
fi
# 5. a --dry-run default exists.
if printf '%s' "$token_src" | grep -Eq 'parity_parse_args|DRY-RUN|dry-run|PUBLISH:-0|PUBLISH=\$\{PUBLISH:-0\}'; then
ok "dry-run default exists (publish mutates only on --publish/PUBLISH=1)"
else
bad "no dry-run default: publish must NOT mutate the registry by default"
fi
# 6. apps carry meta.description.
if printf '%s' "$flake_txt" | grep -Eq 'meta\.description|meta = \{'; then
ok "apps carry meta.description"
else
bad "apps missing meta.description"
fi
# 7. enumerable publish-* naming.
pubs="$(printf '%s' "$flake_txt" | grep -oE '(stage|publish)(-[a-z0-9_]+)?' | sort -u || true)"
if [ -n "$pubs" ]; then
ok "enumerable publish-* / stage-* app naming:"
printf '%s\n' "$pubs" | sed 's/^/ /'
else
bad "no enumerable publish-*/stage-* apps found"
fi
echo ""
echo "Local-equivalent commands (what CI runs is identical):"
echo " nix run .#stage-<arch> # BUILD-parity: stage to .parity-stage, no registry"
echo " nix run .#publish-<arch> # stage + push that arch (DRY-RUN; add --publish)"
echo " nix run .#publish-index # build-free multi-arch assembly from pushed digests"
echo " nix run .#publish # all local arches + index; :latest = digest copy of :TAG"
echo " nix run .#push-staged # replay .parity-stage to the registry (cluster-was-down)"
echo ""
echo "Summary: $pass_n passed, $fail failed."
[ "$fail" -eq 0 ]
flake.lock
Generated
+119
View File
@@ -0,0 +1,119 @@
{
"nodes": {
"flake-utils": {
"inputs": {
"systems": "systems"
},
"locked": {
"lastModified": 1731533236,
"narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "11707dc2f618dd54ca8739b309ec4fc024de578b",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"fleet": {
"inputs": {
"nixpkgs": "nixpkgs",
"nixpkgs-armer": [
"fleet",
"nixpkgs"
],
"nixpkgs-bim": [
"fleet",
"nixpkgs"
],
"nixpkgs-ci": [
"fleet",
"nixpkgs"
],
"nixpkgs-emmett": [
"fleet",
"nixpkgs"
],
"nixpkgs-howard": [
"fleet",
"nixpkgs"
],
"nixpkgs-mermaid": [
"fleet",
"nixpkgs"
],
"nixpkgs-mermaid-gpu": [
"fleet",
"nixpkgs"
],
"nixpkgs-micron": [
"fleet",
"nixpkgs"
],
"nixpkgs-projects": [
"fleet",
"nixpkgs"
]
},
"locked": {
"lastModified": 1779533061,
"narHash": "sha256-orWNYXtYURhEj3X4+xGMAhaEcKRvwXqTtJ8x2jV/M+Q=",
"ref": "refs/heads/main",
"rev": "b818e345ec4470e4b3e335bd2f864183c512116d",
"revCount": 13,
"type": "git",
"url": "https://git.oleks.space/oleks/fleet-pins"
},
"original": {
"type": "git",
"url": "https://git.oleks.space/oleks/fleet-pins"
}
},
"nixpkgs": {
"locked": {
"lastModified": 1777268161,
"narHash": "sha256-bxrdOn8SCOv8tN4JbTF/TXq7kjo9ag4M+C8yzzIRYbE=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "1c3fe55ad329cbcb28471bb30f05c9827f724c76",
"type": "github"
},
"original": {
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "1c3fe55ad329cbcb28471bb30f05c9827f724c76",
"type": "github"
}
},
"root": {
"inputs": {
"flake-utils": "flake-utils",
"fleet": "fleet",
"nixpkgs": [
"fleet",
"nixpkgs-ci"
]
}
},
"systems": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
}
},
"root": "root",
"version": 7
}
flake.nix
+88
View File
@@ -0,0 +1,88 @@
{
description = "parity-lib — shared per-archetype publish-app builders for the ~51 parity repos (cluster #192/#193/#194, emmett#44)";
inputs = {
# Shared fleet pin so the ~51 consumers stay binary-cache aligned.
fleet.url = "git+https://git.oleks.space/oleks/fleet-pins";
nixpkgs.follows = "fleet/nixpkgs-ci";
flake-utils.url = "github:numtide/flake-utils";
};
outputs =
{
self,
nixpkgs,
flake-utils,
...
}:
{
# ---------------------------------------------------------------------
# lib.* — system-INDEPENDENT entry points. A consumer calls
# parity.lib.mkParityBuilders { pkgs = <its own pkgs>; }
# and gets the six mk*Publish builders back, OR uses the per-builder
# convenience wrappers below which take pkgs as the first argument.
# ---------------------------------------------------------------------
lib =
let
builders = pkgs: import ./lib/builders.nix { inherit pkgs; };
wrap = name: pkgs: args: (builders pkgs).${name} args;
in
{
mkParityBuilders = builders;
mkPyPiWheelPublish = wrap "mkPyPiWheelPublish";
mkS390xNpmPublish = wrap "mkS390xNpmPublish";
mkGenericBinaryPublish = wrap "mkGenericBinaryPublish";
mkNix2ContainerPublish = wrap "mkNix2ContainerPublish";
mkGoBinaryPublish = wrap "mkGoBinaryPublish";
mkHelmPublish = wrap "mkHelmPublish";
};
}
// flake-utils.lib.eachDefaultSystem (
system:
let
pkgs = import nixpkgs { inherit system; };
inherit (pkgs) lib;
pipelineDoctor = pkgs.writeShellApplication {
name = "pipeline-doctor";
runtimeInputs = with pkgs; [
coreutils
gnugrep
gnused
findutils
jq
];
text = ''exec bash ${./ci/pipeline-doctor.sh} "$@"'';
};
# Smoke check: instantiate every builder with stub args so the apps eval.
builders = import ./lib/builders.nix { inherit pkgs; };
smoke = pkgs.runCommand "parity-lib-smoke" { } ''
: "${
builtins.toString (
lib.attrNames (builders.mkPyPiWheelPublish {
pname = "demo";
version = "0.0.0";
wheel = pkgs.hello;
})
)
}"
touch $out
'';
in
{
packages = {
pipeline-doctor = pipelineDoctor;
default = pipelineDoctor;
};
apps.pipeline-doctor = {
type = "app";
program = lib.getExe pipelineDoctor;
meta.description = "Assert the parity contract for a repo and print local-equivalent commands (cluster #193).";
};
checks.smoke = smoke;
}
);
}
lib/builders.nix
+742
View File
@@ -0,0 +1,742 @@
# Per-archetype publish-app BUILDERS (cluster #192/#194, emmett#44).
#
# `mkParityBuilders { pkgs }` returns the six mk*Publish builders plus the
# shared building blocks. Each mk*Publish takes a small attrset and returns an
# attrset of flake apps following the corrected parity standard:
#
# stage-<arch> BUILD-parity, cluster-independent: writes the artifact to the
# on-disk stage (./.parity-stage/<arch>), NO registry contact.
# publish-<arch> stage + push that arch. DRY-RUN by default (--publish to push).
# publish-index build-free multi-arch assembly from digest-pinned refs
# (regctl), fail-closed if a required arch was not pushed.
# publish all locally-buildable arches + publish-index; ':latest' is a
# digest copy of ':TAG' done LAST as the idempotent mutation.
# push-staged replay artifacts from ./.parity-stage to the registry.
#
# Not every archetype yields every app: a single-arch wheel/binary/npm addon
# has no multi-arch index, so it exposes stage/publish/push-staged only. The OCI
# (nix2container) builder is the one that yields the full index/latest set.
{ pkgs }:
let
inherit (pkgs) lib;
shellLib = ../ci/parity-lib.sh;
# Common preamble: source the shared shell helpers from the store and apply
# any per-app coordinate overrides BEFORE the helpers are used.
preamble =
{
registryHost ? "git.oleks.space",
registryOwner ? "oleks",
passEntry ? "infra/gitea/personal_access_token_packages_rw",
}:
''
set -euo pipefail
# shellcheck source=/dev/null
. ${shellLib}
# Coordinate overrides are read by the sourced lib; export so shellcheck
# treats them as used (SC2034) and child processes inherit them.
export PARITY_REGISTRY_HOST=${lib.escapeShellArg registryHost}
export PARITY_REGISTRY_OWNER=${lib.escapeShellArg registryOwner}
export PARITY_PASS_ENTRY=${lib.escapeShellArg passEntry}
export PARITY_STAGE_DIR="''${PARITY_STAGE_DIR:-''${PWD}/.parity-stage}"
'';
mkApp = drv: desc: {
type = "app";
program = lib.getExe drv;
meta.description = desc;
};
baseInputs = with pkgs; [
coreutils
curl
findutils
gnugrep
gnused
pass
];
# =========================================================================
# PyPI wheel (single-arch). Stage a built wheel into .parity-stage, then
# POST it to the Gitea PyPI registry (idempotent: 409 = already present).
# Args: { pname, version|versionFn, wheel (drv|path producing *.whl),
# arch ? "s390x", registryHost?, registryOwner?, passEntry? }
# =========================================================================
mkPyPiWheelPublish =
{
pname,
version,
wheel,
arch ? "s390x",
registryHost ? "git.oleks.space",
registryOwner ? "oleks",
passEntry ? "infra/gitea/personal_access_token_packages_rw",
extractCmd ? null,
}:
let
head = preamble { inherit registryHost registryOwner passEntry; };
# The wheel input may be a directory (Nix store output) we glob for *.whl,
# or an explicit single .whl path.
wheelRef = "${wheel}";
stageText = ''
${head}
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
echo "→ staging ${pname} $VERSION wheel (${arch}, no registry contact)"
d="$(parity_stage_reset ${lib.escapeShellArg arch})"
src=${lib.escapeShellArg wheelRef}
if [ -d "$src" ]; then
# shellcheck disable=SC2044
for w in $(find "$src" -name '*.whl'); do cp "$w" "$d/"; done
else
cp "$src" "$d/"
fi
whl="$(find "$d" -name '*.whl' | head -1)"
if [ -z "$whl" ]; then echo "BLOCKER(no-wheel): no *.whl under $src" >&2; exit 1; fi
printf 'pypi %s %s\n' "${pname}" "$VERSION" >"$d/.parity-meta"
sha256sum "$whl"
echo " staged: $whl"
'';
stage = pkgs.writeShellApplication {
name = "stage-${arch}";
runtimeInputs = baseInputs ++ [ pkgs.nix ];
text = stageText;
};
publish = pkgs.writeShellApplication {
name = "publish-${arch}";
runtimeInputs = baseInputs ++ [ pkgs.nix ];
text = ''
${head}
parity_parse_args "Build + publish the ${pname} ${arch} wheel to the Gitea PyPI registry" "$@"
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
${lib.getExe stage}
d="$(parity_stage_path ${lib.escapeShellArg arch})"
whl="$(find "$d" -name '*.whl' | head -1)"
if [ "$PARITY_PUBLISH" != "1" ]; then
echo "[dry-run] would POST $(basename "$whl") -> https://$PARITY_REGISTRY_HOST/api/packages/$PARITY_REGISTRY_OWNER/pypi (name=${pname} version=$VERSION)"
echo "[dry-run] re-run with --publish / PUBLISH=1 to upload."
exit 0
fi
parity_devtag_guard
tok="$(parity_resolve_token)"
parity_registry_preflight
sha="$(sha256sum "$whl" | cut -d' ' -f1)"
http="$(curl -s -o /dev/null -w '%{http_code}' -X POST \
-H "Authorization: token $tok" \
-F ':action=file_upload' -F 'protocol_version=1' \
-F "sha256_digest=$sha" -F "name=${pname}" -F "version=$VERSION" \
-F 'filetype=bdist_wheel' -F "content=@$whl" \
"https://$PARITY_REGISTRY_HOST/api/packages/$PARITY_REGISTRY_OWNER/pypi")"
case "$http" in
200 | 201 | 409) echo "Published ${pname} $VERSION (HTTP $http)" ;;
*) echo "BLOCKER(upload-failed): POST returned HTTP $http" >&2; exit 1 ;;
esac
'';
};
pushStaged = mkPushStagedGeneric {
inherit
pname
arch
registryHost
registryOwner
passEntry
;
kind = "pypi";
};
in
{
"stage-${arch}" = mkApp stage "Stage the ${pname} ${arch} wheel into .parity-stage (no registry contact).";
"publish-${arch}" = mkApp publish "Stage + publish the ${pname} ${arch} wheel (dry-run by default; --publish to push).";
"publish" = mkApp publish "Publish all locally-buildable arches (${arch} only for this PyPI repo).";
"push-staged" = pushStaged;
};
# =========================================================================
# s390x npm native addon (single-arch). Stage a ready-to-publish package dir
# (the .node + a generated package.json) then `npm publish`.
# Args: { pname (npm full name, e.g. @rollup/rollup-linux-s390x-gnu),
# version, nodeFile (drv|path to the *.node), nodeFileName,
# packageJson (string, with $version interpolated by the app),
# arch ? "s390x", registry* }
# =========================================================================
mkS390xNpmPublish =
{
pname,
version,
nodeFile,
nodeFileName,
packageJson,
arch ? "s390x",
registryHost ? "git.oleks.space",
registryOwner ? "oleks",
passEntry ? "infra/gitea/personal_access_token_packages_rw",
}:
let
head = preamble { inherit registryHost registryOwner passEntry; };
stage = pkgs.writeShellApplication {
name = "stage-${arch}";
runtimeInputs = baseInputs ++ [ pkgs.nodejs ];
text = ''
${head}
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
echo "→ staging ${pname}@$VERSION (${arch}, no registry contact)"
d="$(parity_stage_reset ${lib.escapeShellArg arch})"
install -m 0644 ${lib.escapeShellArg "${nodeFile}"} "$d/${nodeFileName}"
cat >"$d/package.json" <<EOF
${packageJson}
EOF
printf 'npm %s %s\n' "${pname}" "$VERSION" >"$d/.parity-meta"
echo " staged into $d:"
ls -lh "$d"
'';
};
publish = pkgs.writeShellApplication {
name = "publish-${arch}";
runtimeInputs = baseInputs ++ [ pkgs.nodejs ];
text = ''
${head}
parity_parse_args "Build + publish the ${pname} ${arch} npm addon to the Gitea npm registry" "$@"
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
${lib.getExe stage}
d="$(parity_stage_path ${lib.escapeShellArg arch})"
if [ "$PARITY_PUBLISH" != "1" ]; then
echo "[dry-run] would 'npm publish' $d -> https://$PARITY_REGISTRY_HOST/api/packages/$PARITY_REGISTRY_OWNER/npm/"
echo "[dry-run] re-run with --publish / PUBLISH=1 to push."
exit 0
fi
parity_devtag_guard
tok="$(parity_resolve_token)"
parity_registry_preflight
reg="https://$PARITY_REGISTRY_HOST/api/packages/$PARITY_REGISTRY_OWNER/npm/"
( cd "$d"
set +e
out="$(npm publish --registry="$reg" \
"--//$PARITY_REGISTRY_HOST/api/packages/$PARITY_REGISTRY_OWNER/npm/:_authToken=$tok" 2>&1)"
rc=$?
set -e
printf '%s\n' "$out" | sed "s/$tok/***REDACTED***/g"
if [ "$rc" -ne 0 ]; then
if printf '%s' "$out" | grep -qiE 'already exists|409|conflict'; then
echo "Version already published — idempotent success."; exit 0
fi
exit "$rc"
fi )
echo "Published ${pname}@$VERSION"
'';
};
in
{
"stage-${arch}" = mkApp stage "Stage the ${pname} ${arch} npm addon into .parity-stage (no registry contact).";
"publish-${arch}" = mkApp publish "Stage + publish the ${pname} ${arch} npm addon (dry-run by default; --publish to push).";
"publish" = mkApp publish "Publish all locally-buildable arches (${arch} only for this npm repo).";
"push-staged" = mkPushStagedGeneric {
inherit
pname
arch
registryHost
registryOwner
passEntry
;
kind = "npm";
};
};
# =========================================================================
# Generic binary (single-arch). Stage a built binary, PUT it to the Gitea
# generic registry (idempotent: 201 created / 409 exists).
# Args: { pname, version, binary (drv|path), assetName, arch ? "s390x",
# registry* }
# =========================================================================
mkGenericBinaryPublish =
{
pname,
version,
binary,
assetName,
arch ? "s390x",
registryHost ? "git.oleks.space",
registryOwner ? "oleks",
passEntry ? "infra/gitea/personal_access_token_packages_rw",
}:
let
head = preamble { inherit registryHost registryOwner passEntry; };
stage = pkgs.writeShellApplication {
name = "stage-${arch}";
runtimeInputs = baseInputs ++ [
pkgs.nix
pkgs.file
];
text = ''
${head}
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
echo "→ staging ${pname} $VERSION (${arch}, no registry contact)"
d="$(parity_stage_reset ${lib.escapeShellArg arch})"
cp ${lib.escapeShellArg "${binary}"} "$d/${assetName}"
printf 'generic %s %s %s\n' "${pname}" "$VERSION" "${assetName}" >"$d/.parity-meta"
file "$d/${assetName}" 2>/dev/null || true
echo " staged: $d/${assetName}"
'';
};
publish = pkgs.writeShellApplication {
name = "publish-${arch}";
runtimeInputs = baseInputs ++ [
pkgs.nix
pkgs.file
];
text = ''
${head}
parity_parse_args "Build + publish the ${pname} ${arch} binary to the Gitea generic registry" "$@"
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
${lib.getExe stage}
d="$(parity_stage_path ${lib.escapeShellArg arch})"
if [ "$PARITY_PUBLISH" != "1" ]; then
echo "[dry-run] would PUT $d/${assetName} -> https://$PARITY_REGISTRY_HOST/api/packages/$PARITY_REGISTRY_OWNER/generic/${pname}/$VERSION/${assetName}"
echo "[dry-run] re-run with --publish / PUBLISH=1 to upload."
exit 0
fi
parity_devtag_guard
tok="$(parity_resolve_token)"
parity_registry_preflight
http="$(curl -s -o /dev/null -w '%{http_code}' -X PUT \
-H "Authorization: token $tok" --upload-file "$d/${assetName}" \
"https://$PARITY_REGISTRY_HOST/api/packages/$PARITY_REGISTRY_OWNER/generic/${pname}/$VERSION/${assetName}")"
case "$http" in
201 | 409) echo "Published ${assetName} $VERSION (HTTP $http)" ;;
*) echo "BLOCKER(upload-failed): PUT returned HTTP $http" >&2; exit 1 ;;
esac
'';
};
in
{
"stage-${arch}" = mkApp stage "Stage the ${pname} ${arch} binary into .parity-stage (no registry contact).";
"publish-${arch}" = mkApp publish "Stage + publish the ${pname} ${arch} binary (dry-run by default; --publish to push).";
"publish" = mkApp publish "Publish all locally-buildable arches (${arch} only for this generic-binary repo).";
"push-staged" = mkPushStagedGeneric {
inherit
pname
arch
assetName
registryHost
registryOwner
passEntry
;
kind = "generic";
inherit version;
};
};
# =========================================================================
# Go binary (single-arch generic). Thin alias over the generic-binary builder
# — a cross-compiled Go binary is published to the generic registry exactly
# like any other binary. Kept as a distinct named builder so the archetype is
# explicit in consumer flakes and pipeline-doctor.
# =========================================================================
mkGoBinaryPublish = mkGenericBinaryPublish;
# =========================================================================
# Helm chart (single artifact). Stage `helm package` output (.tgz) then
# `helm push` to the OCI registry (idempotent: a duplicate version errors,
# which we treat as success).
# Args: { pname (chart name), version, chartSrc (drv|path to chart dir),
# ociRepo ? "oci://git.oleks.space/oleks", registry* }
# =========================================================================
mkHelmPublish =
{
pname,
version,
chartSrc,
ociRepo ? "oci://git.oleks.space/oleks",
registryHost ? "git.oleks.space",
registryOwner ? "oleks",
passEntry ? "infra/gitea/personal_access_token_packages_rw",
}:
let
head = preamble { inherit registryHost registryOwner passEntry; };
stage = pkgs.writeShellApplication {
name = "stage-chart";
runtimeInputs = baseInputs ++ [ pkgs.kubernetes-helm ];
text = ''
${head}
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
echo "→ staging helm chart ${pname} $VERSION (no registry contact)"
d="$(parity_stage_reset chart)"
helm package ${lib.escapeShellArg "${chartSrc}"} --version "$VERSION" --app-version "$VERSION" --destination "$d"
printf 'helm %s %s\n' "${pname}" "$VERSION" >"$d/.parity-meta"
echo " staged:"; ls -lh "$d"/*.tgz
'';
};
publish = pkgs.writeShellApplication {
name = "publish-chart";
runtimeInputs = baseInputs ++ [ pkgs.kubernetes-helm ];
text = ''
${head}
parity_parse_args "Package + publish the ${pname} helm chart to ${ociRepo}" "$@"
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
${lib.getExe stage}
d="$(parity_stage_path chart)"
tgz="$(find "$d" -name '*.tgz' | head -1)"
if [ "$PARITY_PUBLISH" != "1" ]; then
echo "[dry-run] would 'helm push $(basename "$tgz")' -> ${ociRepo}"
echo "[dry-run] re-run with --publish / PUBLISH=1 to push."
exit 0
fi
parity_devtag_guard
tok="$(parity_resolve_token)"
parity_registry_preflight
echo "$tok" | helm registry login "$PARITY_REGISTRY_HOST" -u "$PARITY_REGISTRY_OWNER" --password-stdin >/dev/null 2>&1
set +e
out="$(helm push "$tgz" ${lib.escapeShellArg ociRepo} 2>&1)"
rc=$?
set -e
printf '%s\n' "$out" | sed "s/$tok/***REDACTED***/g"
if [ "$rc" -ne 0 ]; then
if printf '%s' "$out" | grep -qiE 'already exists|409|conflict'; then
echo "Chart version already published — idempotent success."; exit 0
fi
exit "$rc"
fi
echo "Published chart ${pname} $VERSION"
'';
};
in
{
"stage-chart" = mkApp stage "Package the ${pname} helm chart into .parity-stage (no registry contact).";
"publish-chart" = mkApp publish "Package + publish the ${pname} helm chart (dry-run by default; --publish to push).";
"publish" = mkApp publish "Publish the ${pname} helm chart.";
"push-staged" = mkPushStagedHelm {
inherit
pname
ociRepo
registryHost
registryOwner
passEntry
;
};
};
# =========================================================================
# nix2container OCI image (MULTI-ARCH). This is the archetype that yields the
# full stage/publish-<arch>/publish-index/publish/push-staged set.
#
# Args: {
# imageName, # e.g. git.oleks.space/oleks/nix-ci
# version|tagFn, # the immutable :TAG
# images, # { <arch> = { copyTo = <nix2container copyTo drv>; }; }
# # only arches buildable on THIS host should appear
# arches ? attrNames images, # the full set the index must cover (fail-closed)
# registry* }
#
# Per-arch publish copies the arch image to <imageName>:<TAG>-<arch>.
# publish-index assembles <imageName>:<TAG> from the per-arch digest-pinned
# tags via `regctl index create`, failing closed if any required arch tag is
# missing. publish runs all local arches, then the index, then copies
# :<TAG> -> :latest as the LAST idempotent mutation.
# =========================================================================
mkNix2ContainerPublish =
{
imageName,
version,
images,
arches ? lib.attrNames images,
registryHost ? "git.oleks.space",
registryOwner ? "oleks",
passEntry ? "infra/gitea/personal_access_token_packages_rw",
}:
let
head = preamble { inherit registryHost registryOwner passEntry; };
localArches = lib.attrNames images;
# per-arch stage: realise the image's copy-to closure locally (no push).
mkArchStage =
arch:
pkgs.writeShellApplication {
name = "stage-${arch}";
runtimeInputs = baseInputs ++ [ pkgs.nix ];
text = ''
${head}
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
echo "→ staging ${imageName}:$VERSION-${arch} (build closure only, no push)"
d="$(parity_stage_reset ${lib.escapeShellArg arch})"
# Realise the nix2container copyTo wrapper (its closure is the image).
out=${lib.escapeShellArg "${images.${arch}.copyTo}"}
printf 'oci %s %s %s\n' ${lib.escapeShellArg imageName} "$VERSION" "${arch}" >"$d/.parity-meta"
printf '%s\n' "$out" >"$d/copy-to-path"
echo " staged copyTo: $out"
'';
};
mkArchPublish =
arch:
pkgs.writeShellApplication {
name = "publish-${arch}";
runtimeInputs = baseInputs ++ [ pkgs.nix ];
text = ''
${head}
parity_parse_args "Build + push ${imageName}:<TAG>-${arch}" "$@"
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
${lib.getExe (mkArchStage arch)}
if [ "$PARITY_PUBLISH" != "1" ]; then
echo "[dry-run] would copy ${imageName}:$VERSION-${arch} (digest-pinned per-arch tag)"
echo "[dry-run] re-run with --publish / PUBLISH=1 to push."
exit 0
fi
parity_devtag_guard
tok="$(parity_resolve_token)"
parity_registry_preflight
creds="$PARITY_REGISTRY_OWNER:$tok"
${lib.escapeShellArg "${images.${arch}.copyTo}"}/bin/copy-to \
--dest-creds "$creds" \
"docker://${imageName}:$VERSION-${arch}"
echo "Pushed ${imageName}:$VERSION-${arch}"
'';
};
archStageApps = lib.listToAttrs (
map (a: {
name = "stage-${a}";
value = mkApp (mkArchStage a) "Stage the ${imageName} ${a} image closure into .parity-stage (no push).";
}) localArches
);
archPublishApps = lib.listToAttrs (
map (a: {
name = "publish-${a}";
value = mkApp (mkArchPublish a) "Build + push ${imageName}:<TAG>-${a} (dry-run by default; --publish to push).";
}) localArches
);
# build-free multi-arch index from the per-arch digest-pinned tags.
indexApp = pkgs.writeShellApplication {
name = "publish-index";
runtimeInputs = baseInputs ++ [ pkgs.regclient ];
text = ''
${head}
parity_parse_args "Assemble the multi-arch index ${imageName}:<TAG> from pushed per-arch tags" "$@"
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
req=(${lib.concatStringsSep " " (map lib.escapeShellArg arches)})
if [ "$PARITY_PUBLISH" != "1" ]; then
echo "[dry-run] would 'regctl index create ${imageName}:$VERSION' from:"
for a in "''${req[@]}"; do echo " ${imageName}:$VERSION-$a"; done
echo "[dry-run] re-run with --publish / PUBLISH=1 to assemble."
exit 0
fi
parity_devtag_guard
tok="$(parity_resolve_token)"
parity_registry_preflight
regctl registry login "$PARITY_REGISTRY_HOST" -u "$PARITY_REGISTRY_OWNER" --pass-stdin <<<"$tok" >/dev/null 2>&1
# Fail-closed: every required arch tag must exist before assembly.
refs=()
for a in "''${req[@]}"; do
ref="${imageName}:$VERSION-$a"
if ! regctl manifest head "$ref" >/dev/null 2>&1; then
echo "BLOCKER(missing-arch): required arch tag $ref was not pushed this run; refusing to assemble a partial index." >&2
exit 1
fi
refs+=("$ref")
done
regctl index create "${imageName}:$VERSION" "''${refs[@]/#/--ref=}"
echo "Assembled index ${imageName}:$VERSION"
'';
};
# publish: all locally-buildable arches -> index -> :latest (last).
publishAll = pkgs.writeShellApplication {
name = "publish";
runtimeInputs = baseInputs ++ [
pkgs.nix
pkgs.regclient
];
text = ''
${head}
parity_parse_args "Publish all local arches of ${imageName} + the multi-arch index; :latest = digest copy of :TAG" "$@"
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
fwd=()
[ "$PARITY_PUBLISH" = "1" ] && fwd=(--publish)
${lib.concatMapStringsSep "\n" (a: " ${lib.getExe (mkArchPublish a)} \"\${fwd[@]}\"") localArches}
${lib.getExe indexApp} "''${fwd[@]}"
if [ "$PARITY_PUBLISH" != "1" ]; then
echo "[dry-run] would copy ${imageName}:$VERSION -> ${imageName}:latest (digest copy, LAST mutation)"
exit 0
fi
tok="$(parity_resolve_token)"
regctl registry login "$PARITY_REGISTRY_HOST" -u "$PARITY_REGISTRY_OWNER" --pass-stdin <<<"$tok" >/dev/null 2>&1
# ':latest' is a pure digest copy of the just-assembled ':TAG', done
# LAST so it is the single idempotent mutation that flips the channel.
regctl image copy "${imageName}:$VERSION" "${imageName}:latest"
echo "Published ${imageName}:$VERSION and flipped :latest"
'';
};
pushStaged = pkgs.writeShellApplication {
name = "push-staged";
runtimeInputs = baseInputs ++ [ pkgs.nix ];
text = ''
${head}
parity_parse_args "Replay staged ${imageName} arch closures from .parity-stage to the registry" "$@"
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
if [ "$PARITY_PUBLISH" != "1" ]; then
echo "[dry-run] would replay staged arch images under $PARITY_STAGE_DIR to ${imageName}:$VERSION-<arch>"
echo "[dry-run] re-run with --publish / PUBLISH=1 to push."
exit 0
fi
parity_devtag_guard
tok="$(parity_resolve_token)"
parity_registry_preflight
creds="$PARITY_REGISTRY_OWNER:$tok"
shopt -s nullglob
for d in "$PARITY_STAGE_DIR"/*/; do
[ -f "$d/copy-to-path" ] || continue
arch="$(basename "$d")"
copyto="$(cat "$d/copy-to-path")"
"$copyto/bin/copy-to" --dest-creds "$creds" "docker://${imageName}:$VERSION-$arch"
echo "Replayed ${imageName}:$VERSION-$arch"
done
'';
};
in
archStageApps
// archPublishApps
// {
"publish-index" = mkApp indexApp "Assemble the multi-arch index ${imageName}:<TAG> from pushed per-arch tags (fail-closed).";
"publish" = mkApp publishAll "Publish all local arches + index; :latest = digest copy of :TAG (last mutation).";
"push-staged" = mkApp pushStaged "Replay staged ${imageName} arch closures from .parity-stage to the registry.";
};
# =========================================================================
# Shared push-staged for single-artifact archetypes (pypi/npm/generic): replay
# the staged file from .parity-stage to the registry without rebuilding.
# =========================================================================
mkPushStagedGeneric =
{
pname,
arch,
kind,
version ? "",
assetName ? "",
registryHost ? "git.oleks.space",
registryOwner ? "oleks",
passEntry ? "infra/gitea/personal_access_token_packages_rw",
}:
let
head = preamble { inherit registryHost registryOwner passEntry; };
# The kind is known at Nix-eval time, so the kind-specific replay body is
# selected here (not via a runtime `case` on a constant, which trips
# shellcheck SC2194).
kindBody = {
pypi = ''
whl="$(find "$d" -name '*.whl' | head -1)"
sha="$(sha256sum "$whl" | cut -d' ' -f1)"
http="$(curl -s -o /dev/null -w '%{http_code}' -X POST \
-H "Authorization: token $tok" -F ':action=file_upload' \
-F 'protocol_version=1' -F "sha256_digest=$sha" \
-F "name=${pname}" -F "version=$VERSION" -F 'filetype=bdist_wheel' \
-F "content=@$whl" \
"https://$PARITY_REGISTRY_HOST/api/packages/$PARITY_REGISTRY_OWNER/pypi")"
case "$http" in
200 | 201 | 409) echo "Replayed ${pname} $VERSION (HTTP $http)" ;;
*) echo "BLOCKER(upload-failed): HTTP $http" >&2; exit 1 ;;
esac
'';
generic = ''
http="$(curl -s -o /dev/null -w '%{http_code}' -X PUT \
-H "Authorization: token $tok" --upload-file "$d/${assetName}" \
"https://$PARITY_REGISTRY_HOST/api/packages/$PARITY_REGISTRY_OWNER/generic/${pname}/$VERSION/${assetName}")"
case "$http" in
201 | 409) echo "Replayed ${assetName} $VERSION (HTTP $http)" ;;
*) echo "BLOCKER(upload-failed): HTTP $http" >&2; exit 1 ;;
esac
'';
npm = ''
reg="https://$PARITY_REGISTRY_HOST/api/packages/$PARITY_REGISTRY_OWNER/npm/"
(
cd "$d"
set +e
out="$(npm publish --registry="$reg" "--//$PARITY_REGISTRY_HOST/api/packages/$PARITY_REGISTRY_OWNER/npm/:_authToken=$tok" 2>&1)"
rc=$?
set -e
printf '%s\n' "$out" | sed "s/$tok/***REDACTED***/g"
if [ "$rc" -ne 0 ]; then
if printf '%s' "$out" | grep -qiE 'already exists|409|conflict'; then
echo "Already published — idempotent."
exit 0
fi
exit "$rc"
fi
)
'';
};
app = pkgs.writeShellApplication {
name = "push-staged";
runtimeInputs = baseInputs ++ lib.optional (kind == "npm") pkgs.nodejs;
text = ''
${head}
parity_parse_args "Replay the staged ${pname} ${arch} artifact (${kind}) from .parity-stage to the registry" "$@"
d="$(parity_stage_path ${lib.escapeShellArg arch})"
if [ ! -d "$d" ]; then
echo "BLOCKER(no-stage): nothing staged at $d — run 'nix run .#stage-${arch}' first." >&2
exit 1
fi
VERSION="$(parity_derive_version ${lib.escapeShellArg version})"
if [ "$PARITY_PUBLISH" != "1" ]; then
echo "[dry-run] would replay $d (${kind}) -> https://$PARITY_REGISTRY_HOST (${pname} $VERSION)"
echo "[dry-run] re-run with --publish / PUBLISH=1 to push."
exit 0
fi
parity_devtag_guard
tok="$(parity_resolve_token)"
parity_registry_preflight
${kindBody.${kind}}
'';
};
in
mkApp app "Replay the staged ${pname} ${arch} artifact (${kind}) from .parity-stage (dry-run by default; --publish to push).";
mkPushStagedHelm =
{
pname,
ociRepo,
registryHost ? "git.oleks.space",
registryOwner ? "oleks",
passEntry ? "infra/gitea/personal_access_token_packages_rw",
}:
let
head = preamble { inherit registryHost registryOwner passEntry; };
app = pkgs.writeShellApplication {
name = "push-staged";
runtimeInputs = baseInputs ++ [ pkgs.kubernetes-helm ];
text = ''
${head}
parity_parse_args "Replay the staged ${pname} helm chart from .parity-stage to ${ociRepo}" "$@"
d="$(parity_stage_path chart)"
tgz="$(find "$d" -name '*.tgz' 2>/dev/null | head -1 || true)"
if [ -z "$tgz" ]; then
echo "BLOCKER(no-stage): no staged chart under $d — run 'nix run .#stage-chart' first." >&2
exit 1
fi
if [ "$PARITY_PUBLISH" != "1" ]; then
echo "[dry-run] would 'helm push $(basename "$tgz")' -> ${ociRepo}"
exit 0
fi
parity_devtag_guard
tok="$(parity_resolve_token)"
parity_registry_preflight
echo "$tok" | helm registry login "$PARITY_REGISTRY_HOST" -u "$PARITY_REGISTRY_OWNER" --password-stdin >/dev/null 2>&1
helm push "$tgz" ${lib.escapeShellArg ociRepo}
echo "Replayed chart $(basename "$tgz")"
'';
};
in
mkApp app "Replay the staged ${pname} helm chart from .parity-stage (dry-run by default; --publish to push).";
in
{
inherit
mkPyPiWheelPublish
mkS390xNpmPublish
mkGenericBinaryPublish
mkNix2ContainerPublish
mkGoBinaryPublish
mkHelmPublish
;
# Shared building blocks, exposed for advanced/bespoke consumers.
inherit mkApp shellLib;
}