Oleks
2201257e89
Implements the shared parity flake-module library so the ~51 parity repos
consume one source of truth instead of hand-inlined publish shells.
- lib.mk{PyPiWheel,S390xNpm,GenericBinary,Nix2Container,GoBinary,Helm}Publish
builders returning stage-<arch>/publish-<arch>/publish-index/publish/
push-staged apps per the corrected emmett#44 standard (build-parity stages to
./.parity-stage with no registry contact; publish dry-runs by default;
publish-index is build-free + fail-closed; :latest is the last digest copy).
- Shared ci/parity-lib.sh: token resolution ($REGISTRY_TOKEN + pass fallback,
never printed), dev-tag guard, version derivation, dry-run gate, preflight.
- pipeline-doctor package/app asserting the parity contract (cluster #193).
Refs cluster #192, #193, #194, emmett#44.
183 lines
6.7 KiB
Markdown
183 lines
6.7 KiB
Markdown
# parity-lib
|
|
|
|
<!-- markdownlint-disable MD013 MD040 -->
|
|
|
|
Shared per-archetype **publish-app builders** for the ~51 "parity" repos
|
|
(cluster #192/#193/#194, emmett#44). Instead of each repo hand-inlining its own
|
|
`stage` / `publish` / `push-staged` shell, every repo consumes ONE versioned
|
|
source of truth: this flake's `lib.mk*Publish` builders generate the flake apps
|
|
that implement the corrected parity standard.
|
|
|
|
## The corrected parity standard (emmett#44)
|
|
|
|
Parity has two halves, and only one of them can be cluster-independent:
|
|
|
|
- **BUILD/STAGE parity is cluster-independent.** `nix run .#stage-<arch>`
|
|
cross-builds the artifact into a local on-disk store (`./.parity-stage/<arch>`
|
|
by convention) and makes **NO registry contact**, so it runs identically on
|
|
emmett even when armer / the registry is down.
|
|
- **PUBLISH parity shares fate with the cluster** (the only registry is
|
|
co-located with it). The publish apps name the blocker (`registry-down`) up
|
|
front and never half-push.
|
|
|
|
Every push entrypoint **DRY-RUNS by default** — it stages and prints what it
|
|
WOULD push. You must pass `--publish` (or set `PUBLISH=1`) to mutate the
|
|
registry, so an accidental local run can never push.
|
|
|
|
## App shape each builder emits
|
|
|
|
| app | parity half | registry | mutates? |
|
|
| --- | --- | --- | --- |
|
|
| `stage-<arch>` | BUILD | none | no |
|
|
| `publish-<arch>` | BUILD + PUBLISH | one arch | only with `--publish` |
|
|
| `publish-index` | PUBLISH | multi-arch assembly (regctl) | only with `--publish` |
|
|
| `publish` | BUILD + PUBLISH | all local arches + index + `:latest` | only with `--publish` |
|
|
| `push-staged` | PUBLISH | replay `.parity-stage` | only with `--publish` |
|
|
|
|
- `publish-index` is **build-free**: it assembles `<image>:<TAG>` from the
|
|
per-arch digest-pinned tags via `regctl index create`, and is **fail-closed** —
|
|
if a required arch was not pushed this run, it refuses to assemble a partial
|
|
index.
|
|
- `publish` runs all locally-buildable arches, then the index, then copies
|
|
`:<TAG>` → `:latest` as the **LAST** (single, idempotent) mutation.
|
|
- `push-staged` replays artifacts from `./.parity-stage` to the registry, for
|
|
when the cluster was down at build time.
|
|
|
|
Single-arch archetypes (PyPI wheel, npm addon, generic/Go binary, Helm chart)
|
|
have no multi-arch index, so they expose `stage-<arch>` / `publish-<arch>` /
|
|
`publish` / `push-staged` only. The nix2container (OCI) builder is the one that
|
|
yields the full `publish-index` / `:latest` set.
|
|
|
|
## Shared building blocks (also exposed)
|
|
|
|
All builders source one shell library (`ci/parity-lib.sh`, materialized in the
|
|
Nix store) so behavior cannot drift between repos:
|
|
|
|
- **Token resolution** — `$REGISTRY_TOKEN` → `pass infra/gitea/...` fallback →
|
|
named hard-fail. The token is **never printed** and scripts run under
|
|
`set -euo pipefail` only (never `set -x`).
|
|
- **Dev-tag guard** — refuses a real (`:latest`/release) publish unless
|
|
`$VERSION` is set or `$CI_COMMIT_TAG` is a `v*` tag.
|
|
- **Version derivation** — `$VERSION` → strip leading `v` + trailing `-N` from
|
|
`$CI_COMMIT_TAG` → the flake's pinned default. Identical for CI and local.
|
|
|
|
## API — `lib.*`
|
|
|
|
`lib` is system-independent. Two ways to consume it:
|
|
|
|
```nix
|
|
# (a) one call, all builders, with your own pkgs:
|
|
parity.lib.mkParityBuilders pkgs # -> { mkPyPiWheelPublish, ... }
|
|
|
|
# (b) per-builder wrapper that takes pkgs as the first argument:
|
|
parity.lib.mkPyPiWheelPublish pkgs { pname = "..."; version = "..."; ... }
|
|
```
|
|
|
|
Exposed attrs:
|
|
|
|
- `lib.mkParityBuilders` — `pkgs -> { the six builders + mkApp + shellLib }`
|
|
- `lib.mkPyPiWheelPublish` — `pkgs -> args -> { apps }`
|
|
- `lib.mkS390xNpmPublish` — `pkgs -> args -> { apps }`
|
|
- `lib.mkGenericBinaryPublish` — `pkgs -> args -> { apps }`
|
|
- `lib.mkNix2ContainerPublish` — `pkgs -> args -> { apps }`
|
|
- `lib.mkGoBinaryPublish` — `pkgs -> args -> { apps }` (alias of generic-binary)
|
|
- `lib.mkHelmPublish` — `pkgs -> args -> { apps }`
|
|
|
|
### Builder arguments
|
|
|
|
```nix
|
|
mkPyPiWheelPublish {
|
|
pname = "asyncpg";
|
|
version = "0.31.0"; # default; overridden by $VERSION / $CI_COMMIT_TAG
|
|
wheel = self.packages.x86_64-linux.default; # drv/dir to glob *.whl, or a .whl path
|
|
arch = "s390x"; # default
|
|
}
|
|
|
|
mkS390xNpmPublish {
|
|
pname = "@rollup/rollup-linux-s390x-gnu";
|
|
version = "4.0.0";
|
|
nodeFile = self.packages.x86_64-linux.addon; # the *.node
|
|
nodeFileName = "rollup.linux-s390x-gnu.node";
|
|
packageJson = ''{ "name": "@rollup/...", "version": "$VERSION", ... }'';
|
|
}
|
|
|
|
mkGenericBinaryPublish { # mkGoBinaryPublish is an alias
|
|
pname = "geesefs";
|
|
version = "0.43.5";
|
|
binary = "${self.packages.x86_64-linux.default}/bin/geesefs";
|
|
assetName = "geesefs-linux-s390x";
|
|
arch = "s390x";
|
|
}
|
|
|
|
mkNix2ContainerPublish {
|
|
imageName = "git.oleks.space/oleks/nix-ci";
|
|
version = "1.2.3";
|
|
images = { # only arches buildable on THIS host
|
|
amd64 = { copyTo = self.packages.x86_64-linux.nix-ci.copyTo; };
|
|
};
|
|
arches = [ "amd64" "arm64" ]; # the full set the index must cover (fail-closed)
|
|
}
|
|
|
|
mkHelmPublish {
|
|
pname = "firecrawl";
|
|
version = "0.1.3";
|
|
chartSrc = ./charts/firecrawl;
|
|
ociRepo = "oci://git.oleks.space/oleks"; # default
|
|
}
|
|
```
|
|
|
|
Common optional args on every builder: `registryHost` (`git.oleks.space`),
|
|
`registryOwner` (`oleks`), `passEntry`
|
|
(`infra/gitea/personal_access_token_packages_rw`).
|
|
|
|
### Consuming it in a parity repo's flake
|
|
|
|
```nix
|
|
{
|
|
inputs.parity.url = "git+https://git.oleks.space/oleks/parity-lib";
|
|
outputs = { self, nixpkgs, parity, flake-utils, ... }:
|
|
flake-utils.lib.eachDefaultSystem (system:
|
|
let pkgs = import nixpkgs { inherit system; };
|
|
in {
|
|
apps = parity.lib.mkGenericBinaryPublish pkgs {
|
|
pname = "geesefs";
|
|
version = "0.43.5";
|
|
binary = "${self.packages.${system}.default}/bin/geesefs";
|
|
assetName = "geesefs-linux-s390x";
|
|
};
|
|
});
|
|
}
|
|
```
|
|
|
|
`.woodpecker.yaml` stays thin — it runs the identical app, so CI and local
|
|
cannot drift:
|
|
|
|
```yaml
|
|
commands:
|
|
- PUBLISH=1 nix run .#publish
|
|
```
|
|
|
|
## pipeline-doctor (cluster #193)
|
|
|
|
`nix run .#pipeline-doctor -- <repo-path>` asserts the parity contract on a
|
|
repo: archetype declared, consumes parity-lib, token =
|
|
`$REGISTRY_TOKEN` + `pass` fallback and never printed, dev-tag guard present, a
|
|
`--dry-run` default exists, apps carry `meta.description`, and an enumerable
|
|
`publish-*` / `stage-*` naming. It prints the local-equivalent commands. It is
|
|
read-only (no token, no registry contact) and exits non-zero if any required
|
|
check fails.
|
|
|
|
## Verifying changes locally
|
|
|
|
```bash
|
|
nix eval .#lib --apply builtins.attrNames --json
|
|
nix flake show
|
|
nix build .#pipeline-doctor
|
|
shellcheck ci/*.sh
|
|
statix check .
|
|
markdownlint-cli2 README.md
|
|
```
|
|
|
|
The git.oleks.space server runs a pre-receive linter; the commands above mirror
|
|
it.
|