Derive core_scaling from the derivation env, not a 35-entry name table #4

New Issue

2026-07-07T20:13:21+03:00

oleks commented

2026-07-07 20:13:21 +03:00

Problem

core_scaling (the Amdahl parallel fraction that drives the cores/node axis of the sweep) currently comes only from the ~35-entry HEAVY substring table. Any package not on that list gets DEFAULT scaling 0.30 regardless of whether it actually parallelizes — so the cores/node recommendation is a guess for the long tail.

Proposal

The nix derivation show JSON exposes each drv's env. Read a real per-derivation parallelism signal from it before falling back to the table:

enableParallelBuilding = "1" → high scaling (~0.85).
presence of NIX_BUILD_CORES / -j in env/args / makeFlags → parallel.
dontBuild / fixed-output / trivial (see #2) → 0.0.

Keep the HEAVY table as an override for known pathological cases (e.g. ghc scales poorly despite parallel flags), but derive the default from the drv itself. More robust across arbitrary packages than the hardcoded list.

Independent of #1–#3; touches costmodel.cost() only.

## Problem `core_scaling` (the Amdahl parallel fraction that drives the cores/node axis of the sweep) currently comes only from the ~35-entry `HEAVY` substring table. Any package not on that list gets `DEFAULT` scaling 0.30 regardless of whether it actually parallelizes — so the cores/node recommendation is a guess for the long tail. ## Proposal The `nix derivation show` JSON exposes each drv's `env`. Read a real per-derivation parallelism signal from it before falling back to the table: - `enableParallelBuilding = "1"` → high scaling (~0.85). - presence of `NIX_BUILD_CORES` / `-j` in `env`/`args` / `makeFlags` → parallel. - `dontBuild` / fixed-output / trivial (see #2) → 0.0. Keep the `HEAVY` table as an override for known pathological cases (e.g. `ghc` scales poorly despite parallel flags), but derive the default from the drv itself. More robust across arbitrary packages than the hardcoded list. Independent of #1–#3; touches `costmodel.cost()` only.

admin referenced this issue

2026-07-11 10:59:45 +03:00

Stable machine-readable --json sizing contract for CI/arbitrage consumption #25

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: oleks/nix-estimator#4