Commit Graph

30 Commits

Author SHA1 Message Date
Oleks cc2eabe321 Merge wave2B branch (CLI integration #12-#16) 2026-07-08 17:43:17 +03:00
Oleks 5f1a75fb12 tests+docs: cover new CLI surfaces; document flags + mine subcommand (#12-#16)
- test_cli.py: --max-jobs/--node-ram-gb accepted and affect output, --timeline
  renders + carries JSON assignments, --provision emits YAML (stdout) with
  report on stderr, mine subcommand writes JSON (stubbed runner), plain
  estimate back-compat.
- README: usage for --max-jobs/--node-ram-gb/--timeline/--provision + a mining
  section; refreshed Status/roadmap and layout.
2026-07-08 17:41:46 +03:00
Oleks 38aaa76a11 cli: wire max-jobs/node-ram-gb/timeline/provision + mine subcommand (#12-#16)
- --max-jobs / --node-ram-gb pass through to estimate(); shown in report header
  and JSON.
- --timeline recomputes the recommended shape's schedule and renders a compact
  per-node text Gantt (also emitted under 'timeline' in --json).
- --provision [CLASS] (+ --provision-name) renders the EphemeralBuilder YAML to
  stdout while the human/JSON report goes to stderr, so it pipes to kubectl.
- new 'mine' subcommand: nix-estimate mine <installables...> [--repeat N] [-o]
  runs mine_history, merges repeats via merge_histories, writes {name: minutes}.
  Default estimate path preserved: 'mine' is dispatched on the leading token,
  so 'nix-estimate <attr> [flags]' is unchanged.
2026-07-08 17:41:24 +03:00
Oleks 9c23a539d4 estimate: widen scale_to_cores to float; add schedule_for helper (#13/#16)
- scale_to_cores now annotates cores as float (eff_cores = cores/max_jobs is
  a float; silences pyright).
- schedule_for() returns the concrete (makespan, per-lane assignments) for a
  single (machines x cores) shape, reusing the grid's durations, for --timeline.
2026-07-08 17:40:52 +03:00
Oleks dbf72e45bc style: auto-format from pre-push hooks 2026-07-08 17:28:56 +03:00
Oleks 24a351521c Merge wave2A branch a1d32f0a68a315902 (features) 2026-07-08 17:23:45 +03:00
Oleks b3e2d0948d Merge wave2A branch aafcffb723f2fc144 (features) 2026-07-08 17:23:43 +03:00
Oleks a9efc705c5 Merge wave2A branch a604e4a5f4fb7c756 (features) 2026-07-08 17:23:40 +03:00
Oleks 84ccac046d schedule+estimate: RAM-per-job budget caps node concurrency (#15) 2026-07-08 17:22:10 +03:00
Oleks 34340df921 schedule+estimate: per-node max_jobs concurrent builds sharing cores (#13) 2026-07-08 17:17:19 +03:00
Oleks 0abe485b2a schedule: makespan can return per-machine (task,start,finish) assignments (#16) 2026-07-08 16:38:25 +03:00
Oleks 565744a35d estimate: snap clamped node count onto node_grid, fix _recommend KeyError (#18) 2026-07-08 16:36:45 +03:00
Oleks 327a8edcda miner: mine real per-drv build times from nix internal-json logs (#12) 2026-07-08 16:35:39 +03:00
Oleks 65832fa3d6 provision: render_ephemeral_builder emits EphemeralBuilder CR (#14) 2026-07-08 16:34:33 +03:00
Oleks b8fe19e3f2 style: auto-format from pre-push hooks 2026-07-08 16:30:48 +03:00
Oleks 9711b31b00 Merge wave1 branch abc4312fcbec99f37 (bug fixes) 2026-07-08 16:30:02 +03:00
Oleks 182445fad6 Merge wave1 branch a616f0dd7c6bd39e1 (bug fixes) 2026-07-08 16:30:02 +03:00
Oleks bc0d0d87e2 Merge wave1 branch a0507fd220ee127e3 (bug fixes) 2026-07-08 16:30:02 +03:00
Oleks 53bbcb678d Merge wave1 branch a28a4552ac56e246c (bug fixes) 2026-07-08 16:30:02 +03:00
Oleks 07c921cbf4 costmodel: history prefix match picks longest key deterministically (#7) 2026-07-08 16:29:02 +03:00
Oleks 3e21363697 costmodel: anchor HEAVY table matches on package-name token (#6) 2026-07-08 16:28:19 +03:00
Oleks ffc4494486 graph: prefer nix build --dry-run --json, fall back to text scraper (#8)
Probe --json support via nix build --help and parse structured output
when available (robust across Nix versions and locales). Falls back to
the #5 text scraper on older Nix or unexpected payloads, feeding the
same tri-state contract.
2026-07-08 16:25:29 +03:00
Oleks 52fc3e1ecf estimate: normalize node/core grids to ascending, unique (#11)
_recommend assumes sorted, deduped grids (core_grid[-1] = biggest, knee
sweeps run low->high). The CLI parsed --nodes/--cores in raw order, so
e.g. --cores 32,8,16 yielded nonsense. Sort+dedupe in cli.py and reject
non-positive/non-integer values; also normalize defensively inside
estimate() so the function is correct for any caller. Add estimate/cli
tests.
2026-07-08 16:25:23 +03:00
Oleks 50e1353ebf schedule: heap-based makespan dispatch, O(n log n) (#10) 2026-07-08 16:25:01 +03:00
Oleks cde91893a9 schedule: keep zero-duration predecessors in critical_path chain (#9) 2026-07-08 16:25:01 +03:00
Oleks 9eb7237f04 graph: tri-state to_build_set — empty means nothing to build (#5)
Refactor dry-run text parsing into parse_dry_run_text(text, returncode)
so an empty set (warm cache) is distinct from None (unparseable). Only
None triggers the cold-cache whole-closure fallback in cli.py; an empty
set now correctly reports 'nothing to build'.
2026-07-08 16:24:13 +03:00
Oleks 31fa848ed0 estimator: --cold flag (#1) + trivial-glue cost tier (#2)
#1 --cold/--full forces the whole-closure cold-cache estimate, so a
fully-built target no longer dead-ends at 'nothing to build'.

#2 cost NixOS assembly-glue derivations (units, activation, /etc, the
system closure) at ~0 instead of the flat 1min DEFAULT, so a deep glue
chain stops fabricating the span. Critical-path report now shows the
heaviest derivations on the path (not the terminal glue), and the
recommendation note branches on whether the span is one heavy drv or a
long cheap chain. Adds tests/test_costmodel.py.
2026-07-07 20:22:54 +03:00
Oleks 10cd66e247 style: auto-format from pre-push hooks 2026-07-07 20:06:04 +03:00
Oleks 1416587933 graph: support Nix 2.30+ versioned derivation-show schema
Nix >=2.34 wraps the closure under {version, derivations}, uses bare drv
keys, and moved edges from inputDrvs to inputs.drvs. dry-run still emits
full store paths, so normalize every drv path to basename for a
schema-agnostic join. Fixes closure collapsing to 2 derivations.
2026-07-07 20:06:02 +03:00
Oleks a0852ff479 nix-estimator v0.1: DAG → node×core builder-config estimator
Reads the to-build derivation DAG from Nix (derivation show -r + --dry-run),
applies a heuristic/history build-cost model with Amdahl core-scaling, and
computes critical-path / peak-concurrency / list-scheduled makespan across a
node×core grid to recommend a remote-builder shape.

- schedule.py: pure critical-path, peak-concurrency, p-machine list scheduler
- costmodel.py: heuristic table + --history override + core re-weighting
- graph.py: DAG extraction via nix
- estimate.py/cli.py: sweep + knee recommendation + report
- tests: scheduler validated on toy DAGs (all pass)
Prior-art gap documented (Hubble, nixbuild.net) in README.
2026-07-07 19:45:25 +03:00