Wire the recommendation's RAM estimate into provisioned ramGb #19

New Issue

2026-07-08T20:44:06+03:00

issuer-agent commented

2026-07-08 20:44:06 +03:00

--provision currently sets each EphemeralBuilder's spec.requirements.ramGb from a coarse default (2 GiB/core × cores) unless --node-ram-gb is passed explicitly. But since #15 the estimator already models per-derivation RAM (costmodel.ram_gb + RAM_HEAVY) and the scheduler tracks peak concurrent RAM per node. We should feed that modeled peak-RAM-per-node into the provisioned ramGb so the emitted CR requests enough memory for the actual build, instead of a flat per-core guess.

Implementation:

In estimate.py, compute a per-node peak RAM figure for the recommended shape (the max over the schedule of sum(gb_per_job of co-resident jobs on a node) — the scheduler's RAM accounting from #15 already has the pieces; expose it, e.g. add a peak_ram_gb_per_node to the Estimate or a helper alongside schedule_for). Use costmodel.ram_gb for per-drv needs and honor max_jobs.
In cli.py --provision, pass ram_gb = the modeled peak-per-node (rounded up, min 1) when --node-ram-gb is not given; an explicit --node-ram-gb still overrides. Keep the 2 GiB/core path only as a final fallback when no model figure is available (e.g. nothing to build).
Add tests: a closure with a RAM-heavy derivation yields a provisioned ramGb >= that derivation's ram_gb (not just 2×cores); explicit --node-ram-gb still wins; fallback path when to_build is empty.

Context: follow-up to #14/#15, discovered while reconciling the provision renderer.

`--provision` currently sets each EphemeralBuilder's `spec.requirements.ramGb` from a coarse default (2 GiB/core × cores) unless `--node-ram-gb` is passed explicitly. But since #15 the estimator already models per-derivation RAM (`costmodel.ram_gb` + `RAM_HEAVY`) and the scheduler tracks peak concurrent RAM per node. We should feed that modeled peak-RAM-per-node into the provisioned `ramGb` so the emitted CR requests enough memory for the actual build, instead of a flat per-core guess. **Implementation**: - In estimate.py, compute a per-node peak RAM figure for the recommended shape (the max over the schedule of sum(gb_per_job of co-resident jobs on a node) — the scheduler's RAM accounting from #15 already has the pieces; expose it, e.g. add a `peak_ram_gb_per_node` to the Estimate or a helper alongside `schedule_for`). Use `costmodel.ram_gb` for per-drv needs and honor `max_jobs`. - In cli.py `--provision`, pass `ram_gb` = the modeled peak-per-node (rounded up, min 1) when `--node-ram-gb` is not given; an explicit `--node-ram-gb` still overrides. Keep the 2 GiB/core path only as a final fallback when no model figure is available (e.g. nothing to build). - Add tests: a closure with a RAM-heavy derivation yields a provisioned ramGb >= that derivation's ram_gb (not just 2×cores); explicit --node-ram-gb still wins; fallback path when to_build is empty. **Context**: follow-up to #14/#15, discovered while reconciling the provision renderer.

admin added the enhancement label 2026-07-08 20:44:57 +03:00

oleks referenced this issue from a commit

2026-07-08 20:53:37 +03:00

provision: size ramGb from modeled peak per-node RAM (#19)

oleks referenced this issue from a commit

2026-07-08 20:53:37 +03:00

Merge ramGb-from-model wiring (#19)

oleks commented

2026-07-08 20:54:19 +03:00

Implemented on main (HEAD 4016f78, 86 tests passing). Added estimate.peak_ram_gb_per_node(...) which runs the recommended shape's schedule uncapped and sweep-lines each node's task intervals (grouping lanes by lane // max_jobs), summing costmodel.ram_gb per co-resident build to find the busiest node's peak concurrent RAM. --provision now sizes each EphemeralBuilder's spec.requirements.ramGb from that modeled peak (rounded up, min 1) instead of the flat 2 GiB/core default; an explicit --node-ram-gb still overrides, and a nothing-to-build closure falls back to the per-core default. Verified end-to-end: a chromium-containing closure now emits ramGb: 10 (its modeled peak) rather than 2×cores. Tests cover the heavy-derivation sizing, co-resident RAM summation under max_jobs, the explicit-override path, and the empty-closure fallback.

Implemented on main (HEAD 4016f78, 86 tests passing). Added `estimate.peak_ram_gb_per_node(...)` which runs the recommended shape's schedule uncapped and sweep-lines each node's task intervals (grouping lanes by `lane // max_jobs`), summing `costmodel.ram_gb` per co-resident build to find the busiest node's peak concurrent RAM. `--provision` now sizes each EphemeralBuilder's `spec.requirements.ramGb` from that modeled peak (rounded up, min 1) instead of the flat 2 GiB/core default; an explicit `--node-ram-gb` still overrides, and a nothing-to-build closure falls back to the per-core default. Verified end-to-end: a chromium-containing closure now emits `ramGb: 10` (its modeled peak) rather than 2×cores. Tests cover the heavy-derivation sizing, co-resident RAM summation under max_jobs, the explicit-override path, and the empty-closure fallback.

oleks closed this issue

2026-07-08 20:54:23 +03:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: oleks/nix-estimator#19

Wire the recommendation's RAM estimate into provisioned ramGb #19

Wire the recommendation's RAM estimate into provisioned `ramGb` #19