playbook reports a finished load trial as the run's current load findings #5

New Issue

2026-09-12T15:29:16+03:00

issuer-agent commented

2026-09-12 15:29:16 +03:00

What

collect_load in playbook.xsh runs only the collector's status and analyze subcommands against the default trial directory. Nothing in the pipeline ever starts the collector. Starting it is listed as a P0 prerequisite in series.xsh that the playbook itself does not perform.

The second half is what makes it persistent. loadtrend start refuses to append to samples older than its staleness window (checkStaleData in cmd/loadtrend/daemon.go), because the analyzer fits a trend first-sample-to-last and appending across an idle gap would turn the gap into the slope. That refusal is correct. But nothing rotates a finished trial aside, so the directory stays unstartable and every later run re-reads the same dead trial.

Evidence (2026-09-12)

Run out/20260912-105825 rendered, in its load section, 2457 aggregate samples spanning 20h27m10s with CPU PSI stall bursts up to 85% and a memory-reclaim stall — from a trial whose newest sample was 73h45m old. On emmett the equivalent trial ended 87h17m before its run. Neither host had a collector running.

The status output does print a STALE: line, so the data is labelled. But it is still rendered inside the run's load section beside genuinely fresh sections, and the summary's own reading order tells the reader to start from the Not ok list, which was empty.

Fix direction

Either of:

have collect_load detect a stale trial and render a refusal in place of the analysis, rather than the analysis of dead data; or
rotate a finished trial aside and start a collector as part of P0.

A rotation arm now exists at ~/claude-arms/loadtrend-trial-rotate.go (dry-run by default, moves never deletes, refuses a trial that is still live by sample age or by live sampler pid). It was used to unstick both hosts, and fresh collectors are now running on carry and emmett.

Limits

n=2 hosts, one observed run each. I did not test whether an in-progress trial younger than the staleness window renders correctly.

## What `collect_load` in `playbook.xsh` runs only the collector's `status` and `analyze` subcommands against the default trial directory. Nothing in the pipeline ever starts the collector. Starting it is listed as a P0 prerequisite in `series.xsh` that the playbook itself does not perform. The second half is what makes it persistent. `loadtrend start` refuses to append to samples older than its staleness window (`checkStaleData` in `cmd/loadtrend/daemon.go`), because the analyzer fits a trend first-sample-to-last and appending across an idle gap would turn the gap into the slope. That refusal is correct. But nothing rotates a finished trial aside, so the directory stays unstartable and every later run re-reads the same dead trial. ## Evidence (2026-09-12) Run `out/20260912-105825` rendered, in its load section, `2457 aggregate samples spanning 20h27m10s` with CPU PSI stall bursts up to 85% and a memory-reclaim stall — from a trial whose newest sample was **73h45m old**. On emmett the equivalent trial ended 87h17m before its run. Neither host had a collector running. The status output does print a `STALE:` line, so the data is labelled. But it is still rendered inside the run's load section beside genuinely fresh sections, and the summary's own reading order tells the reader to start from the Not ok list, which was empty. ## Fix direction Either of: - have `collect_load` detect a stale trial and render a refusal in place of the analysis, rather than the analysis of dead data; or - rotate a finished trial aside and start a collector as part of P0. A rotation arm now exists at `~/claude-arms/loadtrend-trial-rotate.go` (dry-run by default, moves never deletes, refuses a trial that is still live by sample age or by live sampler pid). It was used to unstick both hosts, and fresh collectors are now running on carry and emmett. ## Limits n=2 hosts, one observed run each. I did not test whether an in-progress trial younger than the staleness window renders correctly.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: oleks/analytics-playbook#5