analytics-playbook

Entrypoints to the token-usage and plugin-workflow tooling shipped by the plugins under ~/projects/claude-plugins, plus one playbook that runs all of it and leaves a digestible SUMMARY.md behind.

Not a plugin — a sibling of the plugin checkouts that drives their bin/ tools.

Requirements

• just and xonsh on PATH (NixOS: add both to environment.systemPackages)
• the plugin checkouts as siblings: ../token-budget, ../plugin-analytics, ../inference-arbitrage, ../plugin-publishing, ../hook-instrument, ../decision-flow, ../agents-routing, ../load-trend, ../hooks-to-go
• go for the load-trend build; claude for the skill/insights recipes

Running from a git worktree breaks the sibling assumption: the default root resolves to <repo>/.claude/worktrees, which holds no checkouts. Set PLUGIN_ROOT for the recipes, or pass --root to playbook.xsh directly.

Use

just                 # list recipes
just extractions     # enumerate EVERY extraction, and which ones playbook skips
just playbook        # collect into out/<timestamp>/ (PROFILES/DAYS/PLUGINS override)
just series          # run the extractions in dependency order (see below)
just digest          # print out/latest/SUMMARY.md — the agent's reading entrypoint
just insights        # optional: claude -p ranks the summary (costs tokens)

just playbook is not the whole surface

It collects the deterministic groups. It is not every extraction this workspace can perform, and for a long time nothing said so — 17 of the 20 enumerated extractions are outside it, including the entire model-driven half, the hooks-to-go checks, the audit-snapshot store, and the routing census. just extractions prints the list with that in playbook column.

just series runs them in dependency order, in six phases:

phase	what	why here
P0	loadtrend start, pa scan	cannot be back-filled — a run started before the load collector records an empty load/ group, and you cannot sample a moment that has passed
P1	playbook --route-sample 0	the collection pass, taking a routing census rather than the newest-5 sample
P2	snapshot list, prune dry-run, digest, h2g	deterministic, reads the workspace rather than the run
P3	offload-audit then the other claude -p passes	costs tokens, needs --with-model
P4	boundary-classify, stability-classify, filing-plan	deterministic scripts whose input only P3 produces — this is why playbook.xsh excludes them on purpose, and why snapshots.json is empty in a workspace that has never run P3
P5	ia-snapshot diff	needs two snapshots, so needs P3 twice

Phases run in order; within a phase, order is irrelevant and a failing extraction is recorded rather than fatal, because a partial dataset beats none. Extractions needing a caller-supplied argument (a target repo, a judgements file) are listed as MANUAL and stood down rather than guessed at.

Per-tool recipes are grouped by plugin prefix: tokens* (token-budget), pa* (plugin-analytics), ia* (inference-arbitrage), pp* (plugin-publishing checks — never the prune/update), hi* (hook-instrument cost and wiring), df*/route-stats (decision-flow, agents-routing), load* (load-trend), h2g (hooks-to-go xsh checks), and skill <name> for the prompt-only skills (reflection:*, anti-patterns:*, decision-flow:session-audit, …) via claude -p.

What the playbook produces

out/<stamp>/
  SUMMARY.md          run table, failures, hogs + composition + context curves,
                      pa dashboard, offload digests, publishing drift, guard
                      precision, hook cost + wiring, routing, host load — each
                      section led by the population it covered
  manifest.json       rc / stderr / duration per step, plus a `coverage` block
  tokens/<profile>/   cc-tokens daily|sessions|projects|models|composition|hogs|payloads
                      (.json + .txt), and context/<session>.json per flagged hog
  pa/                 inventory|utilization|cost|hooks|firings|triggers|hygiene|tags|
                      dupes|heatmap (.json) + report.md
  ia/<plugin>/        inventory.json, scan.json, digest.txt, snapshots.json
  pp/                 stale-grants, cached-binaries, stranded-content (.json),
                      clones, freshness
  df/                 guard-precision.json
  route/<profile>/    route-stats per sampled transcript
  hi/                 instrument report / failures / bursts / gaps, wiring/<plugin>.json
  load/               loadtrend status + analyze
out/latest -> <stamp>

The playbook is collection-only: no claude -p, no prune, no update, no deploy. Every step is independent; a missing or failing tool is recorded in the run table as a blind spot, never fatal.

Coverage is recorded, not assumed

Every group states the population it covered and how its subjects were chosen, in manifest.json's coverage block and at the head of each SUMMARY.md section. This exists because a sample used to render as fleet telemetry: the route group reported ten transcripts with no n and no denominator, and ia scanned only the first profile while tokens and pa covered every one (oleks/analytics-playbook#1, oleks/analytics-playbook#3).

• route samples the newest transcripts per profile — recency, not randomness. --route-sample 0 runs a census instead; empty (turns: 0) transcripts are reported separately and excluded from n.
• ia passes every profile as a repeated --root, and the summary prints the coverage.roots each scan.json reports it actually read — not what was asked for. candidate-digest emits a bare array with no coverage field by contract, so digests inherit that table rather than stating their own.
• hi's instrument report --all-profiles reads every profile log on the host, which is not restricted to the profiles passed with --profile.

What it deliberately does not run

inference-arbitrage's boundary-classify, stability-classify and filing-plan are excluded on purpose. They are deterministic, but they consume candidate judgments whose determinism tests are, in the tool's own words, "irreducibly semantic" and "judged by the analyst agent". Running them here would either break the collection-only property or grade fabricated input. Producing those judgments is the inference-arbitrage:offload-audit skill's job (oleks/analytics-playbook#2).

playbook.xsh is written in xonsh's Python subset, so python3 playbook.xsh runs it identically where xonsh is absent.