Commit Graph
3 Commits
Author SHA1 Message Date
0a006cb027 ci(perf): enforce the journey counters stable on master (#1829)
* ci(perf): enforce the journey counters stable on master

Promote the J1, J2 and J3 counters that were identical in all 55 measured
iterations of the 11 master runs from 2026-10-03 to 2026-10-04 to
journey-baselines.json, and check them in the Performance journeys job
(still warn-only), in one step together with #1828's two validated J1
entries. None of the new ones has Principle 3 evidence, so each carries a
"guard only, not validated" note that the checker prints with a failure.
A performance-tools test keeps the job's --only list equal to the journey
entries. Number formatting uses three decimals, the precision of the
layout-shift scores.

J2 renderer.layoutShiftScore is 0.233, not the window's 0.222: every
master run from #1814 (page Back buttons in the header) on reads 0.233.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci(perf): check the journey counters whenever a summary was written

Review follow-up (Greptile): the check ran only after the composite
action succeeded, so a failed job-summary report after a written
summary.json skipped every baseline. It now runs unless the job was
cancelled, as long as the action produced a summary path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: 4gray <fourgray@proton.me>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 11:43:55 +02:00
4grayandClaude Opus 5.5 af44e2368b ci(performance): give the initial-bytes ratchet slack and a labelled override (#1744)
* ci(performance): give the initial-bytes ratchet slack and a labelled override

The exact renderer.initialBytes counter failed PRs for reasons outside
their diff: two concurrent merges left master 108 bytes over the baseline
for hours, and bundler identifier renaming moves the counter by hundreds of
bytes. PRs growing it by 243 and 302 bytes had no way to pass at all,
because the direction check refuses any raised baseline.

- Counter entries accept `slack` (integer, entry unit): the ratchet enforces
  `value + slack`, reports how much slack a measurement uses, and still
  prints the tighten hint below `value`. renderer.initialBytes gets 4096
  bytes, so growth can accumulate at most 4 KiB past the last lowered
  baseline while regressions such as +35 KB still fail.
- check-baseline-direction.mjs compares `value + slack`, treats widened
  slack like a widened tolerance, and takes `--allow-increase`, which
  reports weakened entries as ALLOWED instead of failing.
- CI passes `--allow-increase` only when the pull request (or, for a master
  push, the pull request merged as the pushed commit) carries the
  perf-baseline-increase label, read from the API so a job re-run picks up
  a label added later.

This change widens the slack itself, so its own PR needs the label.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0166PWobUjpoiWt8E9bBdsCs

* fix(performance): report slack usage only for entries that have slack

A wall-clock measurement above `value` but within `value × toleranceRatio`
fell into the slack branch and was reported as using "slack", conflating
timing tolerance with counter slack. Only entries with `slack` report it now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0166PWobUjpoiWt8E9bBdsCs

* fix(performance): scope the push-time label and refuse raised counter values

- A master push compares the whole push, but the label was read from the
  head commit's PR only, so one labelled PR could cover another commit's
  increase in the same push. The label now counts only when the push added
  exactly one first-parent commit (a squash or merge of one PR); any other
  push that weakens a baseline fails.
- Raising a counter's `value` while narrowing its `slack` lowered the
  enforced limit and was reported as "lowered". A counter's value is the
  measured evidence and only moves down, so that raise is now a weakening
  that needs the label.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0166PWobUjpoiWt8E9bBdsCs

* fix(performance): treat a counter/wall-clock type switch as a weakening

Turning `{ value: 100, slack: 10 }` into `{ value: 105, toleranceRatio: 1 }`
skipped the raised-counter-value rule and was reported as a lowered limit.
Switching an entry between counter and wall-clock now needs the
perf-baseline-increase label.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0166PWobUjpoiWt8E9bBdsCs

* ci(performance): paginate the label lookups of the direction check

The labels endpoint returns 30 entries per page by default, so a PR with
more labels could miss perf-baseline-increase. Both lookups now request
100 per page and paginate, like the release-note gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0166PWobUjpoiWt8E9bBdsCs

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-28 14:55:15 +02:00
4grayandClaude Fable 5.1 7abaad29e7 chore(performance): commit the initial-bytes baseline and ratchet checker (#1693)
Second step of the performance-journeys ratchet, stacked on #1692 (merge that first; this PR retargets to `master` automatically).

- `tools/performance/journey-baselines.json`: J1 `launch` / `renderer.initialBytes` = **2,739,510 bytes**, the ubuntu runner's production build of `apps/web` at this content (after #1692 stopped bundling `package.json` into `main.js`). A local macOS build of this pre-#1695 code is 2 bytes smaller in `main.js` (the eager locale imports); once #1695 removes them the two are byte-identical. Correction to an earlier version of this description: the "556-byte macOS vs Linux difference" was almost entirely `package.json` text embedded in `main.js`, which moved with every script edit in this stack, plus this 2-byte residue. The runner is the canonical measurer; the CI run on the stacked #1694 branch (this content plus the job) is where the number is confirmed.
- `tools/performance/check-journey-ratchet.mjs` compares a journey summary with the baselines: a counter above its value fails (exact, no slack), wall-clock entries fail above `value × toleranceRatio`, a baseline without a measurement fails so dropping a measurement cannot disable the ratchet, values below baseline print a "tighten" hint, and measured counters without a baseline are noted only. After review: checking nothing (empty file, or `--only` naming a missing entry) fails; a counter is read only from `counters` and a wall-clock entry only from `wallClock`; the repeatable `--only <journey>/<counter>` flag scopes a check.
- Root scripts: `perf:initial-bytes:check` (measures into its own `dist/performance/initial-bytes.summary.json`, then checks `--only launch/renderer.initialBytes`) and `perf:ratchet:check` (full check); `perf:tools:test` runs both test files, as does `pnpm nx test performance-tools`.
- `docs/architecture/performance-journeys.md` gains the Ratchet section (file format, rules, "baselines only move down"); the validation map lists the check.

The CI job that runs the check on every PR is #1694; C1 (lazy Angular date locales, #1695) then lowers the baseline with the measured output as evidence.

Note: `ci.yml` only triggers on pull requests targeting `master`, so this stacked PR shows no Actions runs until #1692 merges. The evidence runs above were dispatched with `gh workflow run ci.yml --ref <branch>`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-26 13:43:33 +02:00