ci(perf): add the weekly baseline tightening workflow

Plan item B4. performance-ratchet.yml measures master on three runners
(initial bytes and the journeys) and tighten-baselines.mjs lowers every
baseline that all three runs beat, opening a no-release-note PR only when
the file changed. The journeys steps move into a composite action shared
with ci.yml's Performance journeys job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
4grayandClaude Opus 5.5 committed 2026-09-29 22:52:13 +02:00
1 parent 963a431bb7
commit a7f1ecfe49
9 files changed
+1110 -58

No files matched your search

+41 -1
View File
@@ -525,7 +525,9 @@ the job for every PR until #1734. Treat the job as blocking before merging;
making it a required check is a maintainer decision.
The runtime counters come from the `Performance journeys` job of the same
workflow, on `ubuntu-latest` only. It runs `pnpm run perf:journeys` under
workflow, on `ubuntu-latest` only. Through the
`.github/actions/performance-journeys` composite action it runs
`pnpm run perf:journeys` under
`xvfb-run` (the Nx target builds `electron-backend:build-performance`, the
Playwright config starts the Xtream mock), writes the measurements to the job
summary and uploads `dist/performance/journeys/` as the `performance-journeys`
@@ -554,6 +556,44 @@ in all eighteen runner iterations; the `spawnToFirstCardMs` P50 ranged from
differ from a Mac (12 and 571 there, the fast path without the Linux-only
`getWindowState` call), so take J1 baseline values from the runner only.
### Weekly tightening
`.github/workflows/performance-ratchet.yml` lowers baselines without waiting
for someone to act on a "tighten" hint. Every Monday, and on
`workflow_dispatch`, three `ubuntu-latest` jobs measure the same commit
independently: the production `apps/web` build with
`measure-initial-bytes.mjs --summary`, then the journeys through the
`.github/actions/performance-journeys` composite action, which the
`Performance journeys` job above uses too. A failed journey run does not stop
its job; its entries are then unmeasured in that run. A final job runs
`tools/performance/tighten-baselines.mjs` on the three runs:
- an entry is lowered only when every run measured it and every measurement
is strictly below `value`; the new `value` is the largest of the three (for
a wall-clock entry the largest per-run summary value, which is the largest
P50 for a `.p50` entry);
- a counter marked `counterStability.<name>.stable: false` in any run is
kept, and the report says which run and which iterations disagreed;
- `value` never goes up, `slack` and `toleranceRatio` never change, and no
entry is added or removed: the result must pass
`check-baseline-direction.mjs` without `--allow-increase`, which both the
script and the job check;
- a lowered entry gets `updatedAt`, `measuredWith` and `evidenceRun` (the
workflow run URL); `evidencePr` is set to the tightening PR once it exists.
When the file changed and the run is on `master`, the job pushes
`automation/performance-ratchet` and opens (or updates) a pull request with
the per-run table, the diff and the run URL, labelled `no-release-note`. It
pushes with the existing `PAT` secret, as the Windows MPV pin refresh does,
because a pull request pushed with `GITHUB_TOKEN` starts no CI. When no
baseline was below its value in all three runs, the workflow ends without a
pull request. A dispatch on another branch measures and prints the diff but
never opens one; use `gh workflow run performance-ratchet.yml --ref <branch>`
to validate a change to the workflow. Review the pull request like a manual
tightening: if `master` moved since the measured commit, the
`Initial bytes ratchet` job on the pull request is what shows that the new
value still holds (the concurrent-merge effect above).
## Charset parse benchmark
V8 stores a string as two-byte UTF-16 once one character falls outside
+4 -1
View File
@@ -260,7 +260,10 @@ bytes on the initial path (the J1 counter `renderer.initialBytes`).
`tools/performance/journey-baselines.json`; baselines only move down. CI runs
the same check in the `Initial bytes ratchet` job of `ci.yml` for PRs that
target `master` and for `master` pushes (dispatch it with
`gh workflow run ci.yml --ref <branch>` for a stacked branch). `perf:journeys` builds the `electron-performance` configuration and runs every
`gh workflow run ci.yml --ref <branch>` for a stacked branch). The weekly
`performance-ratchet.yml` workflow lowers baselines through a bot PR; validate
a change to it with `gh workflow run performance-ratchet.yml --ref <branch>`,
which measures but opens no PR off `master`. `perf:journeys` builds the `electron-performance` configuration and runs every
journey spec against the Xtream mock: J1 launch, then J2 open-source (a
second set of launches, each followed by the click on the portal card), both
written to the same summary file; its probe specs run with