ci(performance): fail when the web app's initial bytes grow (#1694)

Third step of the performance-journeys ratchet, stacked on #1693 (which is stacked on #1692; merge in order, GitHub retargets each to `master`).

- New `Initial bytes ratchet` job in `.github/workflows/ci.yml` (ubuntu-latest): install, `pnpm nx build web --skip-nx-cache` (production configuration, the one users download), then `pnpm run perf:initial-bytes:check`. The job fails when `renderer.initialBytes` exceeds `tools/performance/journey-baselines.json`.
- `dist/performance/` is uploaded as the `performance-journey-summary` artifact on every run, so a failing or tightenable run carries its evidence.
- After review: the job first runs the new `tools/performance/check-baseline-direction.mjs`, which compares `journey-baselines.json` with the revision the change is measured against (the target branch of a pull request, `github.event.before` for a `master` push, `master` for a manual dispatch) and fails on any raised enforced limit (`value × toleranceRatio`), any widened or newly added tolerance, or any removed entry, so a PR cannot grow the payload and raise the baseline to match (lowered limits and new entries pass; a target branch without the file has nothing to weaken). Node tests cover it.
- Docs: the performance-journeys contract and the validation map name the job, and the contract now states that this runner is the canonical measurer (take baseline values from its output, not from a local build).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
4grayandClaude Fable 5.1 authored and GitHub committed 2026-09-26 13:44:01 +02:00
1 parent 7abaad29e7
commit d3b6e548cc
8 files changed
+511 -11

No files matched your search

+29 -6
View File
@@ -2,12 +2,10 @@
IPTVnator measures performance through a small set of everyday user journeys.
Each journey has deterministic counters that are asserted exactly, and
wall-clock timings that are recorded as evidence. Counters are meant to be
ratcheted in CI: a committed baseline that may only be lowered, and only with
the measured output as evidence. This document is the contract for that loop;
`tools/performance/` holds the scripts. The measurement script lands first;
the baseline file and the CI job follow in their own PRs (#1693, #1694), so
until they merge the reported number is informational, not enforced.
wall-clock timings that are recorded as evidence. Counters are ratcheted in CI:
a committed baseline may only be lowered, and only with the measured output as
evidence. This document is the contract for that loop; `tools/performance/`
holds the scripts.
## Journeys
@@ -104,6 +102,31 @@ pnpm run perf:initial-bytes:check # measure dist/apps/web into dist/performanc
pnpm run perf:ratchet:check # check every baseline against dist/performance/journey-summary.json
```
CI runs `perf:initial-bytes:check` in the `Initial bytes ratchet` job of
`.github/workflows/ci.yml` after a production build of `apps/web`, and uploads
`dist/performance/` as the `performance-journey-summary` artifact. Like the
rest of that workflow it runs for pull requests that target `master` and for
pushes to `master`; a stacked PR that targets another branch gets no run until
it is retargeted, so dispatch one with `gh workflow run ci.yml --ref <branch>`
when you need the number. A PR that grows the counter fails that job.
That runner is the canonical measurer: take baseline values from its output,
not from a local build. A local macOS build of the code before #1695 is 2
bytes smaller in `main.js` (the eager locale imports); since #1695 the two
have been byte-identical. (An apparent 556-byte platform difference during
the first measurements was otherwise `package.json` text embedded in
`main.js`, which moved with every script edit; #1692 fixed that by importing
only the version.)
The job also refuses a weakened baselines file:
`tools/performance/check-baseline-direction.mjs` compares
`journey-baselines.json` with the revision the change is measured against
(the target branch of a pull request, the previous head of a `master` push,
`master` for a manual dispatch) and fails when any
entry's enforced limit (`value × toleranceRatio`) went up, a tolerance widened
or an entry disappeared, so a PR cannot grow the payload and raise the
baseline to match. Lowered limits and new entries pass.
Baselines only move down. Lower `value` in the same PR as the change that
earned it, set `updatedAt` and `evidencePr`, and paste the measurement output
into the PR. Never raise a value to make a PR pass: if growth is a deliberate
+4 -2
View File
@@ -165,8 +165,10 @@ pnpm nx test performance-tools
`perf:initial-bytes` reads the built `dist/apps/web/index.html` and sums the
bytes on the initial path (the J1 counter `renderer.initialBytes`).
`perf:initial-bytes:check` then fails if the value exceeds
`tools/performance/journey-baselines.json`; baselines only move down. The
contract, what counts and how to add a counter are in the
`tools/performance/journey-baselines.json`; baselines only move down. CI runs
the same check in the `Initial bytes ratchet` job of `ci.yml` for PRs that
target `master` and for `master` pushes (dispatch it with
`gh workflow run ci.yml --ref <branch>` for a stacked branch). The contract, what counts and how to add a counter are in the
[performance journeys](performance-journeys.md) document.
## Logging