mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-08 17:06:15 -08:00
ci(performance): fail when the web app's initial bytes grow (#1694)
Third step of the performance-journeys ratchet, stacked on #1693 (which is stacked on #1692; merge in order, GitHub retargets each to `master`). - New `Initial bytes ratchet` job in `.github/workflows/ci.yml` (ubuntu-latest): install, `pnpm nx build web --skip-nx-cache` (production configuration, the one users download), then `pnpm run perf:initial-bytes:check`. The job fails when `renderer.initialBytes` exceeds `tools/performance/journey-baselines.json`. - `dist/performance/` is uploaded as the `performance-journey-summary` artifact on every run, so a failing or tightenable run carries its evidence. - After review: the job first runs the new `tools/performance/check-baseline-direction.mjs`, which compares `journey-baselines.json` with the revision the change is measured against (the target branch of a pull request, `github.event.before` for a `master` push, `master` for a manual dispatch) and fails on any raised enforced limit (`value × toleranceRatio`), any widened or newly added tolerance, or any removed entry, so a PR cannot grow the payload and raise the baseline to match (lowered limits and new entries pass; a target branch without the file has nothing to weaken). Node tests cover it. - Docs: the performance-journeys contract and the validation map name the job, and the contract now states that this runner is the canonical measurer (take baseline values from its output, not from a local build). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
7abaad29e7
commit
d3b6e548cc
8 files changed
+511
-11
No files matched your search
@@ -2,12 +2,10 @@
|
||||
|
||||
IPTVnator measures performance through a small set of everyday user journeys.
|
||||
Each journey has deterministic counters that are asserted exactly, and
|
||||
wall-clock timings that are recorded as evidence. Counters are meant to be
|
||||
ratcheted in CI: a committed baseline that may only be lowered, and only with
|
||||
the measured output as evidence. This document is the contract for that loop;
|
||||
`tools/performance/` holds the scripts. The measurement script lands first;
|
||||
the baseline file and the CI job follow in their own PRs (#1693, #1694), so
|
||||
until they merge the reported number is informational, not enforced.
|
||||
wall-clock timings that are recorded as evidence. Counters are ratcheted in CI:
|
||||
a committed baseline may only be lowered, and only with the measured output as
|
||||
evidence. This document is the contract for that loop; `tools/performance/`
|
||||
holds the scripts.
|
||||
|
||||
## Journeys
|
||||
|
||||
@@ -104,6 +102,31 @@ pnpm run perf:initial-bytes:check # measure dist/apps/web into dist/performanc
|
||||
pnpm run perf:ratchet:check # check every baseline against dist/performance/journey-summary.json
|
||||
```
|
||||
|
||||
CI runs `perf:initial-bytes:check` in the `Initial bytes ratchet` job of
|
||||
`.github/workflows/ci.yml` after a production build of `apps/web`, and uploads
|
||||
`dist/performance/` as the `performance-journey-summary` artifact. Like the
|
||||
rest of that workflow it runs for pull requests that target `master` and for
|
||||
pushes to `master`; a stacked PR that targets another branch gets no run until
|
||||
it is retargeted, so dispatch one with `gh workflow run ci.yml --ref <branch>`
|
||||
when you need the number. A PR that grows the counter fails that job.
|
||||
|
||||
That runner is the canonical measurer: take baseline values from its output,
|
||||
not from a local build. A local macOS build of the code before #1695 is 2
|
||||
bytes smaller in `main.js` (the eager locale imports); since #1695 the two
|
||||
have been byte-identical. (An apparent 556-byte platform difference during
|
||||
the first measurements was otherwise `package.json` text embedded in
|
||||
`main.js`, which moved with every script edit; #1692 fixed that by importing
|
||||
only the version.)
|
||||
|
||||
The job also refuses a weakened baselines file:
|
||||
`tools/performance/check-baseline-direction.mjs` compares
|
||||
`journey-baselines.json` with the revision the change is measured against
|
||||
(the target branch of a pull request, the previous head of a `master` push,
|
||||
`master` for a manual dispatch) and fails when any
|
||||
entry's enforced limit (`value × toleranceRatio`) went up, a tolerance widened
|
||||
or an entry disappeared, so a PR cannot grow the payload and raise the
|
||||
baseline to match. Lowered limits and new entries pass.
|
||||
|
||||
Baselines only move down. Lower `value` in the same PR as the change that
|
||||
earned it, set `updatedAt` and `evidencePr`, and paste the measurement output
|
||||
into the PR. Never raise a value to make a PR pass: if growth is a deliberate
|
||||
|
||||
@@ -165,8 +165,10 @@ pnpm nx test performance-tools
|
||||
`perf:initial-bytes` reads the built `dist/apps/web/index.html` and sums the
|
||||
bytes on the initial path (the J1 counter `renderer.initialBytes`).
|
||||
`perf:initial-bytes:check` then fails if the value exceeds
|
||||
`tools/performance/journey-baselines.json`; baselines only move down. The
|
||||
contract, what counts and how to add a counter are in the
|
||||
`tools/performance/journey-baselines.json`; baselines only move down. CI runs
|
||||
the same check in the `Initial bytes ratchet` job of `ci.yml` for PRs that
|
||||
target `master` and for `master` pushes (dispatch it with
|
||||
`gh workflow run ci.yml --ref <branch>` for a stacked branch). The contract, what counts and how to add a counter are in the
|
||||
[performance journeys](performance-journeys.md) document.
|
||||
|
||||
## Logging
|
||||
|
||||
Reference in new issue
Block a user