mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-14 05:06:16 -08:00
ci(perf): add the weekly baseline tightening workflow
Plan item B4. performance-ratchet.yml measures master on three runners (initial bytes and the journeys) and tighten-baselines.mjs lowers every baseline that all three runs beat, opening a no-release-note PR only when the file changed. The journeys steps move into a composite action shared with ci.yml's Performance journeys job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
963a431bb7
commit
a7f1ecfe49
9 files changed
+1110
-58
No files matched your search
@@ -525,7 +525,9 @@ the job for every PR until #1734. Treat the job as blocking before merging;
|
||||
making it a required check is a maintainer decision.
|
||||
|
||||
The runtime counters come from the `Performance journeys` job of the same
|
||||
workflow, on `ubuntu-latest` only. It runs `pnpm run perf:journeys` under
|
||||
workflow, on `ubuntu-latest` only. Through the
|
||||
`.github/actions/performance-journeys` composite action it runs
|
||||
`pnpm run perf:journeys` under
|
||||
`xvfb-run` (the Nx target builds `electron-backend:build-performance`, the
|
||||
Playwright config starts the Xtream mock), writes the measurements to the job
|
||||
summary and uploads `dist/performance/journeys/` as the `performance-journeys`
|
||||
@@ -554,6 +556,44 @@ in all eighteen runner iterations; the `spawnToFirstCardMs` P50 ranged from
|
||||
differ from a Mac (12 and 571 there, the fast path without the Linux-only
|
||||
`getWindowState` call), so take J1 baseline values from the runner only.
|
||||
|
||||
### Weekly tightening
|
||||
|
||||
`.github/workflows/performance-ratchet.yml` lowers baselines without waiting
|
||||
for someone to act on a "tighten" hint. Every Monday, and on
|
||||
`workflow_dispatch`, three `ubuntu-latest` jobs measure the same commit
|
||||
independently: the production `apps/web` build with
|
||||
`measure-initial-bytes.mjs --summary`, then the journeys through the
|
||||
`.github/actions/performance-journeys` composite action, which the
|
||||
`Performance journeys` job above uses too. A failed journey run does not stop
|
||||
its job; its entries are then unmeasured in that run. A final job runs
|
||||
`tools/performance/tighten-baselines.mjs` on the three runs:
|
||||
|
||||
- an entry is lowered only when every run measured it and every measurement
|
||||
is strictly below `value`; the new `value` is the largest of the three (for
|
||||
a wall-clock entry the largest per-run summary value, which is the largest
|
||||
P50 for a `.p50` entry);
|
||||
- a counter marked `counterStability.<name>.stable: false` in any run is
|
||||
kept, and the report says which run and which iterations disagreed;
|
||||
- `value` never goes up, `slack` and `toleranceRatio` never change, and no
|
||||
entry is added or removed: the result must pass
|
||||
`check-baseline-direction.mjs` without `--allow-increase`, which both the
|
||||
script and the job check;
|
||||
- a lowered entry gets `updatedAt`, `measuredWith` and `evidenceRun` (the
|
||||
workflow run URL); `evidencePr` is set to the tightening PR once it exists.
|
||||
|
||||
When the file changed and the run is on `master`, the job pushes
|
||||
`automation/performance-ratchet` and opens (or updates) a pull request with
|
||||
the per-run table, the diff and the run URL, labelled `no-release-note`. It
|
||||
pushes with the existing `PAT` secret, as the Windows MPV pin refresh does,
|
||||
because a pull request pushed with `GITHUB_TOKEN` starts no CI. When no
|
||||
baseline was below its value in all three runs, the workflow ends without a
|
||||
pull request. A dispatch on another branch measures and prints the diff but
|
||||
never opens one; use `gh workflow run performance-ratchet.yml --ref <branch>`
|
||||
to validate a change to the workflow. Review the pull request like a manual
|
||||
tightening: if `master` moved since the measured commit, the
|
||||
`Initial bytes ratchet` job on the pull request is what shows that the new
|
||||
value still holds (the concurrent-merge effect above).
|
||||
|
||||
## Charset parse benchmark
|
||||
|
||||
V8 stores a string as two-byte UTF-16 once one character falls outside
|
||||
|
||||
@@ -260,7 +260,10 @@ bytes on the initial path (the J1 counter `renderer.initialBytes`).
|
||||
`tools/performance/journey-baselines.json`; baselines only move down. CI runs
|
||||
the same check in the `Initial bytes ratchet` job of `ci.yml` for PRs that
|
||||
target `master` and for `master` pushes (dispatch it with
|
||||
`gh workflow run ci.yml --ref <branch>` for a stacked branch). `perf:journeys` builds the `electron-performance` configuration and runs every
|
||||
`gh workflow run ci.yml --ref <branch>` for a stacked branch). The weekly
|
||||
`performance-ratchet.yml` workflow lowers baselines through a bot PR; validate
|
||||
a change to it with `gh workflow run performance-ratchet.yml --ref <branch>`,
|
||||
which measures but opens no PR off `master`. `perf:journeys` builds the `electron-performance` configuration and runs every
|
||||
journey spec against the Xtream mock: J1 launch, then J2 open-source (a
|
||||
second set of launches, each followed by the click on the portal card), both
|
||||
written to the same summary file; its probe specs run with
|
||||
|
||||
Reference in new issue
Block a user