mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-08 17:06:15 -08:00
ci(perf): add the weekly baseline tightening workflow (#1760)
This commit is contained in:
1 parent
97f56e219c
commit
ec8b931dbf
9 files changed
+1142
-58
No files matched your search
@@ -628,7 +628,9 @@ the job for every PR until #1734. Treat the job as blocking before merging;
|
||||
making it a required check is a maintainer decision.
|
||||
|
||||
The runtime counters come from the `Performance journeys` job of the same
|
||||
workflow, on `ubuntu-latest` only. It runs `pnpm run perf:journeys` under
|
||||
workflow, on `ubuntu-latest` only. Through the
|
||||
`.github/actions/performance-journeys` composite action it runs
|
||||
`pnpm run perf:journeys` under
|
||||
`xvfb-run` (the Nx target builds `electron-backend:build-performance`, the
|
||||
Playwright config starts the Xtream mock), writes the measurements to the job
|
||||
summary and uploads `dist/performance/journeys/` as the `performance-journeys`
|
||||
@@ -661,6 +663,51 @@ it as `stable: false` because the dashboard flicker it reports is a race
|
||||
there (see [Settle window](#settle-window)). Add the runner's number once
|
||||
that flicker is fixed and the counter is deterministic.
|
||||
|
||||
### Weekly tightening
|
||||
|
||||
`.github/workflows/performance-ratchet.yml` lowers baselines without waiting
|
||||
for someone to act on a "tighten" hint. Every Monday, and on
|
||||
`workflow_dispatch`, three `ubuntu-latest` jobs measure the same commit
|
||||
independently: the production `apps/web` build with
|
||||
`measure-initial-bytes.mjs --summary`, then the journeys through the
|
||||
`.github/actions/performance-journeys` composite action, which the
|
||||
`Performance journeys` job above uses too. A failed journey run does not stop
|
||||
its job; its entries are then unmeasured in that run. A final job runs
|
||||
`tools/performance/tighten-baselines.mjs` on the three runs:
|
||||
|
||||
- an entry is lowered only when every run measured it and every measurement
|
||||
is strictly below `value`; the new `value` is the largest of the three (for
|
||||
a wall-clock entry the largest per-run summary value, which is the largest
|
||||
P50 for a `.p50` entry);
|
||||
- a counter marked `counterStability.<name>.stable: false` in any run is
|
||||
kept, and the report says which run and which iterations disagreed;
|
||||
- `value` never goes up, `slack` and `toleranceRatio` never change, and no
|
||||
entry is added or removed: the result must pass
|
||||
`check-baseline-direction.mjs` without `--allow-increase`, which both the
|
||||
script and the job check;
|
||||
- a lowered entry gets `updatedAt`, `measuredWith` and `evidenceRun` (the
|
||||
workflow run URL); `evidencePr` is set to the tightening PR once it exists.
|
||||
|
||||
When the file changed and the run is on `master`, the job pushes
|
||||
`automation/performance-ratchet` and opens (or updates) a pull request with
|
||||
the per-run table, the diff and the run URL, labelled `no-release-note`. It
|
||||
pushes with the existing `PAT` secret, as the Windows MPV pin refresh does,
|
||||
because a pull request pushed with `GITHUB_TOKEN` starts no CI. Each run
|
||||
replaces the branch with one fresh commit, except when the open tightening
|
||||
pull request carries a commit the workflow did not make (a review edit, an
|
||||
"Update branch" merge): then it leaves the branch alone with a warning, and
|
||||
the numbers stay in the job summary. When no
|
||||
baseline was below its value in all three runs, the workflow ends without a
|
||||
pull request. A dispatch on another branch measures and prints the diff but
|
||||
never opens one, so `gh workflow run performance-ratchet.yml --ref <branch>`
|
||||
validates a change to the workflow once the file is on `master`. GitHub only
|
||||
dispatches workflows that exist on the default branch, so before the first
|
||||
merge of a new or renamed workflow add a temporary `push` trigger for the
|
||||
branch and drop it before review, as #1760 did. Review the pull request like a manual
|
||||
tightening: if `master` moved since the measured commit, the
|
||||
`Initial bytes ratchet` job on the pull request is what shows that the new
|
||||
value still holds (the concurrent-merge effect above).
|
||||
|
||||
## Charset parse benchmark
|
||||
|
||||
V8 stores a string as two-byte UTF-16 once one character falls outside
|
||||
|
||||
@@ -260,7 +260,12 @@ bytes on the initial path (the J1 counter `renderer.initialBytes`).
|
||||
`tools/performance/journey-baselines.json`; baselines only move down. CI runs
|
||||
the same check in the `Initial bytes ratchet` job of `ci.yml` for PRs that
|
||||
target `master` and for `master` pushes (dispatch it with
|
||||
`gh workflow run ci.yml --ref <branch>` for a stacked branch). `perf:journeys` builds the `electron-performance` configuration and runs every
|
||||
`gh workflow run ci.yml --ref <branch>` for a stacked branch). The weekly
|
||||
`performance-ratchet.yml` workflow lowers baselines through a bot PR; validate
|
||||
a change to it with `gh workflow run performance-ratchet.yml --ref <branch>`,
|
||||
which measures but opens no PR off `master`. Dispatch needs the workflow file
|
||||
on `master`; before that, see the temporary-trigger note under Weekly
|
||||
tightening in the performance journeys document. `perf:journeys` builds the `electron-performance` configuration and runs every
|
||||
journey spec against the Xtream mock: J1 launch, then J2 open-source (a
|
||||
second set of launches, each followed by the click on the portal card), both
|
||||
written to the same summary file; its probe specs run with
|
||||
|
||||
Reference in new issue
Block a user