From d1839220785b2d2a7bd52cb79f05b1138abe9c3a Mon Sep 17 00:00:00 2001 From: 4gray Date: Tue, 13 Oct 2026 08:16:51 +0200 Subject: [PATCH] ci(perf): make the Performance journeys job blocking Drop continue-on-error from the Performance journeys job (plan item B3). All 71 master runs from 2026-10-06 to 2026-10-12 passed the journey counter check with the baselines of their own commit, and no enforced counter differed between the measured iterations of a run. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 7 ++++--- docs/architecture/performance-journeys.md | 18 +++++++++++++----- docs/architecture/validation-map.md | 2 +- 3 files changed, 18 insertions(+), 9 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b5a581c80..1d1d2fb2a 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -299,9 +299,10 @@ jobs: # (launch, open-source, playback, search) is six fresh Electron # processes plus one seeding run. timeout-minutes: 30 - # Warn-only for the first two weeks of plan item B3: a failure is - # visible on the run but does not fail the workflow. - continue-on-error: true + # Blocking since 2026-10-13 (plan item B3): no master run of the + # preceding week failed the counter check, and no enforced counter + # varied between iterations (see Ratchet in + # docs/architecture/performance-journeys.md). env: NX_SKIP_NX_CACHE: true diff --git a/docs/architecture/performance-journeys.md b/docs/architecture/performance-journeys.md index fac0b08be..853168a85 100644 --- a/docs/architecture/performance-journeys.md +++ b/docs/architecture/performance-journeys.md @@ -1299,9 +1299,18 @@ requests that change nothing but Markdown, `docs/**`, `.plans/**`, `.codex/**`, `.claude/**`, `.changes/**` or `apps/website/**` (the E2E workflow's ignore list plus release notes); any other file, including root build inputs such as `.nvmrc`, `nx.json` or `tsconfig.base.json`, runs it. -Pushes to `master` and manual dispatches always run it. The job is warn-only (`continue-on-error: true`) for its first two -weeks (plan item B3): a regression marks the job failed without failing the -workflow. Making it required is a maintainer decision. +Pushes to `master` and manual dispatches always run it. The job was warn-only +(`continue-on-error: true`) from #1717 until 2026-10-13 (plan item B3); since +then a regression fails the workflow. The switch followed a replay of the +week before: all 71 `master` runs from 2026-10-06 to 2026-10-12 passed the +counter check with the baselines of their own commit, and none of the +enforced counters differed between the measured iterations of a run. Against +the baselines of 2026-10-13 the older runs fail only where a later PR lowered +a value on purpose (J2 `renderer.layoutShiftScore` 0.233 → 0 in #1895) or +where J4 was not measured yet. `master` has no branch protection, so the job +is not a required status check: like the `Initial bytes ratchet` job, it +blocks by failing the `CI` workflow, and a maintainer decides whether to +merge past it. After the `Run the performance journeys` step, the job runs `check-journey-ratchet.mjs --only …` on the summary that step wrote, for the @@ -1310,8 +1319,7 @@ journey entries of `journey-baselines.json` (every entry except `performance-tools` test keeps that `--only` list equal to those entries, so a baseline cannot be added without being enforced. The step is in the job, not in the composite action, so the weekly tightening still measures a run -that would fail it. While the job is warn-only, a regression fails the job -and not the workflow. +that would fail it. A regression fails the job and the workflow. #### Enforced journey counters diff --git a/docs/architecture/validation-map.md b/docs/architecture/validation-map.md index a4c4de08a..f638b78cc 100644 --- a/docs/architecture/validation-map.md +++ b/docs/architecture/validation-map.md @@ -331,7 +331,7 @@ target `master` and for `master` pushes (dispatch it with `gh workflow run ci.yml --ref ` for a stacked branch). The `Performance journeys` job checks the journey counters listed with `--only` in its `Check the journey counters against the baselines` step against the -same file (warn-only); a new journey baseline +same file and fails the workflow on a regression; a new journey baseline must be added to that list too, which `pnpm nx test performance-tools` checks. The weekly `performance-ratchet.yml` workflow lowers baselines through a bot PR; validate