mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-08 17:06:15 -08:00
ci(performance): give the initial-bytes ratchet slack and a labelled override (#1744)
* ci(performance): give the initial-bytes ratchet slack and a labelled override The exact renderer.initialBytes counter failed PRs for reasons outside their diff: two concurrent merges left master 108 bytes over the baseline for hours, and bundler identifier renaming moves the counter by hundreds of bytes. PRs growing it by 243 and 302 bytes had no way to pass at all, because the direction check refuses any raised baseline. - Counter entries accept `slack` (integer, entry unit): the ratchet enforces `value + slack`, reports how much slack a measurement uses, and still prints the tighten hint below `value`. renderer.initialBytes gets 4096 bytes, so growth can accumulate at most 4 KiB past the last lowered baseline while regressions such as +35 KB still fail. - check-baseline-direction.mjs compares `value + slack`, treats widened slack like a widened tolerance, and takes `--allow-increase`, which reports weakened entries as ALLOWED instead of failing. - CI passes `--allow-increase` only when the pull request (or, for a master push, the pull request merged as the pushed commit) carries the perf-baseline-increase label, read from the API so a job re-run picks up a label added later. This change widens the slack itself, so its own PR needs the label. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0166PWobUjpoiWt8E9bBdsCs * fix(performance): report slack usage only for entries that have slack A wall-clock measurement above `value` but within `value × toleranceRatio` fell into the slack branch and was reported as using "slack", conflating timing tolerance with counter slack. Only entries with `slack` report it now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0166PWobUjpoiWt8E9bBdsCs * fix(performance): scope the push-time label and refuse raised counter values - A master push compares the whole push, but the label was read from the head commit's PR only, so one labelled PR could cover another commit's increase in the same push. The label now counts only when the push added exactly one first-parent commit (a squash or merge of one PR); any other push that weakens a baseline fails. - Raising a counter's `value` while narrowing its `slack` lowered the enforced limit and was reported as "lowered". A counter's value is the measured evidence and only moves down, so that raise is now a weakening that needs the label. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0166PWobUjpoiWt8E9bBdsCs * fix(performance): treat a counter/wall-clock type switch as a weakening Turning `{ value: 100, slack: 10 }` into `{ value: 105, toleranceRatio: 1 }` skipped the raised-counter-value rule and was reported as a lowered limit. Switching an entry between counter and wall-clock now needs the perf-baseline-increase label. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0166PWobUjpoiWt8E9bBdsCs * ci(performance): paginate the label lookups of the direction check The labels endpoint returns 30 entries per page by default, so a PR with more labels could miss perf-baseline-increase. Both lookups now request 100 per page and paginate, like the release-note gate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0166PWobUjpoiWt8E9bBdsCs --------- Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
4grayandClaude Opus 5.5 authored and GitHub committed
1 parent
f0765012fa
commit
af44e2368b
7 files changed
+436
-52
No files matched your search
@@ -130,6 +130,10 @@ jobs:
|
||||
name: Initial bytes ratchet
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
# pull-requests: read lets the direction check read the PR's labels.
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
@@ -153,12 +157,24 @@ jobs:
|
||||
# the revision this one is measured against instead: the target
|
||||
# branch for a pull request, the previous head for a master push,
|
||||
# master for a manual dispatch. Any raised limit, widened tolerance
|
||||
# or removed entry fails.
|
||||
# or slack, or removed entry fails, unless a maintainer put the
|
||||
# perf-baseline-increase label on the pull request. For a master
|
||||
# push the label counts only when the push added exactly one
|
||||
# first-parent commit (a squash or merge of one PR) and that
|
||||
# commit's PR carries it: the check compares the whole push, so a
|
||||
# multi-commit push cannot borrow one PR's label for another's
|
||||
# increase. Labels are read from the API, not the event payload,
|
||||
# so re-running this job after adding the label picks it up.
|
||||
- name: Refuse baseline increases against the previous revision
|
||||
env:
|
||||
EVENT_NAME: ${{ github.event_name }}
|
||||
BASE_REF: ${{ github.base_ref }}
|
||||
BEFORE_SHA: ${{ github.event.before }}
|
||||
HEAD_SHA: ${{ github.sha }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
REPOSITORY: ${{ github.repository }}
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
INCREASE_LABEL: perf-baseline-increase
|
||||
run: |
|
||||
set -euo pipefail
|
||||
case "$EVENT_NAME" in
|
||||
@@ -178,9 +194,32 @@ jobs:
|
||||
esac
|
||||
git show "FETCH_HEAD:tools/performance/journey-baselines.json" > /tmp/base-journey-baselines.json 2>/dev/null ||
|
||||
rm -f /tmp/base-journey-baselines.json
|
||||
case "$EVENT_NAME" in
|
||||
pull_request)
|
||||
labels="$(gh api "repos/$REPOSITORY/issues/$PR_NUMBER/labels?per_page=100" --paginate --jq '.[].name')"
|
||||
;;
|
||||
push)
|
||||
parent="$(gh api "repos/$REPOSITORY/commits/$HEAD_SHA" --jq '.parents[0].sha')"
|
||||
if [ "$parent" = "$BEFORE_SHA" ]; then
|
||||
labels="$(gh api "repos/$REPOSITORY/commits/$HEAD_SHA/pulls?per_page=100" --paginate --jq '.[].labels[].name')"
|
||||
else
|
||||
echo "This push added more than one commit; the $INCREASE_LABEL label is not consulted."
|
||||
labels=""
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
labels=""
|
||||
;;
|
||||
esac
|
||||
allow=()
|
||||
if grep -qxF "$INCREASE_LABEL" <<< "$labels"; then
|
||||
echo "The $INCREASE_LABEL label is set; weakened baselines are reported, not failed."
|
||||
allow=(--allow-increase)
|
||||
fi
|
||||
node tools/performance/check-baseline-direction.mjs \
|
||||
--base /tmp/base-journey-baselines.json \
|
||||
--head tools/performance/journey-baselines.json
|
||||
--head tools/performance/journey-baselines.json \
|
||||
"${allow[@]}"
|
||||
|
||||
# The production configuration is what users download; measuring
|
||||
# any other build would ratchet a number nobody ships.
|
||||
@@ -190,8 +229,8 @@ jobs:
|
||||
CI: true
|
||||
NX_TASKS_RUNNER_DYNAMIC_OUTPUT: false
|
||||
|
||||
# Fails when renderer.initialBytes exceeds the value committed in
|
||||
# tools/performance/journey-baselines.json. Baselines only move
|
||||
# Fails when renderer.initialBytes exceeds value + slack committed
|
||||
# in tools/performance/journey-baselines.json. Baselines only move
|
||||
# down, with the printed measurement as evidence; the contract is
|
||||
# docs/architecture/performance-journeys.md.
|
||||
- name: Check renderer.initialBytes against the baseline
|
||||
|
||||
Reference in new issue
Block a user