Files
iptvnator/.github/workflows/ci.yml
6e229f85b6 test(perf): add the J3 playback journey (#1774)
* test(perf): add the J3 playback journey

J3 clicks a live channel of an Xtream portal and ends at the built-in
HTML5 player's first `playing` event, with `loadedmetadata` as a
secondary phase. It follows J2: every iteration is a fresh J1 launch on
a copy of a profile seeded through the app's dialogs, and the click
happens after the app has settled in the portal's first live category.

The portal is the mock's `live-fallback` account, whose `.ts` live URLs
serve the local H.264/AAC MPEG-TS fixture that mpegts.js plays through
MSE on every platform. The marketing accounts' local live bytes are
zero-filled and never reach `playing`. Seeding selects the HTML5 player
and the `ts` stream format; the catalog's picsum.photos logos are
cancelled from the test side so no request leaves the machine.

Counters: renderer.ipcCallsToPlaying, renderer.httpRequestsToPlaying,
renderer.domMutationsToPlaying, renderer.layoutShiftScore and
renderer.longTasks; wall-clock click->loadedmetadata and click->playing.
renderer.ipcSerialDepthToPlaying is listed as unavailable until the
serial-depth helper lands. No baseline yet.

The probe gains a media-event terminal; J2's pre-click settle moves to
journey-click-settle.ts so both journeys share it unchanged, and the
probe spec's jsdom fixtures move to a shared test helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): show J3's HTTP boundary margins and watch 1 s after playing

Review follow-up. renderer.httpRequestsToPlaying compares the ledger's
arrival stamps with the renderer's click and playing stamps, which come
from different processes on the same host clock. Each iteration now
records the distance of the nearest request on either side of both
boundaries, so a count a clock difference could flip is visible.

Requests after playing were a single snapshot taken right after the
probe; the test now watches the ledger for a fixed 1 s after playing.
A live stream never leaves the mock quiet, so J2's quiet wait does not
apply.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: 4gray <fourgray@proton.me>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 08:37:02 +02:00

506 lines
22 KiB
YAML

name: CI
on:
push:
branches:
- master
pull_request:
branches:
- master
workflow_dispatch:
# Superseded PR pushes cancel their still-running checks. Non-PR runs get a
# unique group (run_id) because GitHub keeps at most one pending run per
# group even with cancel-in-progress: false — a shared ref group would let a
# rapid master push silently replace a queued sibling and leave a merged
# commit without a lint/test record.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
permissions:
contents: read
jobs:
actionlint:
name: Workflow lint
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout code
uses: actions/checkout@v7
# Image pinned by digest (tag 1.7.12). False positives are
# suppressed in .github/actionlint.yaml; shellcheck runs at
# warning+ severity so style/info notes in long release scripts
# don't fail CI while real quoting/logic bugs still do.
- name: Run actionlint
uses: docker://rhysd/actionlint:1.7.12@sha256:b1934ee5f1c509618f2508e6eb47ee0d3520686341fec936f3b79331f9315667
with:
args: -color
env:
SHELLCHECK_OPTS: --severity=warning
release-note-gate:
name: Release note gate
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
pull-requests: read
steps:
- name: Checkout code
uses: actions/checkout@v7
# The gate scripts are dependency-free Node, so this job skips
# pnpm install entirely and stays cheap.
- name: Validate release note format
run: node tools/release/build-release-notes.mjs --validate
# Labels are fetched live rather than read from the (stale) event
# payload, so applying `no-release-note` and re-running the check
# works without a new push. Policy lives in a unit-tested script,
# not in workflow bash.
- name: Require a release note for user-visible changes
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
set -euo pipefail
gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?per_page=100" \
--paginate --jq '[.[] | {filename, status}]' |
jq -s 'add // []' > /tmp/pr-files.json
gh api "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/labels?per_page=100" \
--paginate --jq '[.[].name]' |
jq -s 'add // []' > /tmp/pr-labels.json
jq -n \
--slurpfile files /tmp/pr-files.json \
--slurpfile labels /tmp/pr-labels.json \
'{files: $files[0], labels: $labels[0]}' |
node tools/release/check-release-note-gate.mjs
lint:
name: Lint
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout code
uses: actions/checkout@v7
with:
# nx affected needs the merge-base with the PR target branch.
fetch-depth: 0
- name: Install pnpm
uses: pnpm/action-setup@v6.0.10
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version-file: '.nvmrc'
cache: 'pnpm'
- name: Install dependencies
run: pnpm install --frozen-lockfile
# PRs lint only affected projects for faster feedback; root config
# or lockfile changes make every project affected, so the
# module-boundary and max-lines rules cannot be dodged this way.
- name: Lint affected projects (PR)
if: github.event_name == 'pull_request'
run: pnpm nx affected --target=lint --base=origin/${{ github.base_ref }} --head=HEAD --parallel=3 --output-style=static
env:
CI: true
NX_TASKS_RUNNER_DYNAMIC_OUTPUT: false
- name: Lint all projects (master)
if: github.event_name != 'pull_request'
run: pnpm nx run-many --target=lint --all --parallel=3 --output-style=static
env:
CI: true
NX_TASKS_RUNNER_DYNAMIC_OUTPUT: false
initial-bytes-ratchet:
name: Initial bytes ratchet
runs-on: ubuntu-latest
timeout-minutes: 30
# pull-requests: read lets the direction check read the PR's labels.
permissions:
contents: read
pull-requests: read
steps:
- name: Checkout code
uses: actions/checkout@v7
- name: Install pnpm
uses: pnpm/action-setup@v6.0.10
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version-file: '.nvmrc'
cache: 'pnpm'
- name: Install dependencies
run: pnpm install --frozen-lockfile
# The ratchet below compares a measurement with the baselines file
# of the same commit, so it cannot see a change that grows the
# payload and raises the baseline to match. Compare the file with
# the revision this one is measured against instead: the target
# branch for a pull request, the previous head for a master push,
# master for a manual dispatch. Any raised limit, widened tolerance
# or slack, or removed entry fails, unless a maintainer put the
# perf-baseline-increase label on the pull request. For a master
# push the label counts only when the push added exactly one
# first-parent commit (a squash or merge of one PR) and that
# commit's PR carries it: the check compares the whole push, so a
# multi-commit push cannot borrow one PR's label for another's
# increase. Labels are read from the API, not the event payload,
# so re-running this job after adding the label picks it up.
- name: Refuse baseline increases against the previous revision
env:
EVENT_NAME: ${{ github.event_name }}
BASE_REF: ${{ github.base_ref }}
BEFORE_SHA: ${{ github.event.before }}
HEAD_SHA: ${{ github.sha }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPOSITORY: ${{ github.repository }}
GH_TOKEN: ${{ github.token }}
INCREASE_LABEL: perf-baseline-increase
run: |
set -euo pipefail
case "$EVENT_NAME" in
pull_request)
git fetch --no-tags --depth=1 origin "$BASE_REF"
;;
push)
if [ -z "$BEFORE_SHA" ] || [ "$BEFORE_SHA" = "0000000000000000000000000000000000000000" ]; then
echo "No previous revision for this push; nothing to compare against."
exit 0
fi
git fetch --no-tags --depth=1 origin "$BEFORE_SHA"
;;
*)
git fetch --no-tags --depth=1 origin master
;;
esac
git show "FETCH_HEAD:tools/performance/journey-baselines.json" > /tmp/base-journey-baselines.json 2>/dev/null ||
rm -f /tmp/base-journey-baselines.json
case "$EVENT_NAME" in
pull_request)
labels="$(gh api "repos/$REPOSITORY/issues/$PR_NUMBER/labels?per_page=100" --paginate --jq '.[].name')"
;;
push)
parent="$(gh api "repos/$REPOSITORY/commits/$HEAD_SHA" --jq '.parents[0].sha')"
if [ "$parent" = "$BEFORE_SHA" ]; then
labels="$(gh api "repos/$REPOSITORY/commits/$HEAD_SHA/pulls?per_page=100" --paginate --jq '.[].labels[].name')"
else
echo "This push added more than one commit; the $INCREASE_LABEL label is not consulted."
labels=""
fi
;;
*)
labels=""
;;
esac
allow=()
if grep -qxF "$INCREASE_LABEL" <<< "$labels"; then
echo "The $INCREASE_LABEL label is set; weakened baselines are reported, not failed."
allow=(--allow-increase)
fi
node tools/performance/check-baseline-direction.mjs \
--base /tmp/base-journey-baselines.json \
--head tools/performance/journey-baselines.json \
"${allow[@]}"
# The production configuration is what users download; measuring
# any other build would ratchet a number nobody ships.
- name: Build web app (production)
run: pnpm nx build web --skip-nx-cache
env:
CI: true
NX_TASKS_RUNNER_DYNAMIC_OUTPUT: false
# Fails when renderer.initialBytes exceeds value + slack committed
# in tools/performance/journey-baselines.json. Baselines only move
# down, with the printed measurement as evidence; the contract is
# docs/architecture/performance-journeys.md.
- name: Check renderer.initialBytes against the baseline
run: pnpm run perf:initial-bytes:check
- name: Upload journey summary
if: always()
uses: actions/upload-artifact@v7
with:
name: performance-journey-summary
path: dist/performance/
retention-days: 14
# Decides whether a pull request can move the performance journeys, so
# the journeys job below does not spend a runner on docs-only changes.
# Pushes to master and manual dispatches always run them.
performance-journeys-scope:
name: Performance journeys scope
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
run: ${{ steps.scope.outputs.run }}
steps:
# The PR checkout is the merge commit: its first parent is the
# target branch, so two commits are enough to diff the PR.
- name: Checkout code
if: github.event_name == 'pull_request'
uses: actions/checkout@v7
with:
fetch-depth: 2
# Anything can move a journey (application code, the harness, root
# build inputs such as .nvmrc, nx.json or tsconfig.base.json), so
# the filter lists what cannot: the same paths the E2E workflow
# ignores, plus release notes.
- name: Detect journey-relevant changes
id: scope
env:
EVENT_NAME: ${{ github.event_name }}
run: |
set -euo pipefail
if [ "$EVENT_NAME" != "pull_request" ]; then
echo "run=true" >> "$GITHUB_OUTPUT"
exit 0
fi
relevant="$(git diff --name-only HEAD^1 HEAD |
grep -vE '\.md$|^docs/|^\.plans/|^\.codex/|^\.claude/|^\.changes/|^apps/website/' || true)"
if [ -n "$relevant" ]; then
echo "Journey-relevant changes:"
echo "$relevant"
echo "run=true" >> "$GITHUB_OUTPUT"
else
echo "No journey-relevant changes; skipping the performance journeys."
echo "run=false" >> "$GITHUB_OUTPUT"
fi
# Runs the journey benchmarks (docs/architecture/performance-journeys.md)
# on the canonical Linux runner and uploads the summary as evidence.
performance-journeys:
name: Performance journeys
needs: performance-journeys-scope
if: needs.performance-journeys-scope.outputs.run == 'true'
runs-on: ubuntu-latest
# The electron-performance build is the bulk of the time; each journey
# (launch, open-source, playback) is six fresh Electron processes plus
# one seeding run.
timeout-minutes: 30
# Warn-only for the first two weeks of plan item B3: a failure is
# visible on the run but does not fail the workflow.
continue-on-error: true
env:
NX_SKIP_NX_CACHE: true
steps:
- name: Checkout code
uses: actions/checkout@v7
- name: Install pnpm
uses: pnpm/action-setup@v6.0.10
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version-file: '.nvmrc'
cache: 'pnpm'
- name: Install dependencies
run: pnpm install --frozen-lockfile
# Electron dependency check, the xvfb run, the summary lookup and
# the job-summary report; shared with performance-ratchet.yml.
- name: Run the performance journeys
uses: ./.github/actions/performance-journeys
- name: Upload journey summaries
if: always()
uses: actions/upload-artifact@v7
with:
name: performance-journeys
path: dist/performance/journeys/
if-no-files-found: warn
retention-days: 14
unit-and-typecheck:
name: Unit Tests and Typechecks
runs-on: ubuntu-latest
timeout-minutes: 45
permissions:
contents: read
# The scope step lists the PR's changed files through the API.
pull-requests: read
steps:
- name: Checkout code
uses: actions/checkout@v7
- name: Install pnpm
uses: pnpm/action-setup@v6.0.10
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version-file: '.nvmrc'
cache: 'pnpm'
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Validate agent guidance
run: pnpm run agents:validate
- name: Validate Nx dependency version policy
run: pnpm run deps:nx:validate
- name: Validate Vite dev-server transform filter patch
run: pnpm run deps:vite:test
- name: Validate electron-builder keychain password patch
run: pnpm run deps:electron-builder:test
- name: Validate stylesheet Nx inputs
run: pnpm run styles:inputs:validate
- name: Typecheck web and Electron entry points
run: pnpm run typecheck:ci
- name: Typecheck Jest spec programs
run: pnpm run typecheck:spec:test && pnpm run typecheck:spec
- name: Check i18n drift
run: pnpm run i18n:check
# Node specs for the benchmark and journey harness (about ten
# seconds). They live in the E2E app, which the unit coverage scope
# below treats as out of scope, so this step is not gated by it.
- name: Test the performance harness
run: pnpm nx run electron-backend-e2e:test-performance-harness --skip-nx-cache
env:
CI: true
NX_TASKS_RUNNER_DYNAMIC_OUTPUT: false
# A pull request whose changes cannot reach any Tier A test (docs,
# notes, other workflows, website, E2E and mock-server apps, release
# and packaging tooling, a scripts-only package.json edit) skips the
# suite. The allowlist lives in a unit-tested script; anything it
# does not know runs everything, and master always runs everything.
- name: Decide unit coverage scope
id: scope
env:
EVENT_NAME: ${{ github.event_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
CHANGED_FILES: ${{ github.event.pull_request.changed_files }}
run: |
set -euo pipefail
if [ "$EVENT_NAME" != "pull_request" ]; then
echo "Not a pull request; running the full suite."
echo "run=true" >> "$GITHUB_OUTPUT"
exit 0
fi
# The list-files endpoint stops at 3,000 files; a truncated
# list could hide a file that needs the suite.
if [ "${CHANGED_FILES:-0}" -ge 3000 ]; then
echo "PR changes ${CHANGED_FILES} files, beyond the API listing limit; running the full suite."
echo "run=true" >> "$GITHUB_OUTPUT"
exit 0
fi
git fetch --no-tags --depth=1 origin "$BASE_SHA"
gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?per_page=100" \
--paginate --jq '.[] | .filename, (.previous_filename // empty)' |
node tools/coverage/unit-coverage-scope.mjs --base FETCH_HEAD --github-output
# pnpm only runs build scripts for allow-listed packages
# (pnpm-workspace.yaml), so electron's postinstall never fetches
# its binary. `require('electron')` then downloads it lazily, and
# parallel Jest workers and Tier A projects that spawn Electron
# race on the same download: one executes the half-written binary
# (ETXTBSY). Fetch it once before the Tier A suite, the only step
# here whose specs spawn Electron. Unlike a bare install.js, the
# helper also runs the binary, so a partial extraction fails here.
# The Tier A runner calls it too; this step only separates a
# download failure from test failures.
- name: Install the Electron binary
if: steps.scope.outputs.run == 'true'
run: node tools/testing/ensure-electron-binary.mjs
# Jest's transform cache (TypeScript/Angular transpilation plus
# coverage instrumentation, keyed by file content) is persisted
# between runs. Only master pushes and maintainer dispatches save
# it; pull requests restore it and never write, so a PR cannot plant
# an entry that a later master run would read.
- name: Restore Jest transform cache
if: steps.scope.outputs.run == 'true' && github.event_name != 'push'
uses: actions/cache/restore@v6
with:
path: ${{ runner.temp }}/jest-cache
key: jest-transform-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml', 'tsconfig.base.json', 'jest.preset.js', 'jest.web-esm.workspace.ts', 'apps/web/jest.config.ts') }}-${{ github.run_id }}
restore-keys: |
jest-transform-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml', 'tsconfig.base.json', 'jest.preset.js', 'jest.web-esm.workspace.ts', 'apps/web/jest.config.ts') }}-
- name: Run Tier A unit coverage suite
if: steps.scope.outputs.run == 'true'
run: pnpm run coverage:ci
env:
CI: true
NX_TASKS_RUNNER_DYNAMIC_OUTPUT: false
JEST_CACHE_DIRECTORY: ${{ runner.temp }}/jest-cache
- name: Validate coverage tooling (suite skipped)
if: steps.scope.outputs.run != 'true'
run: pnpm run coverage:tools:test && pnpm run coverage:policy:check
# Master pushes start from an empty cache, so the saved cache holds
# exactly the current tree and does not grow run over run.
- name: Save Jest transform cache
if: >-
steps.scope.outputs.run == 'true' &&
(github.event_name == 'workflow_dispatch' ||
(github.event_name == 'push' && github.ref == 'refs/heads/master'))
uses: actions/cache/save@v6
with:
path: ${{ runner.temp }}/jest-cache
key: jest-transform-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml', 'tsconfig.base.json', 'jest.preset.js', 'jest.web-esm.workspace.ts', 'apps/web/jest.config.ts') }}-${{ github.run_id }}
- name: Run Tier B/C validation commands
run: node tools/coverage/check-coverage-policy.mjs --run-non-tier-a
env:
CI: true
NX_TASKS_RUNNER_DYNAMIC_OUTPUT: false
- name: Upload unit coverage artifact
if: always() && steps.scope.outputs.run == 'true'
uses: actions/upload-artifact@v7
with:
name: unit-coverage
path: |
coverage/merged/
retention-days: 14
- name: Upload unit coverage to Codecov
if: always() && steps.scope.outputs.run == 'true'
uses: codecov/codecov-action@v7
with:
files: ./coverage/merged/lcov.info,./coverage/merged/cobertura-coverage.xml
flags: unit
name: iptvnator-unit
fail_ci_if_error: false
handle_no_reports_found: true
disable_search: true
token: ${{ secrets.CODECOV_TOKEN }}