Commit Graph
7 Commits
Author SHA1 Message Date
5f21e240e9 test(performance): add J4 search journey (#1816)
* test(performance): add J4 search journey

Measures typing a six-character query into the header search box on
/workspace/search until the global search results settle, on a profile
with the M3U fixture and the mock's existing 12,000-item `large` Xtream
catalog. Counters: bridge calls and SQL statements per search (with a
per-keystroke breakdown), serial IPC depth, DOM mutations, change-detection
ticks, layout shift and long tasks; wall-clock last keystroke to settled
and first keystroke to first result.

Runs in the existing journeys target and the warn-only CI job, whose
summary now prints the per-keystroke table. Moves J3's picsum artwork
blocker into a shared helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(performance): J4 settles only on the final term's query

The probe could settle on cards of an earlier term that stay visible while
the final term debounces. The journey now stamps every dbGlobalSearch trace
event in the main process, and the record requires the last query between
the sentinels to be for the final term and to have completed before the
end sentinel. Iterations with a keydown gap over 250 ms (below the 350 ms
debounce) are rejected; gaps and the final query are kept as evidence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(performance): anchor J4's SQL count to the journey sentinels

renderer.sqlStatementsPerSearch was the difference of test-side samples
taken before the first key and after the end sentinel had been read, so
database work in either gap could be counted. The main process now reads
main.sqlStatements when the start and end sentinels arrive, and the counter
is their difference; sqlStatementsAfterSettled starts at the end sentinel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: 4gray <fourgray@proton.me>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 18:46:57 +02:00
b315f8564d fix(electron): show the main window once its document has loaded; enforce J1 IPC and mutation counters (#1828)
* fix(electron): show the main window once its document has loaded; enforce J1 IPC and mutation counters

Re-lands #1788, which merged into #1782's branch after #1782 had already
reached master, so none of it is on master.

The hidden main window was shown on ready-to-show only. On Linux under
X11, when the startup scripts run before the window's first frame, the
next frame comes about a second later: nothing is on screen and the
splash's requestAnimationFrame waits, so J1's first card came ~940 ms
after load instead of ~480 ms in most runner launches (18 bridge calls /
1,031-1,033 DOM mutations instead of 15 / 558).

The window is now shown at ready-to-show or the main frame's
did-finish-load, whichever comes first, with the splash colour as its
background so showing before the first paint does not flash. The journey
gate keeps the app's did-finish-load listeners away from its about:blank
detour, as it already does for ready-to-show.

Three dispatched runs on this branch (37192092882, 37192097790,
37192103151) read 15 calls and 558 mutations in all 18 iterations,
stable: true. Both become baselines (slack 0), and the Performance
journeys job checks them with check-journey-ratchet.mjs --only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(perf): record the evidence PR of the J1 runtime baselines

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: 4gray <fourgray@proton.me>
2026-10-06 10:34:08 +02:00
e4cf48fdc2 test(perf): count change-detection ticks in the electron-performance build (#1776)
* test(perf): count change-detection ticks in the electron-performance build

J1 renderer.cdTicksToFirstCard, J2 renderer.cdTicksToFirstPage and the
J1 idle baseline renderer.cdTicksIdle30s. Angular's ɵsetProfiler is only
reachable through the dev-mode window.ng global, so the electron-performance
configuration alone swaps environment.ts for environment.performance.ts,
which re-exports the production AppConfig and wraps ApplicationRef._tick.
Production and PWA sources and output are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): refuse a J1 idle window that opened late after the settle point

Addresses review: the idle window opens in the settle timer's callback while
the settle point is that timer's deadline, so a late callback left ticks
uncounted between the two.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: 4gray <fourgray@proton.me>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 14:23:15 +02:00
6e229f85b6 test(perf): add the J3 playback journey (#1774)
* test(perf): add the J3 playback journey

J3 clicks a live channel of an Xtream portal and ends at the built-in
HTML5 player's first `playing` event, with `loadedmetadata` as a
secondary phase. It follows J2: every iteration is a fresh J1 launch on
a copy of a profile seeded through the app's dialogs, and the click
happens after the app has settled in the portal's first live category.

The portal is the mock's `live-fallback` account, whose `.ts` live URLs
serve the local H.264/AAC MPEG-TS fixture that mpegts.js plays through
MSE on every platform. The marketing accounts' local live bytes are
zero-filled and never reach `playing`. Seeding selects the HTML5 player
and the `ts` stream format; the catalog's picsum.photos logos are
cancelled from the test side so no request leaves the machine.

Counters: renderer.ipcCallsToPlaying, renderer.httpRequestsToPlaying,
renderer.domMutationsToPlaying, renderer.layoutShiftScore and
renderer.longTasks; wall-clock click->loadedmetadata and click->playing.
renderer.ipcSerialDepthToPlaying is listed as unavailable until the
serial-depth helper lands. No baseline yet.

The probe gains a media-event terminal; J2's pre-click settle moves to
journey-click-settle.ts so both journeys share it unchanged, and the
probe spec's jsdom fixtures move to a shared test helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): show J3's HTTP boundary margins and watch 1 s after playing

Review follow-up. renderer.httpRequestsToPlaying compares the ledger's
arrival stamps with the renderer's click and playing stamps, which come
from different processes on the same host clock. Each iteration now
records the distance of the nearest request on either side of both
boundaries, so a count a clock difference could flip is visible.

Requests after playing were a single snapshot taken right after the
probe; the test now watches the ledger for a fixed 1 s after playing.
A live stream never leaves the mock quiet, so J2's quiet wait does not
apply.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: 4gray <fourgray@proton.me>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 08:37:02 +02:00
28b022ff48 test(perf): add the J2 open-source journey (#1730)
* test(perf): add the J2 open-source journey

Measure the click on the Xtream portal card until the category list and
the first page of the opened section are painted, in the same fresh
process as J1 after its counters are final and the app has settled.

- journey-renderer-probe: optional click start (capture-phase listener on
  window, start sentinel before the app sees the click, entries before the
  click dropped), companion selectors, recent-input layout shifts tallied
- journey-main-ipc-capture: optional start sentinel; counts calls between
  the two sentinels
- journey-mock-request-ledger: loopback proxy that counts every request
  the app sends to the mock without storing credentials
- open-source-journey-record: J2 counters and evidence
- journey-run / journey-summary: every journey spec of one perf:journeys
  run adds its entry to the same summary.json
- docs: J2 contract in performance-journeys.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): stop echoing the request URL from the ledger spec's upstream

CodeQL flagged the fake upstream as reflected XSS. It now records what it
received server-side and answers with a fixed text/plain body.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): address J2 review findings

- Sentinels use cancelSourceProbe: the preload traces the call before
  forwarding it and SOURCE_HEALTH_CANCEL is an in-memory map lookup, so a
  marker no longer runs a SQLite query on the worker ahead of the measured
  work (Codex P1). A spec pins that handler contract.
- A run is started only in the Playwright runner, replacing inherited
  values, and carries a random harness.runId; summaries from another
  invocation are never merged (Greptile P1, Codex P2).
- The mock ledger tracks in-flight requests; settling and the HTTP window
  require none in flight (Codex P2).
- clickToFirstPagePaintMs reports click to the committed paint next to
  the terminal-batch clickToFirstPageMs (Codex P1).
- The Playwright attachment carries the whole summary (Greptile P2).
- jsdom probe specs wait for the post-paint cutoff instead of a fixed
  40 ms, which flaked when the harness runs all files in parallel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(validation): describe perf:journeys as running J1 and J2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): settle J2 on pending bridge calls and start HTTP at the click

- The journey IPC capture pairs every traced start with its success or
  error and exposes the calls still in flight. J2 settles only when J1's
  capture, installed before the document loaded, has none pending, so a
  slow startup call cannot resolve after the click and count as J2.
- The mock HTTP window starts at the renderer's click stamp instead of
  the test-side mark taken before Playwright's actionability checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): restart the J2 quiet period when pending work completes

Both waits in the open-source journey (settling before the click, closing
the mock window after the terminal) now use one waitForJourneyQuiet
helper that compares whole samples, in-flight counts included. The poll
that first sees a request or bridge call complete restarts the quiet
period, so the window is never measured from a poll at which work was
still pending. A fake-clock spec covers the in-flight to zero case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): align J2 with the main-process counters from #1715

After rebasing on #1715, J1 measures main.sqlStatementsBeforeReadyToShow,
so J2's reason for listing main.sqlStatementsToFirstPage as unavailable
(no countable channel) was stale. State the actual limit: the running
total is read from the test process and cannot be bounded at the click
or the first-page batch. The performance-journeys CI job comment now
names both journeys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): launch J2 without SQL counting and stamp mock requests in sub-ms

- runLaunchJourney takes the launch instrumentation; only J1 turns on the
  main-process counters and IPTVNATOR_PERF_COUNT_SQL, so J2's click is not
  measured under the hook that wraps every SQLite statement. The flags are
  built in journey-launch-environment.ts, which the SQL opt-in guard now
  expects, and a launch record without main counters is rejected.
- The mock ledger stamps arrivals with performance.timeOrigin +
  performance.now(), the same sub-millisecond epoch as the renderer's
  click, so a request later in the click's millisecond is not counted
  before it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): reject J2 iterations with activity after settling

- The open-source record compares the settle snapshot with what the probe
  and the IPC capture counted up to the click event, and with the mock
  requests between the snapshot and the click stamp. Any change means
  background work began during Playwright's actionability checks and
  could land in J2, so the iteration is rejected.
- The SQL opt-in guard also checks who passes mainCounters: true: only
  measureLaunchJourney may, and J2 must pass false.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): count the long task that dispatches the J2 click

A long task's startTime precedes the click event's timestamp when the
listener runs inside it, so the start-time filter dropped the task that
performs the interaction. Long tasks now count when their range overlaps
the window: on one main thread only the dispatching task can overlap the
click. Layout shifts keep the start-time filter. J1 is unchanged (its
window starts at -Infinity).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): bound J2's late-request check by the quiet sample's mark

The late-activity check compared requests against a fresh ledger mark
taken after waitForQuiet returned. A request that arrived while the final
quiet sample was still reading the IPC capture advanced that mark and
escaped the check. The boundary is now the ledger position read by the
accepted sample itself, like its DOM and IPC counts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): end J2's HTTP window at the accepted quiet sample

The post-terminal window read the ledger after waitForMockQuiet returned,
so a request arriving in between was counted although its completion was
never waited for. waitForMockQuiet now returns the ledger position its
accepted sample read; later requests are kept as evidence
(httpRequestsAfterSettledByRoute) instead of the counter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): observe late mock requests before reading J2's ledger

httpRequestsAfterSettledByRoute read the ledger right after the accepted
quiet sample, so late requests had no chance to appear in it. The ledger
is now read after another quiet interval; the counter stays bounded by
the quiet sample's mark and late traffic shows up in the evidence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): fail the J2 quiet wait when a sample stalls past its deadline

waitForJourneyQuiet accepted a sample that returned unchanged after a
stall longer than the timeout as the end of a quiet period, before the
deadline check ran. The deadline is now checked first, so a stalled
sample fails the wait instead of letting the click go ahead unobserved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): detach J1's IPC capture before the J2 click

J2 used J1's capture to see pending launch bridge calls while settling,
but its ipcMain listener stayed attached and ran for every bridge call of
the measured click. The capture can now be detached; J2 detaches J1's
right after settling, before the click.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): sample both J2 settle captures in one main-process snapshot

The settle sample read J1's capture (pending calls) and J2's capture
(call count) in two evaluate calls, so a call starting in between was
counted with a stale zero in flight and its completion went unseen. Both
states are now read in one synchronous pass, where no ipcMain event can
be handled in between.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: 4gray <fourgray@proton.me>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 20:37:45 +02:00
1d9a563d1a test(performance): count startup phases and SQL statements for the J1 launch journey (#1715)
* test(performance): count startup phases and SQL statements for the J1 launch journey

Implements plan item A2. With IPTVNATOR_PERF_CAPTURE=1 the main process
keeps named counters and registers a main-only performance:read-counters
IPC handler; without the flag nothing is counted and the handler does not
exist.

- debug-trace.ts owns the registry; traceStartupPhase replaces the
  trace('startup', ...) sites and counts main.startupPhases.
- The database worker counts executed statements through better-sqlite3's
  Statement prototype (the verbose callback expands every statement and
  made bulk inserts 2-4x slower) and posts the count over its message
  port, flushed before every other worker message. The main-thread shared
  connection is counted through a new connection observer in the shared
  database library.
- The first main window freezes main.modulesRegisteredBeforeWindow at
  creation and main.sqlStatementsBeforeReadyToShow at ready-to-show.
- The journey gate drops the ready-to-show that Electron emits for the
  about:blank detour, so the app sees the real document's first paint,
  and taps the counters handler; the J1 record reads both counters after
  the renderer probe completes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(database): require one SQL statement per exec during initialization

The performance capture counts one exec call as one statement, because
SQL cannot be split reliably in the counter (trigger bodies contain
semicolons). The historical-upgrade driver now wraps exec on every
connection initDatabase opens and fails on a batch, so that counting
assumption holds for the fresh profile and all historical schemas.
Documents the definition in the counter and the architecture docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(performance): count SQL statements only for the launch journey

Codex review: the M3U import, refresh-cancellation and Xtream benchmarks
also run with IPTVNATOR_PERF_CAPTURE=1, so the statement hook wrapped
every row of their bulk inserts and changed what they measure.

SQL counting now also needs IPTVNATOR_PERF_COUNT_SQL=1, which only the
launch journey sets; a harness test fails if another source sets it.
Startup phases, the window snapshot and the read handler stay on the
capture flag. Without SQL counting no ready-to-show listener is attached,
so a zero is never reported for statements nobody counted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: 4gray <fourgray@proton.me>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 21:01:26 +02:00
0f1cad4b16 test(performance): add the J1 launch-to-usable journey benchmark (#1698)
* test(performance): add the J1 launch-to-usable journey benchmark

Implements plan items A1, A3 (J1 only) and the minimal A4 from
.plans/2026-09-25-performance-journeys-ratchet.md.

- journey-renderer-probe.ts: init-script probe counting DOM mutations,
  layout shifts and long tasks until the first source card is visible on
  /workspace with the splash removed; unit-tested with jsdom fixtures.
- journey-main-ipc-capture.ts: counts bridge invocations from the preload's
  renderer-API trace channel up to a sentinel call the probe fires, so the
  IPC counter is exact without touching production code.
- launch.journey.ts + playwright.journeys.config.ts: seeded profile (one
  M3U source, one Xtream portal on the loopback mock), one warm-up and five
  measured iterations, fresh process and data directory each, writing
  dist/performance/journeys/<timestamp>/summary.json with exact counters
  and P50/P90 wall-clock.
- Nx target electron-backend-e2e:journeys and root script perf:journeys.
- docs/architecture/performance-journeys.md, README, context and
  validation map entries.

renderer.cdTicksToFirstCard and main.sqlStatementsBeforeReadyToShow are
reported as unavailable with the reason instead of being faked.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* test(performance): gate the renderer load so the probes never race startup

Review follow-up for #1698.

- journey-renderer-gate.cjs: a main-process hook loaded with `-r` (the
  mechanism Playwright uses for its own loader) makes the first
  loadFile/loadURL navigate to about:blank and holds the real load until
  the test releases it. Playwright reports no page before a navigation
  commits, so this is what lets the renderer probe be registered on the
  page before the real document exists; the IPC capture is installed
  before the release too. A safety timeout releases the gate on its own
  and marks the iteration invalid. Unit-tested with a fake BrowserWindow.
- launch-journey-app.ts: registers the probe on the parked page, releases
  the gate, waits for the real document to commit, fails fast when the
  probe is missing, and refuses an iteration whose gate timed out, saw a
  second load, or released before the probe was in place.
- journey-renderer-probe.ts: entries delivered live after the terminal
  batch are buffered and filtered by the same cutoff as queued ones, and
  the cutoff is sampled in a timer queued from the first rAF, i.e. after
  the card's frame is painted, so the render task's long task and layout
  shift are consistently included.
- launch-journey-record.ts: layoutShiftScore rounded to three decimals; a
  0.0001 shift flipped in and out of the cutoff between iterations.
- playwright.journeys.config.ts: reuse a mock server left on the journeys
  port locally (its fixtures are deterministic); CI still starts its own.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* test(performance): validate the journey gate after the probe completes

Codex follow-up on #1698: the gate state returned by release() cannot see a
reload or recovery navigation that happens before the first card. Re-read the
live state once the renderer probe has finished and validate that instead,
so an iteration spanning an extra navigation is rejected.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* test(performance): fail an iteration whose performance observers were unavailable

Codex follow-up on #1698: a renderer that cannot observe layout-shift or
longtask entries used to pass the probe with zero counters, which a ratchet
could not tell apart from a genuine zero. The probe assertion now rejects
such an iteration and names the missing observer.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: 4gray <fourgray@proton.me>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-26 19:18:19 +02:00