Files
iptvnator/apps/electron-backend-e2e/src/performance/journey-main-ipc-capture.ts
T
28b022ff48 test(perf): add the J2 open-source journey (#1730)
* test(perf): add the J2 open-source journey

Measure the click on the Xtream portal card until the category list and
the first page of the opened section are painted, in the same fresh
process as J1 after its counters are final and the app has settled.

- journey-renderer-probe: optional click start (capture-phase listener on
  window, start sentinel before the app sees the click, entries before the
  click dropped), companion selectors, recent-input layout shifts tallied
- journey-main-ipc-capture: optional start sentinel; counts calls between
  the two sentinels
- journey-mock-request-ledger: loopback proxy that counts every request
  the app sends to the mock without storing credentials
- open-source-journey-record: J2 counters and evidence
- journey-run / journey-summary: every journey spec of one perf:journeys
  run adds its entry to the same summary.json
- docs: J2 contract in performance-journeys.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): stop echoing the request URL from the ledger spec's upstream

CodeQL flagged the fake upstream as reflected XSS. It now records what it
received server-side and answers with a fixed text/plain body.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): address J2 review findings

- Sentinels use cancelSourceProbe: the preload traces the call before
  forwarding it and SOURCE_HEALTH_CANCEL is an in-memory map lookup, so a
  marker no longer runs a SQLite query on the worker ahead of the measured
  work (Codex P1). A spec pins that handler contract.
- A run is started only in the Playwright runner, replacing inherited
  values, and carries a random harness.runId; summaries from another
  invocation are never merged (Greptile P1, Codex P2).
- The mock ledger tracks in-flight requests; settling and the HTTP window
  require none in flight (Codex P2).
- clickToFirstPagePaintMs reports click to the committed paint next to
  the terminal-batch clickToFirstPageMs (Codex P1).
- The Playwright attachment carries the whole summary (Greptile P2).
- jsdom probe specs wait for the post-paint cutoff instead of a fixed
  40 ms, which flaked when the harness runs all files in parallel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(validation): describe perf:journeys as running J1 and J2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): settle J2 on pending bridge calls and start HTTP at the click

- The journey IPC capture pairs every traced start with its success or
  error and exposes the calls still in flight. J2 settles only when J1's
  capture, installed before the document loaded, has none pending, so a
  slow startup call cannot resolve after the click and count as J2.
- The mock HTTP window starts at the renderer's click stamp instead of
  the test-side mark taken before Playwright's actionability checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): restart the J2 quiet period when pending work completes

Both waits in the open-source journey (settling before the click, closing
the mock window after the terminal) now use one waitForJourneyQuiet
helper that compares whole samples, in-flight counts included. The poll
that first sees a request or bridge call complete restarts the quiet
period, so the window is never measured from a poll at which work was
still pending. A fake-clock spec covers the in-flight to zero case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): align J2 with the main-process counters from #1715

After rebasing on #1715, J1 measures main.sqlStatementsBeforeReadyToShow,
so J2's reason for listing main.sqlStatementsToFirstPage as unavailable
(no countable channel) was stale. State the actual limit: the running
total is read from the test process and cannot be bounded at the click
or the first-page batch. The performance-journeys CI job comment now
names both journeys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): launch J2 without SQL counting and stamp mock requests in sub-ms

- runLaunchJourney takes the launch instrumentation; only J1 turns on the
  main-process counters and IPTVNATOR_PERF_COUNT_SQL, so J2's click is not
  measured under the hook that wraps every SQLite statement. The flags are
  built in journey-launch-environment.ts, which the SQL opt-in guard now
  expects, and a launch record without main counters is rejected.
- The mock ledger stamps arrivals with performance.timeOrigin +
  performance.now(), the same sub-millisecond epoch as the renderer's
  click, so a request later in the click's millisecond is not counted
  before it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): reject J2 iterations with activity after settling

- The open-source record compares the settle snapshot with what the probe
  and the IPC capture counted up to the click event, and with the mock
  requests between the snapshot and the click stamp. Any change means
  background work began during Playwright's actionability checks and
  could land in J2, so the iteration is rejected.
- The SQL opt-in guard also checks who passes mainCounters: true: only
  measureLaunchJourney may, and J2 must pass false.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): count the long task that dispatches the J2 click

A long task's startTime precedes the click event's timestamp when the
listener runs inside it, so the start-time filter dropped the task that
performs the interaction. Long tasks now count when their range overlaps
the window: on one main thread only the dispatching task can overlap the
click. Layout shifts keep the start-time filter. J1 is unchanged (its
window starts at -Infinity).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): bound J2's late-request check by the quiet sample's mark

The late-activity check compared requests against a fresh ledger mark
taken after waitForQuiet returned. A request that arrived while the final
quiet sample was still reading the IPC capture advanced that mark and
escaped the check. The boundary is now the ledger position read by the
accepted sample itself, like its DOM and IPC counts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): end J2's HTTP window at the accepted quiet sample

The post-terminal window read the ledger after waitForMockQuiet returned,
so a request arriving in between was counted although its completion was
never waited for. waitForMockQuiet now returns the ledger position its
accepted sample read; later requests are kept as evidence
(httpRequestsAfterSettledByRoute) instead of the counter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): observe late mock requests before reading J2's ledger

httpRequestsAfterSettledByRoute read the ledger right after the accepted
quiet sample, so late requests had no chance to appear in it. The ledger
is now read after another quiet interval; the counter stays bounded by
the quiet sample's mark and late traffic shows up in the evidence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): fail the J2 quiet wait when a sample stalls past its deadline

waitForJourneyQuiet accepted a sample that returned unchanged after a
stall longer than the timeout as the end of a quiet period, before the
deadline check ran. The deadline is now checked first, so a stalled
sample fails the wait instead of letting the click go ahead unobserved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): detach J1's IPC capture before the J2 click

J2 used J1's capture to see pending launch bridge calls while settling,
but its ipcMain listener stayed attached and ran for every bridge call of
the measured click. The capture can now be detached; J2 detaches J1's
right after settling, before the click.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(perf): sample both J2 settle captures in one main-process snapshot

The settle sample read J1's capture (pending calls) and J2's capture
(call count) in two evaluate calls, so a call starting in between was
counted with a stale zero in flight and its completion went unseen. Both
states are now read in one synchronous pass, where no ipcMain event can
be handled in between.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: 4gray <fourgray@proton.me>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 20:37:45 +02:00

312 lines
12 KiB
TypeScript

import type { ElectronApplication } from '@playwright/test';
/**
* Main-process side of the journey IPC counter.
*
* With `IPTVNATOR_TRACE_IPC=1` the preload wraps every bridge method (except
* `on*` / `remove*` listener registrations) and sends one trace event per
* invocation on the renderer-API trace channel before forwarding the call
* (see `apps/electron-backend/src/app/api/main.preload.ts`). This capture
* subscribes to that channel from the test side and counts `start` events
* until the renderer probe's sentinel call arrives. Renderer-to-main IPC is
* delivered in order, so every call started before the sentinel is counted
* and nothing after it is.
*
* A journey that starts inside a running app (J2 "Open a source") passes
* `startSentinelId`: the probe sends that id before the app handles the
* start click, calls before it are only tallied in `callsBeforeStart`, and
* `callsBeforeSentinel` then counts the calls between the two sentinels.
*/
/** Literal of `DEBUG_TRACE_EVENT_CHANNEL` in `services/debug-trace.ts`. */
export const JOURNEY_RENDERER_API_TRACE_CHANNEL = 'IPTVNATOR_DEBUG_TRACE_EVENT';
export const JOURNEY_MAIN_IPC_STATE_KEY = '__iptvnatorJourneyMainIpcCapture';
export interface JourneyMainIpcCaptureOptions {
readonly channel: string;
readonly sentinelId: string;
readonly sentinelMethod: string;
/** Start marker; absent: counting starts when the capture is installed. */
readonly startSentinelId?: string;
readonly stateKey: string;
}
export interface JourneyMainIpcSentinelState {
readonly occurrences: number;
readonly receivedEpochMs: number | null;
}
export interface JourneyMainIpcCaptureState {
readonly callsAfterSentinel: number;
/** Calls before the start marker; always 0 without one. */
readonly callsBeforeStart: number;
/** Calls from the start marker (or install) up to the sentinel. */
readonly callsBeforeSentinel: number;
readonly callsByMethod: Record<string, number>;
/**
* Bridge calls started but not yet completed, per method. The preload
* follows every `start` with exactly one `success` or `error` (sync and
* async results alike), so a capture installed before the document
* loads sees every pair.
*/
readonly inFlightByMethod: Record<string, number>;
readonly installedEpochMs: number;
readonly malformedEvents: number;
readonly processStartEpochMs: number;
readonly senderIds: number[];
readonly sentinel: JourneyMainIpcSentinelState;
/** Completions without a start seen by this capture (installed late). */
readonly unmatchedCompletions: number;
/** Null when the capture has no start marker. */
readonly start: JourneyMainIpcSentinelState | null;
}
export async function installJourneyMainIpcCapture(
electronApp: ElectronApplication,
options: JourneyMainIpcCaptureOptions
): Promise<void> {
await electronApp.evaluate(({ ipcMain }, input) => {
const target = globalThis as unknown as Record<string, unknown>;
if (target[input.stateKey] !== undefined) {
throw new Error('journey-main-ipc-capture-already-installed');
}
const startSentinelId = input.startSentinelId ?? null;
const state = {
callsAfterSentinel: 0,
callsBeforeStart: 0,
callsBeforeSentinel: 0,
callsByMethod: {} as Record<string, number>,
installedEpochMs: Date.now(),
malformedEvents: 0,
processStartEpochMs: Date.now() - process.uptime() * 1000,
inFlightByMethod: {} as Record<string, number>,
senderIds: [] as number[],
sentinel: {
occurrences: 0,
receivedEpochMs: null as number | null,
},
unmatchedCompletions: 0,
start:
startSentinelId === null
? null
: {
occurrences: 0,
receivedEpochMs: null as number | null,
},
};
const carries = (args: unknown, id: string): boolean => {
try {
return JSON.stringify(args ?? null).includes(id);
} catch {
return false;
}
};
target[input.stateKey] = state;
const listener = (
event: { sender: { id: number } },
payload: unknown
): void => {
const record =
typeof payload === 'object' && payload !== null
? (payload as Record<string, unknown>)
: null;
if (!record || typeof record['method'] !== 'string') {
state.malformedEvents += 1;
return;
}
const phase = record['phase'];
if (phase === 'success' || phase === 'error') {
const pending = state.inFlightByMethod[record['method']] ?? 0;
if (pending === 0) {
state.unmatchedCompletions += 1;
} else if (pending === 1) {
delete state.inFlightByMethod[record['method']];
} else {
state.inFlightByMethod[record['method']] = pending - 1;
}
return;
}
if (phase !== 'start') {
return;
}
// Sentinels included: their completions arrive like any other.
state.inFlightByMethod[record['method']] =
(state.inFlightByMethod[record['method']] ?? 0) + 1;
const senderId = event.sender.id;
if (!state.senderIds.includes(senderId)) {
state.senderIds.push(senderId);
}
const method = record['method'];
const isMarker = method === input.sentinelMethod;
if (
isMarker &&
state.start !== null &&
startSentinelId !== null &&
carries(record['args'], startSentinelId)
) {
state.start.occurrences += 1;
// A start marker after the sentinel stays unstamped, which
// the assertion rejects.
if (state.sentinel.receivedEpochMs === null) {
state.start.receivedEpochMs ??= Date.now();
}
return;
}
if (isMarker && carries(record['args'], input.sentinelId)) {
state.sentinel.occurrences += 1;
state.sentinel.receivedEpochMs ??= Date.now();
return;
}
if (state.sentinel.receivedEpochMs !== null) {
state.callsAfterSentinel += 1;
return;
}
if (state.start !== null && state.start.receivedEpochMs === null) {
state.callsBeforeStart += 1;
return;
}
state.callsBeforeSentinel += 1;
state.callsByMethod[method] =
(state.callsByMethod[method] ?? 0) + 1;
};
ipcMain.on(input.channel, listener);
// Kept next to the state (which is read as JSON) so the capture can
// be detached from the same main process later.
target[`${input.stateKey}:detach`] = () => {
ipcMain.removeListener(input.channel, listener);
};
}, options);
}
/**
* Removes a capture's listener. J2 detaches J1's capture once it has used it
* to settle, so the launch listener does not run for every bridge call of
* the measured click.
*/
export async function detachJourneyMainIpcCapture(
electronApp: ElectronApplication,
stateKey: string
): Promise<void> {
await electronApp.evaluate((_electron, key) => {
const target = globalThis as unknown as Record<string, unknown>;
const detach = target[`${key}:detach`];
if (typeof detach !== 'function') {
throw new Error('journey-main-ipc-capture-not-attached');
}
(detach as () => void)();
delete target[`${key}:detach`];
}, stateKey);
}
/** Total of `inFlightByMethod`. */
export function countJourneyMainIpcInFlight(
state: JourneyMainIpcCaptureState
): number {
return Object.values(state.inFlightByMethod).reduce(
(total, count) => total + count,
0
);
}
/** Raw state without waiting for the sentinel, for settling checks. */
export async function peekJourneyMainIpcCapture(
electronApp: ElectronApplication,
stateKey: string
): Promise<JourneyMainIpcCaptureState> {
const [state] = await peekJourneyMainIpcCaptures(electronApp, [stateKey]);
return state as JourneyMainIpcCaptureState;
}
/**
* Several captures read in one synchronous pass in the main process. No
* `ipcMain` event can be handled in between, so the states are one coherent
* snapshot: a call counted by one capture is also pending in the other.
*/
export async function peekJourneyMainIpcCaptures(
electronApp: ElectronApplication,
stateKeys: readonly string[]
): Promise<JourneyMainIpcCaptureState[]> {
const states = (await electronApp.evaluate(
(_electron, keys) =>
JSON.parse(
JSON.stringify(
keys.map(
(key) =>
(globalThis as unknown as Record<string, unknown>)[
key
] ?? null
)
)
) as unknown,
[...stateKeys]
)) as (JourneyMainIpcCaptureState | null)[];
if (states.some((state) => !state)) {
throw new Error('journey-main-ipc-capture-missing');
}
return states as JourneyMainIpcCaptureState[];
}
export async function readJourneyMainIpcCapture(
electronApp: ElectronApplication,
stateKey: string,
timeoutMs: number
): Promise<JourneyMainIpcCaptureState> {
const deadline = Date.now() + timeoutMs;
for (;;) {
const state = await electronApp.evaluate(
(_electron, key) =>
JSON.parse(
JSON.stringify(
(globalThis as unknown as Record<string, unknown>)[key]
)
) as unknown,
stateKey
);
const capture = state as JourneyMainIpcCaptureState | null;
if (capture?.sentinel.receivedEpochMs !== null) {
return assertJourneyMainIpcCapture(capture);
}
if (Date.now() >= deadline) {
throw new Error('journey-main-ipc-capture-sentinel-timeout');
}
await new Promise((resolve) => setTimeout(resolve, 25));
}
}
export function assertJourneyMainIpcCapture(
value: unknown
): JourneyMainIpcCaptureState {
const state = value as JourneyMainIpcCaptureState | null | undefined;
if (!state || typeof state.callsBeforeSentinel !== 'number') {
throw new Error('journey-main-ipc-capture-missing');
}
if (state.sentinel.occurrences !== 1) {
throw new Error(
`journey-main-ipc-capture-sentinel-count-${state.sentinel.occurrences}`
);
}
if (state.senderIds.length !== 1) {
throw new Error(
`journey-main-ipc-capture-senders-${state.senderIds.length}`
);
}
if (state.malformedEvents > 0) {
throw new Error('journey-main-ipc-capture-malformed-events');
}
if (state.start !== null) {
if (state.start.occurrences !== 1) {
throw new Error(
`journey-main-ipc-capture-start-count-${state.start.occurrences}`
);
}
if (
state.start.receivedEpochMs === null ||
state.sentinel.receivedEpochMs === null ||
state.start.receivedEpochMs > state.sentinel.receivedEpochMs
) {
throw new Error('journey-main-ipc-capture-sentinel-before-start');
}
}
return state;
}