test(perf): add the J2 open-source journey

Measure the click on the Xtream portal card until the category list and
the first page of the opened section are painted, in the same fresh
process as J1 after its counters are final and the app has settled.

- journey-renderer-probe: optional click start (capture-phase listener on
  window, start sentinel before the app sees the click, entries before the
  click dropped), companion selectors, recent-input layout shifts tallied
- journey-main-ipc-capture: optional start sentinel; counts calls between
  the two sentinels
- journey-mock-request-ledger: loopback proxy that counts every request
  the app sends to the mock without storing credentials
- open-source-journey-record: J2 counters and evidence
- journey-run / journey-summary: every journey spec of one perf:journeys
  run adds its entry to the same summary.json
- docs: J2 contract in performance-journeys.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
4grayandClaude Opus 5.5 committed 2026-09-29 08:58:36 +02:00
1 parent 76dd8c099e
commit 03767586de
18 files changed
+2130 -147

No files matched your search

@@ -4,7 +4,7 @@ import { defineConfig } from '@playwright/test';
/**
* Performance journeys (docs/architecture/performance-journeys.md). One
* worker, no retries: every journey spawns its own Electron processes and
* writes one summary per run. The Xtream mock serves both the M3U playlist
* adds its entry to the run's single summary file. The Xtream mock serves both the M3U playlist
* and the portal on a dedicated loopback port so a normal E2E server on
* 3211 cannot be reused by accident. The mock runs as one node process
* rather than through `nx run …:serve`, whose detached process group outlived
@@ -15,6 +15,10 @@ import { defineConfig } from '@playwright/test';
const xtreamMockPort =
process.env['IPTVNATOR_JOURNEY_XTREAM_MOCK_PORT'] ?? '3231';
// One summary file per invocation: the runner loads this config before it
// forks the worker, so every journey spec sees the same start time.
process.env['IPTVNATOR_JOURNEY_RUN_STARTED_AT'] ??= new Date().toISOString();
export default defineConfig({
fullyParallel: false,
reporter: [['list']],
@@ -0,0 +1,89 @@
import { relative } from 'node:path';
import { test } from '@playwright/test';
import {
electronMainPath,
packagedRendererIndexPath,
workspaceRoot,
} from '../electron-test-fixtures';
import {
recordJourneySummaryEntry,
resolveJourneyRunSummaryPath,
summarizeJourneyIterations,
type JourneyIterationRecord,
type JourneySummaryHarness,
} from '../performance/journey-summary';
/**
* Run-wide settings and summary output shared by the journey specs.
* Contract: docs/architecture/performance-journeys.md.
*/
export const JOURNEY_WARMUP_ITERATIONS = 1;
export const JOURNEY_MEASURED_ITERATIONS = readPositiveInteger(
'IPTVNATOR_JOURNEY_MEASURED_ITERATIONS',
5
);
export const JOURNEY_ITERATION_TIMEOUT_MS = 120_000;
function readPositiveInteger(name: string, fallback: number): number {
const raw = process.env[name];
if (raw === undefined || raw === '') {
return fallback;
}
const value = Number(raw);
if (!Number.isSafeInteger(value) || value < 1) {
throw new Error(`${name} must be a positive integer`);
}
return value;
}
export function logJourneyIteration(
journeyId: string,
record: JourneyIterationRecord
): void {
console.log(
`[journey:${journeyId}] iteration ${record.index}${record.warmup ? ' (warm-up)' : ''} pid=${record.pid} ${JSON.stringify(
{ ...record.counters, ...record.wallClock }
)}`
);
}
/** Summarizes one journey and adds it to this run's summary file. */
export async function writeJourneyRunEntry(
journeyId: string,
iterations: readonly JourneyIterationRecord[],
unavailable: Readonly<Record<string, string>>,
electronVersion: string
): Promise<void> {
const entry = summarizeJourneyIterations(iterations, unavailable);
const harness: JourneySummaryHarness = {
arch: process.arch,
ci: Boolean(process.env['CI']),
electron: electronVersion,
electronMain: relative(workspaceRoot, electronMainPath),
measuredIterations: JOURNEY_MEASURED_ITERATIONS,
node: process.version,
platform: process.platform,
rendererIndex: relative(workspaceRoot, packagedRendererIndexPath),
warmupIterations: JOURNEY_WARMUP_ITERATIONS,
};
const summaryPath = resolveJourneyRunSummaryPath(workspaceRoot);
await recordJourneySummaryEntry(summaryPath, harness, journeyId, entry);
await test.info().attach(`journey-summary-${journeyId}`, {
body: JSON.stringify(entry, null, 2),
contentType: 'application/json',
});
console.log(
`[journey:${journeyId}] summary ${relative(workspaceRoot, summaryPath)}\n${JSON.stringify(
{
counters: entry.counters,
counterStability: entry.counterStability,
unavailable: entry.unavailable,
wallClock: entry.wallClock,
},
null,
2
)}`
);
}
@@ -2,7 +2,11 @@ import { cp, mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { _electron as electron, type Page } from '@playwright/test';
import {
_electron as electron,
type ElectronApplication,
type Page,
} from '@playwright/test';
import { captureElectronProcess } from '../electron-process-lifecycle';
import {
@@ -111,18 +115,42 @@ export function removeLaunchJourneyProfile(directory: string): Promise<void> {
return removeDirectory(directory);
}
/** The running app after J1 ended, for journeys that continue from there. */
export interface LaunchJourneySession {
readonly electronApp: ElectronApplication;
readonly launch: LaunchJourneyMeasurement;
readonly mainWindow: Page;
}
export async function measureLaunchJourney(
templateDirectory: string,
timeoutMs: number
): Promise<LaunchJourneyMeasurement> {
const { launch } = await runLaunchJourney(
templateDirectory,
timeoutMs,
async () => undefined
);
return launch;
}
/**
* Spawns a fresh Electron process on a copy of the seeded profile. The gate
* hook parks the first renderer load on `about:blank`, which gives Playwright
* a page to attach the renderer probe to; the main-process IPC capture is
* installed next, and only then is the real load released. Both captures are
* therefore in place before the renderer runs any script, and the probe,
* capture and gate records still prove it.
* capture and gate records still prove it. `continueJourney` runs in the
* same process after J1's counters are final, before the app is closed.
*/
export async function measureLaunchJourney(
export async function runLaunchJourney<T>(
templateDirectory: string,
timeoutMs: number
): Promise<LaunchJourneyMeasurement> {
timeoutMs: number,
continueJourney: (session: LaunchJourneySession) => Promise<T>
): Promise<{
readonly continuation: T;
readonly launch: LaunchJourneyMeasurement;
}> {
const dataDirectory = await mkdtemp(
join(tmpdir(), 'iptvnator-journey-launch-')
);
@@ -214,7 +242,7 @@ export async function measureLaunchJourney(
const electronVersion = await electronApp.evaluate(
() => process.versions.electron
);
return {
const launch: LaunchJourneyMeasurement = {
electronVersion,
gate,
ipc,
@@ -223,6 +251,12 @@ export async function measureLaunchJourney(
renderer,
spawnEpochMs,
};
const continuation = await continueJourney({
electronApp,
launch,
mainWindow,
});
return { continuation, launch };
} finally {
await closeElectronApplicationAndConfirmExit(
electronApp,
@@ -1,25 +1,18 @@
import { relative } from 'node:path';
import { test } from '@playwright/test';
import {
electronMainPath,
packagedRendererIndexPath,
workspaceRoot,
} from '../electron-test-fixtures';
import {
JOURNEY_SUMMARY_SCHEMA_VERSION,
resolveJourneySummaryPath,
summarizeJourneyIterations,
writeJourneySummary,
type JourneyIterationRecord,
type JourneySummary,
} from '../performance/journey-summary';
import type { JourneyIterationRecord } from '../performance/journey-summary';
import {
LAUNCH_JOURNEY_ID,
LAUNCH_JOURNEY_UNAVAILABLE_COUNTERS,
toLaunchIterationRecord,
} from '../performance/launch-journey-record';
import {
JOURNEY_ITERATION_TIMEOUT_MS,
JOURNEY_MEASURED_ITERATIONS,
JOURNEY_WARMUP_ITERATIONS,
logJourneyIteration,
writeJourneyRunEntry,
} from './journey-run';
import {
LAUNCH_JOURNEY_MOCK_ORIGIN,
measureLaunchJourney,
@@ -32,25 +25,6 @@ import {
* portal card is visible on /workspace with the inline splash removed.
* Contract: docs/architecture/performance-journeys.md.
*/
const WARMUP_ITERATIONS = 1;
const MEASURED_ITERATIONS = readPositiveInteger(
'IPTVNATOR_JOURNEY_MEASURED_ITERATIONS',
5
);
const ITERATION_TIMEOUT_MS = 120_000;
function readPositiveInteger(name: string, fallback: number): number {
const raw = process.env[name];
if (raw === undefined || raw === '') {
return fallback;
}
const value = Number(raw);
if (!Number.isSafeInteger(value) || value < 1) {
throw new Error(`${name} must be a positive integer`);
}
return value;
}
test.describe.configure({ mode: 'serial' });
test('J1 launch to usable', async () => {
@@ -60,62 +34,26 @@ test('J1 launch to usable', async () => {
const iterations: JourneyIterationRecord[] = [];
let electronVersion = 'unknown';
try {
const total = WARMUP_ITERATIONS + MEASURED_ITERATIONS;
const total = JOURNEY_WARMUP_ITERATIONS + JOURNEY_MEASURED_ITERATIONS;
for (let index = 0; index < total; index += 1) {
const warmup = index < WARMUP_ITERATIONS;
const warmup = index < JOURNEY_WARMUP_ITERATIONS;
const measurement = await measureLaunchJourney(
templateDirectory,
ITERATION_TIMEOUT_MS
JOURNEY_ITERATION_TIMEOUT_MS
);
electronVersion = measurement.electronVersion;
const record = toLaunchIterationRecord(index, warmup, measurement);
iterations.push(record);
console.log(
`[journey:launch] iteration ${index}${warmup ? ' (warm-up)' : ''} pid=${record.pid} ${JSON.stringify(
{ ...record.counters, ...record.wallClock }
)}`
);
logJourneyIteration(LAUNCH_JOURNEY_ID, record);
}
} finally {
await removeLaunchJourneyProfile(templateDirectory);
}
const entry = summarizeJourneyIterations(
await writeJourneyRunEntry(
LAUNCH_JOURNEY_ID,
iterations,
LAUNCH_JOURNEY_UNAVAILABLE_COUNTERS
);
const summary: JourneySummary = {
generatedAt: new Date().toISOString(),
harness: {
arch: process.arch,
ci: Boolean(process.env['CI']),
electron: electronVersion,
electronMain: relative(workspaceRoot, electronMainPath),
measuredIterations: MEASURED_ITERATIONS,
node: process.version,
platform: process.platform,
rendererIndex: relative(workspaceRoot, packagedRendererIndexPath),
warmupIterations: WARMUP_ITERATIONS,
},
journeys: { [LAUNCH_JOURNEY_ID]: entry },
schemaVersion: JOURNEY_SUMMARY_SCHEMA_VERSION,
};
const summaryPath = resolveJourneySummaryPath(workspaceRoot);
await writeJourneySummary(summaryPath, summary);
await test.info().attach('journey-summary', {
contentType: 'application/json',
path: summaryPath,
});
console.log(
`[journey:launch] summary ${relative(workspaceRoot, summaryPath)}\n${JSON.stringify(
{
counters: entry.counters,
counterStability: entry.counterStability,
unavailable: entry.unavailable,
wallClock: entry.wallClock,
},
null,
2
)}`
LAUNCH_JOURNEY_UNAVAILABLE_COUNTERS,
electronVersion
);
});
@@ -0,0 +1,198 @@
import type { ElectronApplication, Page } from '@playwright/test';
import { defaultXtreamPortalName } from '../electron-test-fixtures';
import {
installJourneyMainIpcCapture,
JOURNEY_RENDERER_API_TRACE_CHANNEL,
peekJourneyMainIpcCapture,
readJourneyMainIpcCapture,
} from '../performance/journey-main-ipc-capture';
import type { JourneyMockRequestLedger } from '../performance/journey-mock-request-ledger';
import {
armJourneyRendererProbe,
createOpenSourceJourneyProbeOptions,
waitForJourneyRendererProbe,
type JourneyRendererProbeState,
} from '../performance/journey-renderer-probe';
import type {
OpenSourceJourneyMeasurement,
OpenSourceJourneySettle,
} from '../performance/open-source-journey-record';
import type { LaunchJourneySession } from './launch-journey-app';
/**
* J2 "Open a source": runs inside a process that J1 has just launched, after
* J1's counters are final. The app is first allowed to settle (no DOM
* mutation, bridge call or mock request for `QUIET_MS`), so leftovers of the
* startup are not attributed to the click. Then the Xtream portal card on
* the dashboard is clicked and the probe, the IPC capture and the mock
* request ledger measure until the category list and the first page of
* items are painted.
*/
export const OPEN_SOURCE_JOURNEY_MAIN_IPC_STATE_KEY =
'__iptvnatorJourneyOpenSourceMainIpcCapture';
const QUIET_MS = 1_000;
const POLL_MS = 100;
const SETTLE_TIMEOUT_MS = 30_000;
interface ActivitySample {
readonly domMutations: number;
readonly httpRequests: number;
readonly ipcCalls: number;
}
async function readPreStartMutations(
page: Page,
stateKey: string
): Promise<number> {
return page.evaluate((key) => {
const state = (globalThis as unknown as Record<string, unknown>)[
key
] as { preStart?: { domMutations?: number } } | undefined;
const count = state?.preStart?.domMutations;
if (typeof count !== 'number') {
throw new Error('journey-renderer-probe-not-armed');
}
return count;
}, stateKey);
}
/**
* Waits until DOM, bridge and mock traffic have all been unchanged for
* `QUIET_MS`. An app that never settles fails the iteration instead of
* producing a count that includes its background work.
*/
async function waitForQuiet(
electronApp: ElectronApplication,
page: Page,
ledger: JourneyMockRequestLedger,
probeStateKey: string
): Promise<OpenSourceJourneySettle> {
const startedMs = Date.now();
const armMark = ledger.mark();
const sample = async (): Promise<ActivitySample> => ({
domMutations: await readPreStartMutations(page, probeStateKey),
httpRequests: ledger.mark(),
ipcCalls: (
await peekJourneyMainIpcCapture(
electronApp,
OPEN_SOURCE_JOURNEY_MAIN_IPC_STATE_KEY
)
).callsBeforeStart,
});
let last = await sample();
let quietSinceMs = Date.now();
for (;;) {
await new Promise((resolve) => setTimeout(resolve, POLL_MS));
const next = await sample();
const now = Date.now();
if (
next.domMutations !== last.domMutations ||
next.httpRequests !== last.httpRequests ||
next.ipcCalls !== last.ipcCalls
) {
last = next;
quietSinceMs = now;
} else if (now - quietSinceMs >= QUIET_MS) {
return {
preStartDomMutations: next.domMutations,
preStartHttpRequests: next.httpRequests - armMark,
preStartIpcCalls: next.ipcCalls,
quietMs: QUIET_MS,
waitedMs: now - startedMs,
};
}
if (now - startedMs > SETTLE_TIMEOUT_MS) {
throw new Error(
`open-source-journey-not-quiet: ${JSON.stringify(next)}`
);
}
}
}
/** Waits until the mock has seen no request for `QUIET_MS`. */
async function waitForMockQuiet(
ledger: JourneyMockRequestLedger
): Promise<void> {
const startedMs = Date.now();
let count = ledger.mark();
let quietSinceMs = Date.now();
for (;;) {
await new Promise((resolve) => setTimeout(resolve, POLL_MS));
const now = Date.now();
if (ledger.mark() !== count) {
count = ledger.mark();
quietSinceMs = now;
} else if (now - quietSinceMs >= QUIET_MS) {
return;
}
if (now - startedMs > SETTLE_TIMEOUT_MS) {
throw new Error('open-source-journey-mock-not-quiet');
}
}
}
/**
* `spawnLedgerMark` is the ledger position taken before the process was
* spawned, so the launch's own mock traffic is kept as evidence.
*/
export async function measureOpenSourceJourney(
session: LaunchJourneySession,
ledger: JourneyMockRequestLedger,
spawnLedgerMark: number,
timeoutMs: number
): Promise<OpenSourceJourneyMeasurement> {
const { electronApp, mainWindow } = session;
const probeOptions = createOpenSourceJourneyProbeOptions();
const startClick = probeOptions.startClick;
if (!startClick) {
throw new Error('open-source-journey-probe-without-start');
}
await installJourneyMainIpcCapture(electronApp, {
channel: JOURNEY_RENDERER_API_TRACE_CHANNEL,
sentinelId: probeOptions.sentinelId,
sentinelMethod: probeOptions.sentinelMethod,
startSentinelId: startClick.sentinelId,
stateKey: OPEN_SOURCE_JOURNEY_MAIN_IPC_STATE_KEY,
});
await armJourneyRendererProbe(mainWindow, probeOptions);
const card = mainWindow
.locator(startClick.selector)
.filter({ hasText: defaultXtreamPortalName })
.first();
// Hover first so hover effects (and anything they trigger) happen
// before the app settles, not inside the measured window.
await card.hover({ timeout: timeoutMs });
const settle = await waitForQuiet(
electronApp,
mainWindow,
ledger,
probeOptions.stateKey
);
const ledgerMark = ledger.mark();
await card.click({ timeout: timeoutMs });
const renderer: JourneyRendererProbeState =
await waitForJourneyRendererProbe(
mainWindow,
probeOptions.stateKey,
timeoutMs
);
const ipc = await readJourneyMainIpcCapture(
electronApp,
OPEN_SOURCE_JOURNEY_MAIN_IPC_STATE_KEY,
10_000
);
await waitForMockQuiet(ledger);
return {
http: {
beforeClick: ledger
.since(spawnLedgerMark)
.slice(0, ledgerMark - spawnLedgerMark),
requests: ledger.since(ledgerMark),
},
ipc,
pid: session.launch.pid,
renderer,
settle,
};
}
@@ -0,0 +1,83 @@
import { test } from '@playwright/test';
import { startJourneyMockRequestLedger } from '../performance/journey-mock-request-ledger';
import type { JourneyIterationRecord } from '../performance/journey-summary';
import {
OPEN_SOURCE_JOURNEY_ID,
OPEN_SOURCE_JOURNEY_UNAVAILABLE_COUNTERS,
toOpenSourceIterationRecord,
} from '../performance/open-source-journey-record';
import {
JOURNEY_ITERATION_TIMEOUT_MS,
JOURNEY_MEASURED_ITERATIONS,
JOURNEY_WARMUP_ITERATIONS,
logJourneyIteration,
writeJourneyRunEntry,
} from './journey-run';
import {
LAUNCH_JOURNEY_MOCK_ORIGIN,
removeLaunchJourneyProfile,
runLaunchJourney,
seedLaunchJourneyProfile,
} from './launch-journey-app';
import { measureOpenSourceJourney } from './open-source-journey-app';
/**
* J2 "Open a source": click on the Xtream portal card on the dashboard until
* the section's category list and first page of items are painted. Every
* iteration is a fresh J1 launch on a copy of the seeded profile; the click
* happens after J1's terminal condition and after the app has settled.
* The profile is seeded through the mock request ledger, so every request
* the app sends to the mock is counted.
* Contract: docs/architecture/performance-journeys.md.
*/
test.describe.configure({ mode: 'serial' });
test('J2 open a source', async () => {
const ledger = await startJourneyMockRequestLedger(
LAUNCH_JOURNEY_MOCK_ORIGIN
);
const iterations: JourneyIterationRecord[] = [];
let electronVersion = 'unknown';
try {
const templateDirectory = await seedLaunchJourneyProfile(ledger.origin);
try {
const total =
JOURNEY_WARMUP_ITERATIONS + JOURNEY_MEASURED_ITERATIONS;
for (let index = 0; index < total; index += 1) {
const warmup = index < JOURNEY_WARMUP_ITERATIONS;
const spawnLedgerMark = ledger.mark();
const { continuation, launch } = await runLaunchJourney(
templateDirectory,
JOURNEY_ITERATION_TIMEOUT_MS,
(session) =>
measureOpenSourceJourney(
session,
ledger,
spawnLedgerMark,
JOURNEY_ITERATION_TIMEOUT_MS
)
);
electronVersion = launch.electronVersion;
const record = toOpenSourceIterationRecord(
index,
warmup,
continuation
);
iterations.push(record);
logJourneyIteration(OPEN_SOURCE_JOURNEY_ID, record);
}
} finally {
await removeLaunchJourneyProfile(templateDirectory);
}
} finally {
await ledger.close();
}
await writeJourneyRunEntry(
OPEN_SOURCE_JOURNEY_ID,
iterations,
OPEN_SOURCE_JOURNEY_UNAVAILABLE_COUNTERS,
electronVersion
);
});
@@ -1,14 +1,24 @@
import assert from 'node:assert/strict';
import { EventEmitter } from 'node:events';
import { readFileSync } from 'node:fs';
import { resolve } from 'node:path';
import test from 'node:test';
import type { ElectronApplication } from '@playwright/test';
import {
assertJourneyMainIpcCapture,
installJourneyMainIpcCapture,
JOURNEY_RENDERER_API_TRACE_CHANNEL,
peekJourneyMainIpcCapture,
type JourneyMainIpcCaptureOptions,
type JourneyMainIpcCaptureState,
} from './journey-main-ipc-capture';
import { JOURNEY_IPC_SENTINEL_METHOD } from './journey-renderer-probe';
import {
JOURNEY_IPC_SENTINEL_METHOD,
JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
} from './journey-renderer-probe';
const electronBackendSource = resolve(
__dirname,
@@ -20,6 +30,7 @@ function validCapture(
): JourneyMainIpcCaptureState {
return {
callsAfterSentinel: 2,
callsBeforeStart: 0,
callsBeforeSentinel: 7,
callsByMethod: { dbGetAppPlaylists: 1, getSettings: 6 },
installedEpochMs: 1,
@@ -27,6 +38,7 @@ function validCapture(
processStartEpochMs: 0,
senderIds: [1],
sentinel: { occurrences: 1, receivedEpochMs: 2 },
start: null,
...overrides,
};
}
@@ -102,3 +114,181 @@ test('rejects captures that cannot bound the counter exactly', () => {
/malformed/
);
});
/**
* Runs the capture's main-process function in this process against a fake
* `ipcMain`, the way `electronApp.evaluate` runs it in Electron.
*/
function createFakeElectronApp(): {
readonly app: ElectronApplication;
send(
senderId: number,
method: string,
args: unknown[],
phase?: string
): void;
} {
const ipcMain = new EventEmitter();
let channel = '';
const app = {
evaluate: async (
fn: (electron: unknown, arg: unknown) => unknown,
arg: unknown
) => {
channel =
(arg as Partial<JourneyMainIpcCaptureOptions>).channel ??
channel;
return fn({ ipcMain }, arg);
},
} as unknown as ElectronApplication;
return {
app,
send: (senderId, method, args, phase = 'start') => {
ipcMain.emit(
channel,
{ sender: { id: senderId } },
{ args, method, phase }
);
},
};
}
async function withCapture(
options: Partial<JourneyMainIpcCaptureOptions>,
run: (
fake: ReturnType<typeof createFakeElectronApp>,
read: () => Promise<JourneyMainIpcCaptureState>
) => Promise<void>
): Promise<void> {
const stateKey = `__journeyIpcCaptureTest${Math.random()}`;
const fake = createFakeElectronApp();
try {
await installJourneyMainIpcCapture(fake.app, {
channel: JOURNEY_RENDERER_API_TRACE_CHANNEL,
sentinelId: JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
sentinelMethod: JOURNEY_IPC_SENTINEL_METHOD,
stateKey,
...options,
});
await run(fake, () => peekJourneyMainIpcCapture(fake.app, stateKey));
} finally {
delete (globalThis as unknown as Record<string, unknown>)[stateKey];
}
}
test('without a start marker, counts every call from install to the sentinel', async () => {
await withCapture({}, async (fake, read) => {
fake.send(1, 'getSettings', []);
fake.send(1, 'getSettings', [], 'end');
fake.send(1, 'onSomething', []);
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [
JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
]);
fake.send(1, 'dbGetAppPlaylists', []);
const state = assertJourneyMainIpcCapture(await read());
assert.equal(state.start, null);
assert.equal(state.callsBeforeStart, 0);
assert.equal(state.callsBeforeSentinel, 2);
assert.equal(state.callsAfterSentinel, 1);
assert.deepEqual(state.callsByMethod, {
getSettings: 1,
onSomething: 1,
});
});
});
test('with a start marker, counts only the calls between the two sentinels', async () => {
await withCapture(
{ startSentinelId: JOURNEY_OPEN_SOURCE_START_SENTINEL_ID },
async (fake, read) => {
fake.send(1, 'getSettings', []);
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, ['other-playlist']);
assert.equal((await read()).callsBeforeStart, 2);
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
]);
fake.send(1, 'dbGetAppPlaylist', ['playlist-1']);
fake.send(1, 'xtreamRequest', [{ action: 'get_account_info' }]);
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [
JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
]);
fake.send(1, 'getSettings', []);
const state = assertJourneyMainIpcCapture(await read());
assert.equal(state.callsBeforeStart, 2);
assert.equal(state.callsBeforeSentinel, 2);
assert.equal(state.callsAfterSentinel, 1);
assert.deepEqual(state.callsByMethod, {
dbGetAppPlaylist: 1,
xtreamRequest: 1,
});
assert.equal(state.start?.occurrences, 1);
}
);
});
test('rejects a start marker that is missing, repeated or after the sentinel', async () => {
await withCapture(
{ startSentinelId: JOURNEY_OPEN_SOURCE_START_SENTINEL_ID },
async (fake, read) => {
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [
JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
]);
const early = await read();
const state = () => early;
assert.throws(
() => assertJourneyMainIpcCapture(state()),
/start-count-0/
);
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
]);
const late = await read();
assert.equal(late.start?.occurrences, 1);
assert.throws(
() => assertJourneyMainIpcCapture(late),
/sentinel-before-start/
);
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
]);
const repeated = await read();
assert.throws(
() => assertJourneyMainIpcCapture(repeated),
/start-count-2/
);
}
);
assert.throws(
() =>
assertJourneyMainIpcCapture(
validCapture({
start: { occurrences: 1, receivedEpochMs: 3 },
})
),
/sentinel-before-start/
);
assert.throws(
() =>
assertJourneyMainIpcCapture(
validCapture({
start: { occurrences: 1, receivedEpochMs: null },
})
),
/sentinel-before-start/
);
assert.throws(
() =>
assertJourneyMainIpcCapture(
validCapture({
start: { occurrences: 2, receivedEpochMs: 1 },
})
),
/start-count-2/
);
assert.equal(
assertJourneyMainIpcCapture(
validCapture({ start: { occurrences: 1, receivedEpochMs: 1 } })
).callsBeforeSentinel,
7
);
});
@@ -11,6 +11,11 @@ import type { ElectronApplication } from '@playwright/test';
* until the renderer probe's sentinel call arrives. Renderer-to-main IPC is
* delivered in order, so every call started before the sentinel is counted
* and nothing after it is.
*
* A journey that starts inside a running app (J2 "Open a source") passes
* `startSentinelId`: the probe sends that id before the app handles the
* start click, calls before it are only tallied in `callsBeforeStart`, and
* `callsBeforeSentinel` then counts the calls between the two sentinels.
*/
/** Literal of `DEBUG_TRACE_EVENT_CHANNEL` in `services/debug-trace.ts`. */
@@ -21,21 +26,30 @@ export interface JourneyMainIpcCaptureOptions {
readonly channel: string;
readonly sentinelId: string;
readonly sentinelMethod: string;
/** Start marker; absent: counting starts when the capture is installed. */
readonly startSentinelId?: string;
readonly stateKey: string;
}
export interface JourneyMainIpcSentinelState {
readonly occurrences: number;
readonly receivedEpochMs: number | null;
}
export interface JourneyMainIpcCaptureState {
readonly callsAfterSentinel: number;
/** Calls before the start marker; always 0 without one. */
readonly callsBeforeStart: number;
/** Calls from the start marker (or install) up to the sentinel. */
readonly callsBeforeSentinel: number;
readonly callsByMethod: Record<string, number>;
readonly installedEpochMs: number;
readonly malformedEvents: number;
readonly processStartEpochMs: number;
readonly senderIds: number[];
readonly sentinel: {
readonly occurrences: number;
readonly receivedEpochMs: number | null;
};
readonly sentinel: JourneyMainIpcSentinelState;
/** Null when the capture has no start marker. */
readonly start: JourneyMainIpcSentinelState | null;
}
export async function installJourneyMainIpcCapture(
@@ -47,8 +61,10 @@ export async function installJourneyMainIpcCapture(
if (target[input.stateKey] !== undefined) {
throw new Error('journey-main-ipc-capture-already-installed');
}
const startSentinelId = input.startSentinelId ?? null;
const state = {
callsAfterSentinel: 0,
callsBeforeStart: 0,
callsBeforeSentinel: 0,
callsByMethod: {} as Record<string, number>,
installedEpochMs: Date.now(),
@@ -59,6 +75,20 @@ export async function installJourneyMainIpcCapture(
occurrences: 0,
receivedEpochMs: null as number | null,
},
start:
startSentinelId === null
? null
: {
occurrences: 0,
receivedEpochMs: null as number | null,
},
};
const carries = (args: unknown, id: string): boolean => {
try {
return JSON.stringify(args ?? null).includes(id);
} catch {
return false;
}
};
target[input.stateKey] = state;
ipcMain.on(input.channel, (event, payload: unknown) => {
@@ -78,17 +108,22 @@ export async function installJourneyMainIpcCapture(
state.senderIds.push(senderId);
}
const method = record['method'];
let isSentinel = false;
if (method === input.sentinelMethod) {
try {
isSentinel = JSON.stringify(
record['args'] ?? null
).includes(input.sentinelId);
} catch {
isSentinel = false;
const isMarker = method === input.sentinelMethod;
if (
isMarker &&
state.start !== null &&
startSentinelId !== null &&
carries(record['args'], startSentinelId)
) {
state.start.occurrences += 1;
// A start marker after the sentinel stays unstamped, which
// the assertion rejects.
if (state.sentinel.receivedEpochMs === null) {
state.start.receivedEpochMs ??= Date.now();
}
return;
}
if (isSentinel) {
if (isMarker && carries(record['args'], input.sentinelId)) {
state.sentinel.occurrences += 1;
state.sentinel.receivedEpochMs ??= Date.now();
return;
@@ -97,6 +132,10 @@ export async function installJourneyMainIpcCapture(
state.callsAfterSentinel += 1;
return;
}
if (state.start !== null && state.start.receivedEpochMs === null) {
state.callsBeforeStart += 1;
return;
}
state.callsBeforeSentinel += 1;
state.callsByMethod[method] =
(state.callsByMethod[method] ?? 0) + 1;
@@ -104,6 +143,27 @@ export async function installJourneyMainIpcCapture(
}, options);
}
/** Raw state without waiting for the sentinel, for settling checks. */
export async function peekJourneyMainIpcCapture(
electronApp: ElectronApplication,
stateKey: string
): Promise<JourneyMainIpcCaptureState> {
const state = (await electronApp.evaluate(
(_electron, key) =>
JSON.parse(
JSON.stringify(
(globalThis as unknown as Record<string, unknown>)[key] ??
null
)
) as unknown,
stateKey
)) as JourneyMainIpcCaptureState | null;
if (!state) {
throw new Error('journey-main-ipc-capture-missing');
}
return state;
}
export async function readJourneyMainIpcCapture(
electronApp: ElectronApplication,
stateKey: string,
@@ -151,5 +211,19 @@ export function assertJourneyMainIpcCapture(
if (state.malformedEvents > 0) {
throw new Error('journey-main-ipc-capture-malformed-events');
}
if (state.start !== null) {
if (state.start.occurrences !== 1) {
throw new Error(
`journey-main-ipc-capture-start-count-${state.start.occurrences}`
);
}
if (
state.start.receivedEpochMs === null ||
state.sentinel.receivedEpochMs === null ||
state.start.receivedEpochMs > state.sentinel.receivedEpochMs
) {
throw new Error('journey-main-ipc-capture-sentinel-before-start');
}
}
return state;
}
@@ -0,0 +1,110 @@
import assert from 'node:assert/strict';
import { createServer, type Server } from 'node:http';
import type { AddressInfo } from 'node:net';
import test from 'node:test';
import {
countJourneyMockRoutes,
describeJourneyMockRoute,
startJourneyMockRequestLedger,
} from './journey-mock-request-ledger';
test('routes keep the Xtream action but never the credentials', () => {
assert.equal(
describeJourneyMockRoute(
'/player_api.php?username=user1&password=pass1&action=get_live_streams&category_id=3'
),
'/player_api.php?action=get_live_streams'
);
assert.equal(
describeJourneyMockRoute(
'/player_api.php?username=user1&password=pass1'
),
'/player_api.php'
);
assert.equal(
describeJourneyMockRoute('/live/user1/pass1/1001.m3u8'),
'/live/:username/:password/1001.m3u8'
);
assert.equal(
describeJourneyMockRoute('/movie/user1/pass1/2001.mp4?token=x'),
'/movie/:username/:password/2001.mp4'
);
assert.equal(describeJourneyMockRoute('/playlist.m3u'), '/playlist.m3u');
assert.equal(describeJourneyMockRoute(undefined), '/');
});
test('counts requests per route in a stable order', () => {
assert.deepEqual(
countJourneyMockRoutes([
{ epochMs: 1, method: 'GET', route: '/b', sequence: 0 },
{ epochMs: 2, method: 'GET', route: '/a', sequence: 1 },
{ epochMs: 3, method: 'GET', route: '/b', sequence: 2 },
]),
{ '/a': 1, '/b': 2 }
);
});
async function startUpstream(): Promise<{ origin: string; server: Server }> {
const server = createServer((request, response) => {
response.setHeader('x-upstream-host', request.headers.host ?? '');
response.end(`upstream:${request.method}:${request.url}`);
});
await new Promise<void>((resolve) =>
server.listen(0, '127.0.0.1', () => resolve())
);
const { port } = server.address() as AddressInfo;
return { origin: `http://127.0.0.1:${port}`, server };
}
test('forwards every request to the mock and records it from a mark', async () => {
const upstream = await startUpstream();
const ledger = await startJourneyMockRequestLedger(upstream.origin);
try {
assert.notEqual(ledger.origin, upstream.origin);
const first = await fetch(`${ledger.origin}/playlist.m3u`);
assert.equal(await first.text(), 'upstream:GET:/playlist.m3u');
assert.equal(
first.headers.get('x-upstream-host'),
new URL(upstream.origin).host
);
const mark = ledger.mark();
assert.equal(mark, 1);
const second = await fetch(
`${ledger.origin}/player_api.php?username=u&password=p&action=get_account_info`
);
assert.equal(
await second.text(),
'upstream:GET:/player_api.php?username=u&password=p&action=get_account_info'
);
const since = ledger.since(mark);
assert.equal(since.length, 1);
assert.equal(
since[0]?.route,
'/player_api.php?action=get_account_info'
);
assert.equal(since[0]?.method, 'GET');
assert.equal(since[0]?.sequence, 1);
assert.ok(!JSON.stringify(ledger.since(0)).includes('password'));
} finally {
await ledger.close();
await new Promise((resolve) => upstream.server.close(resolve));
}
});
test('answers 502 when the mock is gone and refuses non-HTTP targets', async () => {
const upstream = await startUpstream();
await new Promise((resolve) => upstream.server.close(resolve));
const ledger = await startJourneyMockRequestLedger(upstream.origin);
try {
const response = await fetch(`${ledger.origin}/health`);
assert.equal(response.status, 502);
assert.equal(ledger.mark(), 1);
} finally {
await ledger.close();
}
await assert.rejects(
startJourneyMockRequestLedger('https://127.0.0.1:1'),
/http-only/
);
});
@@ -0,0 +1,134 @@
import {
createServer,
request as httpRequest,
type IncomingMessage,
type Server,
type ServerResponse,
} from 'node:http';
import type { AddressInfo } from 'node:net';
/**
* Counts the HTTP requests the app sends to the Xtream mock.
*
* The journey profile is seeded with the origin of this loopback proxy
* instead of the mock's own origin, so every request for the portal and the
* M3U source passes through it, whether the main process sends it (Xtream
* API, M3U) or the renderer does (artwork served by the mock). The mock's
* `/__control/state` ledger only exists in performance-control mode, which
* disables `/playlist.m3u` and tracks only the 100k scenario, and a
* Playwright request listener sees renderer traffic only.
*
* The ledger keeps the method, the path and, for `player_api.php`, the
* `action` parameter. Query strings carry the portal credentials and are
* never stored.
*/
export interface JourneyMockRequest {
readonly epochMs: number;
readonly method: string;
readonly route: string;
readonly sequence: number;
}
export interface JourneyMockRequestLedger {
close(): Promise<void>;
/** Sequence number of the next request; pass it to `since`. */
mark(): number;
readonly origin: string;
since(mark: number): readonly JourneyMockRequest[];
}
/** `GET /player_api.php?action=get_live_streams&...` → `/player_api.php?action=get_live_streams`. */
export function describeJourneyMockRoute(rawUrl: string | undefined): string {
const url = new URL(rawUrl ?? '/', 'http://journey.invalid');
const action = url.searchParams.get('action');
if (url.pathname.endsWith('/player_api.php')) {
return action === null
? url.pathname
: `${url.pathname}?action=${encodeURIComponent(action)}`;
}
// Stream paths embed the credentials: /live/<user>/<password>/<id>.ts.
const segments = url.pathname.split('/');
if (
['live', 'movie', 'series', 'timeshift'].includes(segments[1] ?? '') &&
segments.length > 3
) {
segments[2] = ':username';
segments[3] = ':password';
}
return segments.join('/');
}
/** Route → count, sorted by route, for evidence. */
export function countJourneyMockRoutes(
requests: readonly JourneyMockRequest[]
): Record<string, number> {
const counts = new Map<string, number>();
for (const entry of requests) {
counts.set(entry.route, (counts.get(entry.route) ?? 0) + 1);
}
return Object.fromEntries(
[...counts.entries()].sort(([left], [right]) =>
left.localeCompare(right)
)
);
}
export async function startJourneyMockRequestLedger(
targetOrigin: string
): Promise<JourneyMockRequestLedger> {
const target = new URL(targetOrigin);
if (target.protocol !== 'http:') {
throw new Error('journey-mock-ledger-http-only');
}
const requests: JourneyMockRequest[] = [];
const forward = (
incoming: IncomingMessage,
outgoing: ServerResponse
): void => {
requests.push({
epochMs: Date.now(),
method: incoming.method ?? 'GET',
route: describeJourneyMockRoute(incoming.url),
sequence: requests.length,
});
const upstream = httpRequest(
{
headers: { ...incoming.headers, host: target.host },
hostname: target.hostname,
method: incoming.method,
path: incoming.url,
port: target.port,
},
(response) => {
outgoing.writeHead(
response.statusCode ?? 502,
response.headers
);
response.pipe(outgoing);
}
);
upstream.on('error', () => {
if (!outgoing.headersSent) {
outgoing.writeHead(502);
}
outgoing.end();
});
incoming.pipe(upstream);
};
const server: Server = createServer(forward);
await new Promise<void>((resolve, reject) => {
server.once('error', reject);
server.listen(0, '127.0.0.1', () => resolve());
});
const { port } = server.address() as AddressInfo;
return {
close: () =>
new Promise<void>((resolve) => {
server.closeAllConnections();
server.close(() => resolve());
}),
mark: () => requests.length,
origin: `http://127.0.0.1:${port}`,
since: (mark) => requests.slice(mark),
};
}
@@ -6,8 +6,12 @@ import { JSDOM } from 'jsdom';
import {
assertJourneyRendererProbeState,
createLaunchJourneyProbeOptions,
createOpenSourceJourneyProbeOptions,
JOURNEY_IPC_SENTINEL_ID,
JOURNEY_IPC_SENTINEL_METHOD,
JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
JOURNEY_OPEN_SOURCE_PROBE_STATE_KEY,
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
JOURNEY_PROBE_STATE_KEY,
journeyRendererProbeScript,
type JourneyRendererProbeOptions,
@@ -108,6 +112,18 @@ function createFixture(
runScripts: 'outside-only',
url,
});
return createFixtureFromDom(
dom,
{ ...createLaunchJourneyProbeOptions(), ...optionOverrides },
bridge
);
}
function createFixtureFromDom(
dom: JSDOM,
options: JourneyRendererProbeOptions,
bridge: boolean
): Fixture {
const { window } = dom;
const observers: FakeObserver[] = [];
const bridgeCalls: unknown[] = [];
@@ -132,10 +148,6 @@ function createFixture(
}),
});
}
const options = {
...createLaunchJourneyProbeOptions(),
...optionOverrides,
};
window.eval(
`(${journeyRendererProbeScript.toString()})(${JSON.stringify(options)})`
);
@@ -407,3 +419,251 @@ test('launch options target the workspace source cards and the shared sentinel',
assert.match(options.cardSelector, /dashboard-recent-sources-rail-card/);
assert.match(options.cardSelector, /app-playlist-item/);
});
// J2 "Open a source": the probe is armed in a loaded document and starts at
// the click on the portal card.
const DASHBOARD_URL = 'http://localhost/workspace/dashboard';
function createOpenSourceFixture(
overrides: { bridge?: boolean } = {}
): Fixture & { readonly card: HTMLElement } {
const dom = new JSDOM(
`<!doctype html><html><body><app-root>
<app-workspace-context-panel></app-workspace-context-panel>
<main></main></app-root></body></html>`,
{
pretendToBeVisual: true,
runScripts: 'outside-only',
url: DASHBOARD_URL,
}
);
const card = dom.window.document.createElement('div');
card.setAttribute('data-test-id', 'dashboard-recent-sources-rail-card');
card.innerHTML = '<a><span class="title">Mock Xtream Portal</span></a>';
dom.window.document.querySelector('main')?.append(card);
const fixture = createFixtureFromDom(
dom,
createOpenSourceJourneyProbeOptions(),
overrides.bridge ?? true
);
return { ...fixture, card };
}
function openSource(
fixture: Fixture,
parts: { categories?: boolean; items?: boolean } = {}
): void {
const { categories = true, items = true } = parts;
const { document, history } = fixture.window;
history.pushState({}, '', '/workspace/xtreams/playlist-1/vod');
document.querySelector('main')?.replaceChildren();
if (categories) {
const category = document.createElement('div');
category.className = 'category-item';
document.querySelector('app-workspace-context-panel')?.append(category);
}
if (items) {
const grid = document.createElement('app-grid-list');
grid.append(
document.createElement('mat-card'),
document.createElement('mat-card')
);
document.querySelector('main')?.append(grid);
}
}
test('a click-started probe only tracks activity before the click inside the start selector', async () => {
const fixture = createOpenSourceFixture();
const { document } = fixture.window;
assert.deepEqual(fixture.state().invalidReasons, []);
assert.equal(fixture.state().start, null);
document.body.append(document.createElement('div'));
await settle();
document.body.click();
await settle();
const before = fixture.state();
assert.equal(before.start, null);
assert.equal(before.counters.domMutations, 0);
assert.equal(before.preStart.domMutations, 1);
assert.equal(typeof before.preStart.lastMutationEpochMs, 'number');
assert.deepEqual(fixture.bridgeCalls, []);
// Already on a page with cards and categories: nothing ends before the
// start.
assert.equal(before.terminal, null);
});
test('the click sends the start sentinel before the app sees it and the end sentinel when the first page is visible', async () => {
const fixture = createOpenSourceFixture();
const { document } = fixture.window;
const order: string[] = [];
fixture.card.addEventListener('click', () => {
order.push(`app:${fixture.bridgeCalls.length}`);
openSource(fixture);
});
(fixture.card.querySelector('.title') as HTMLElement).click();
await settle();
const state = fixture.state();
assert.deepEqual(order, ['app:1']);
assert.deepEqual(fixture.bridgeCalls, [
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
]);
assert.ok(state.start, 'start must be recorded');
assert.equal(state.start.sentinelStatus, 'sent');
assert.equal(
state.start.targetTestId,
'dashboard-recent-sources-rail-card'
);
assert.equal(state.start.pathname, '/workspace/dashboard');
assert.ok(state.start.epochMs <= state.start.listenerEpochMs);
assert.ok(state.terminal, 'terminal must be recorded');
assert.equal(state.terminal.pathname, '/workspace/xtreams/playlist-1/vod');
assert.equal(state.terminal.cardTag, 'mat-card');
assert.equal(state.terminal.cardCount, 2);
assert.deepEqual(state.terminal.companionCounts, [1]);
// main emptied (1) + category (1) + grid (1)
assert.equal(state.counters.domMutations, 3);
assert.equal(state.navigation, null);
assert.equal(state.final, true);
assert.ok(state.terminal.epochMs >= state.start.epochMs);
assert.doesNotThrow(() => assertJourneyRendererProbeState(state));
// One start per armed probe.
document.body.append(document.createElement('div'));
fixture.card.click();
await settle();
assert.equal(fixture.bridgeCalls.length, 2);
});
test('the first page needs the category list as well as the items', async () => {
const fixture = createOpenSourceFixture();
fixture.card.addEventListener('click', () =>
openSource(fixture, { categories: false })
);
fixture.card.click();
await settle();
assert.equal(fixture.state().terminal, null);
const category = fixture.window.document.createElement('div');
category.className = 'category-item';
fixture.window.document
.querySelector('app-workspace-context-panel')
?.append(category);
await settle();
const state = fixture.state();
assert.ok(state.terminal);
assert.equal(state.counters.domMutations, 3);
const skeletons = createOpenSourceFixture();
skeletons.card.addEventListener('click', () => {
openSource(skeletons, { items: false });
const skeleton = skeletons.window.document.createElement('div');
skeleton.className = 'grid-skeleton-card';
skeletons.window.document.querySelector('main')?.append(skeleton);
});
skeletons.card.click();
await settle();
assert.equal(skeletons.state().terminal, null);
});
test('drops performance entries from before the click and keeps recent-input shifts apart', async () => {
const fixture = createOpenSourceFixture();
const [layoutShift, longTask] = fixture.observers as [
FakeObserver,
FakeObserver,
];
const now = () => fixture.window.performance.now();
const beforeClick = now() - 1;
layoutShift.emit([
{
entryType: 'layout-shift',
hadRecentInput: false,
startTime: beforeClick,
value: 3,
},
]);
longTask.emit([
{ duration: 400, entryType: 'longtask', startTime: beforeClick },
]);
fixture.card.click();
await settle(5);
// Delivered after the click but started before it (buffered J1 entries).
layoutShift.emit([
{
entryType: 'layout-shift',
hadRecentInput: false,
startTime: beforeClick,
value: 2,
},
{
entryType: 'layout-shift',
hadRecentInput: true,
startTime: now(),
value: 0.25,
},
{
entryType: 'layout-shift',
hadRecentInput: false,
startTime: now(),
value: 0.125,
},
]);
longTask.emit([
{ duration: 250, entryType: 'longtask', startTime: beforeClick },
{ duration: 90, entryType: 'longtask', startTime: now() },
]);
openSource(fixture);
await settle();
const state = fixture.state();
assert.equal(state.final, true);
assert.equal(state.counters.layoutShiftScore, 0.125);
assert.equal(state.counters.recentInputLayoutShiftScore, 0.25);
assert.equal(state.counters.longTasks, 1);
assert.deepEqual(state.longTaskDurationsMs, [90]);
});
test('rejects a click start whose sentinel could not be sent', async () => {
const fixture = createOpenSourceFixture({ bridge: false });
fixture.card.addEventListener('click', () => openSource(fixture));
fixture.card.click();
await settle();
const state = fixture.state();
assert.equal(state.start?.sentinelStatus, 'bridge-missing');
assert.throws(
() => assertJourneyRendererProbeState(state),
/sentinel-bridge-missing/
);
const started = {
...state,
sentinel: { epochMs: 1, status: 'sent' as const },
};
assert.throws(
() => assertJourneyRendererProbeState(started),
/start-sentinel-bridge-missing/
);
});
test('open-source options start at the portal card and end on the source route', () => {
const options = createOpenSourceJourneyProbeOptions();
assert.equal(options.journey, 'open-source');
assert.equal(options.stateKey, JOURNEY_OPEN_SOURCE_PROBE_STATE_KEY);
assert.notEqual(options.stateKey, JOURNEY_PROBE_STATE_KEY);
assert.equal(options.sentinelId, JOURNEY_OPEN_SOURCE_END_SENTINEL_ID);
assert.equal(
options.startClick?.sentinelId,
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID
);
assert.notEqual(options.sentinelId, JOURNEY_IPC_SENTINEL_ID);
assert.match(
options.startClick?.selector ?? '',
/dashboard-recent-sources-rail-card/
);
assert.match(options.startClick?.selector ?? '', /app-playlist-item/);
assert.equal(options.routeFragment, '/workspace/xtreams/');
assert.match(options.cardSelector, /app-grid-list mat-card/);
assert.match(options.cardSelector, /channel-item/);
assert.deepEqual(options.companionSelectors, [
'app-workspace-context-panel .category-item',
]);
});
@@ -1,32 +1,54 @@
import type { Page } from '@playwright/test';
/**
* Renderer-side probe for the performance journeys (J1 "Launch to usable").
* Renderer-side probe for the performance journeys.
*
* The probe is injected from the test side through `addInitScript` while the
* journey gate (`journey-renderer-gate.cjs`) parks the window on
* `about:blank`, so it runs before any renderer script and never touches
* production code. It
* counts DOM mutations, layout shifts and long tasks until the journey's
* terminal condition and then emits one JSON blob under
* `window.__iptvnatorJourneyProbe`.
* J1 "Launch to usable" starts at document start: the probe is injected from
* the test side through `addInitScript` while the journey gate
* (`journey-renderer-gate.cjs`) parks the window on `about:blank`, so it runs
* before any renderer script and never touches production code. Journeys that
* start with a click (J2 "Open a source") pass `startClick`: the probe is
* evaluated in the loaded document, arms a capture-phase `click` listener on
* `window` (which runs before any listener of the app) and starts counting at
* the first click inside `startClick.selector`. It counts DOM mutations,
* layout shifts and long tasks until the journey's terminal condition and
* then emits one JSON blob under `options.stateKey`.
*
* IPC invocations are not counted here: the bridge object exposed by
* `contextBridge` is frozen, so the probe cannot wrap it. Instead the probe
* fires one sentinel bridge call at the terminal moment; the main-process
* fires one sentinel bridge call at the terminal moment (and, for a click
* start, one start sentinel before the app sees the click); the main-process
* capture (`journey-main-ipc-capture.ts`) counts the preload's renderer-API
* trace events received before that sentinel. Renderer-to-main IPC is
* ordered, so the count is exact regardless of clock skew.
* trace events received between the two. Renderer-to-main IPC is ordered, so
* the count is exact regardless of clock skew.
*/
export const JOURNEY_PROBE_STATE_KEY = '__iptvnatorJourneyProbe';
export const JOURNEY_PROBE_SCHEMA_VERSION = 1;
export const JOURNEY_IPC_SENTINEL_ID = '__iptvnator-journey-sentinel__';
export const JOURNEY_OPEN_SOURCE_PROBE_STATE_KEY =
'__iptvnatorJourneyOpenSourceProbe';
export const JOURNEY_OPEN_SOURCE_START_SENTINEL_ID =
'__iptvnator-journey-open-source-start__';
export const JOURNEY_OPEN_SOURCE_END_SENTINEL_ID =
'__iptvnator-journey-open-source-end__';
/** The Xtream portal card on the dashboard or its row on /workspace/sources. */
export const JOURNEY_OPEN_SOURCE_START_SELECTOR =
'[data-test-id="dashboard-recent-sources-rail-card"], app-playlist-item';
/** Bridge method used for the sentinel: a read-only lookup by id. */
export const JOURNEY_IPC_SENTINEL_METHOD = 'dbGetAppPlaylist';
export interface JourneyRendererProbeStartClick {
/** The journey starts at the first click inside this selector. */
readonly selector: string;
/** Id of the start sentinel sent before the app handles the click. */
readonly sentinelId: string;
}
export interface JourneyRendererProbeOptions {
/** Selector for the element whose visibility ends the journey. */
readonly cardSelector: string;
/** Further selectors that must each match a visible element as well. */
readonly companionSelectors?: readonly string[];
readonly journey: string;
/** Pathname fragment the terminal route must contain. */
readonly routeFragment: string;
@@ -34,13 +56,21 @@ export interface JourneyRendererProbeOptions {
readonly sentinelMethod: string;
/** Element id of the inline splash that must be gone at the end. */
readonly splashId: string;
/** Absent: the journey starts at document start (J1). */
readonly startClick?: JourneyRendererProbeStartClick;
readonly stateKey: string;
}
export interface JourneyRendererProbeCounters {
domMutations: number;
/** Shifts with `hadRecentInput === false` (the CLS definition). */
layoutShiftScore: number;
longTasks: number;
/**
* Shifts with `hadRecentInput === true`. Zero for J1, which has no
* input; a click-started journey runs inside the 500 ms input window.
*/
recentInputLayoutShiftScore: number;
}
export interface JourneyRendererProbeState {
@@ -67,14 +97,30 @@ export interface JourneyRendererProbeState {
readonly domContentLoadedEpochMs: number;
readonly loadEventEndEpochMs: number;
} | null;
/** Click-started journeys: activity before the click, for settling. */
readonly preStart: {
domMutations: number;
lastMutationEpochMs: number | null;
};
readonly schemaVersion: number;
sentinel: {
readonly epochMs: number | null;
readonly status: 'bridge-missing' | 'failed' | 'not-sent' | 'sent';
};
start: {
/** `min(event.timeStamp, listener time)` as epoch milliseconds. */
readonly epochMs: number;
readonly listenerEpochMs: number;
readonly pathname: string;
readonly sentinelStatus: 'bridge-missing' | 'failed' | 'sent';
readonly targetTag: string;
readonly targetTestId: string | null;
} | null;
terminal: {
readonly cardCount: number;
readonly cardTag: string;
readonly cardTestId: string | null;
readonly companionCounts: readonly number[];
readonly epochMs: number;
readonly pathname: string;
} | null;
@@ -92,6 +138,8 @@ export function journeyRendererProbeScript(
return;
}
const epoch = (): number => performance.timeOrigin + performance.now();
const startClick = options.startClick ?? null;
const companionSelectors = options.companionSelectors ?? [];
const bridge = target['electron'] as Record<string, unknown> | undefined;
const state: JourneyRendererProbeState = {
capabilities: {
@@ -102,7 +150,12 @@ export function journeyRendererProbeScript(
? 'documentElement'
: 'document',
},
counters: { domMutations: 0, layoutShiftScore: 0, longTasks: 0 },
counters: {
domMutations: 0,
layoutShiftScore: 0,
longTasks: 0,
recentInputLayoutShiftScore: 0,
},
final: false,
firstCardPaintEpochMs: null,
installed: {
@@ -116,17 +169,30 @@ export function journeyRendererProbeScript(
journey: options.journey,
longTaskDurationsMs: [],
navigation: null,
preStart: { domMutations: 0, lastMutationEpochMs: null },
schemaVersion: 1,
sentinel: { epochMs: null, status: 'not-sent' },
start: null,
terminal: null,
};
target[options.stateKey] = state;
if (
state.installed.scriptCount > 0 ||
state.installed.readyState !== 'loading'
startClick === null &&
(state.installed.scriptCount > 0 ||
state.installed.readyState !== 'loading')
) {
state.invalidReasons.push('probe-installed-after-document-start');
}
// Performance entries before the journey's start belong to an earlier
// journey (buffered entries included) and are dropped.
let fromEpochMs =
startClick === null
? Number.NEGATIVE_INFINITY
: Number.POSITIVE_INFINITY;
const inWindow = (entry: PerformanceEntry, untilEpochMs: number) => {
const entryEpochMs = performance.timeOrigin + entry.startTime;
return entryEpochMs >= fromEpochMs && entryEpochMs <= untilEpochMs;
};
const acceptLayoutShift = (
entries: readonly PerformanceEntry[],
@@ -138,12 +204,15 @@ export function journeyRendererProbeScript(
value?: number;
};
if (
shift.hadRecentInput === true ||
typeof shift.value !== 'number' ||
performance.timeOrigin + shift.startTime > untilEpochMs
!inWindow(entry, untilEpochMs)
) {
continue;
}
if (shift.hadRecentInput === true) {
state.counters.recentInputLayoutShiftScore += shift.value;
continue;
}
state.counters.layoutShiftScore += shift.value;
}
};
@@ -152,10 +221,7 @@ export function journeyRendererProbeScript(
untilEpochMs: number
): void => {
for (const entry of entries) {
if (
entry.duration <= 50 ||
performance.timeOrigin + entry.startTime > untilEpochMs
) {
if (entry.duration <= 50 || !inWindow(entry, untilEpochMs)) {
continue;
}
state.counters.longTasks += 1;
@@ -217,20 +283,26 @@ export function journeyRendererProbeScript(
state.firstCardPaintEpochMs = untilEpochMs;
state.final = true;
};
const sendSentinel = (): void => {
const callSentinel = (id: string): 'bridge-missing' | 'failed' | 'sent' => {
const method = bridge?.[options.sentinelMethod];
if (typeof method !== 'function') {
state.sentinel = { epochMs: null, status: 'bridge-missing' };
return;
return 'bridge-missing';
}
try {
const result: unknown = method.call(bridge, options.sentinelId);
state.sentinel = { epochMs: epoch(), status: 'sent' };
const result: unknown = method.call(bridge, id);
void Promise.resolve(result).catch(() => undefined);
return 'sent';
} catch {
state.sentinel = { epochMs: null, status: 'failed' };
return 'failed';
}
};
const sendSentinel = (): void => {
const status = callSentinel(options.sentinelId);
state.sentinel = {
epochMs: status === 'sent' ? epoch() : null,
status,
};
};
const isVisible = (element: Element | null): element is HTMLElement =>
element instanceof HTMLElement && element.getClientRects().length > 0;
const readNavigation = (): JourneyRendererProbeState['navigation'] => {
@@ -249,8 +321,17 @@ export function journeyRendererProbeScript(
};
};
const countPreStart = (count: number): void => {
if (count === 0) return;
state.preStart.domMutations += count;
state.preStart.lastMutationEpochMs = epoch();
};
const mutationObserver = new MutationObserver((records) => {
if (state.terminal !== null) return;
if (startClick !== null && state.start === null) {
countPreStart(records.length);
return;
}
state.counters.domMutations += records.length;
if (
!location.pathname.includes(options.routeFragment) ||
@@ -260,17 +341,28 @@ export function journeyRendererProbeScript(
}
const card = document.querySelector(options.cardSelector);
if (!isVisible(card)) return;
const companionCounts: number[] = [];
for (const selector of companionSelectors) {
if (!isVisible(document.querySelector(selector))) return;
companionCounts.push(document.querySelectorAll(selector).length);
}
state.terminal = {
cardCount: document.querySelectorAll(options.cardSelector).length,
cardTag: card.tagName.toLowerCase(),
cardTestId: card.getAttribute('data-test-id'),
companionCounts,
epochMs: epoch(),
pathname: location.pathname,
};
mutationObserver.disconnect();
sendSentinel();
state.navigation = readNavigation();
if (state.navigation === null) {
state.invalidReasons.push('load-event-not-finished-at-first-card');
if (startClick === null) {
state.navigation = readNavigation();
if (state.navigation === null) {
state.invalidReasons.push(
'load-event-not-finished-at-first-card'
);
}
}
const ng = target['ng'] as Record<string, unknown> | undefined;
state.capabilities.changeDetectionTicks =
@@ -291,6 +383,37 @@ export function journeyRendererProbeScript(
childList: true,
subtree: true,
});
if (startClick === null) return;
// Capture phase on window runs before every listener of the app, so the
// start sentinel precedes any bridge call the click causes and the
// mutation count starts before the app touches the DOM.
const onClick = (event: Event): void => {
if (state.start !== null) return;
const origin =
event.target instanceof Element
? event.target.closest(startClick.selector)
: null;
if (origin === null) return;
const listenerEpochMs = epoch();
const eventEpochMs = performance.timeOrigin + event.timeStamp;
countPreStart(mutationObserver.takeRecords().length);
const sentinelStatus = callSentinel(startClick.sentinelId);
state.start = {
epochMs:
Number.isFinite(eventEpochMs) && eventEpochMs <= listenerEpochMs
? eventEpochMs
: listenerEpochMs,
listenerEpochMs,
pathname: location.pathname,
sentinelStatus,
targetTag: origin.tagName.toLowerCase(),
targetTestId: origin.getAttribute('data-test-id'),
};
fromEpochMs = state.start.epochMs;
window.removeEventListener('click', onClick, true);
};
window.addEventListener('click', onClick, true);
}
export function createLaunchJourneyProbeOptions(): JourneyRendererProbeOptions {
@@ -306,6 +429,31 @@ export function createLaunchJourneyProbeOptions(): JourneyRendererProbeOptions {
};
}
/**
* Options for J2 "Open a source": the click on the Xtream portal card (or its
* source row) starts the journey; it ends when the section's category list in
* the context panel and the first page of its items are visible.
*/
export function createOpenSourceJourneyProbeOptions(): JourneyRendererProbeOptions {
return {
// Grid cards (VOD/series, the section a portal opens on), content
// cards and live channel rows; skeleton cards are not matched.
cardSelector:
'app-grid-list mat-card, .content-card, [data-test-id="channel-item"]',
companionSelectors: ['app-workspace-context-panel .category-item'],
journey: 'open-source',
routeFragment: '/workspace/xtreams/',
sentinelId: JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
sentinelMethod: JOURNEY_IPC_SENTINEL_METHOD,
splashId: 'initial-splash',
startClick: {
selector: JOURNEY_OPEN_SOURCE_START_SELECTOR,
sentinelId: JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
},
stateKey: JOURNEY_OPEN_SOURCE_PROBE_STATE_KEY,
};
}
/**
* Registers the probe on a page that is still parked on `about:blank` by the
* journey gate, so it is guaranteed to run at the start of the next document.
@@ -317,6 +465,20 @@ export async function installJourneyRendererProbe(
await page.addInitScript(journeyRendererProbeScript, options);
}
/**
* Arms a click-started probe in the current document. Playwright serializes
* the same self-contained script as for `addInitScript`.
*/
export async function armJourneyRendererProbe(
page: Page,
options: JourneyRendererProbeOptions
): Promise<void> {
if (!options.startClick) {
throw new Error('journey-renderer-probe-arm-needs-start-click');
}
await page.evaluate(journeyRendererProbeScript, options);
}
export async function waitForJourneyRendererProbe(
page: Page,
stateKey: string,
@@ -363,6 +525,11 @@ export function assertJourneyRendererProbeState(
`journey-renderer-probe-sentinel-${state.sentinel.status}`
);
}
if (state.start !== null && state.start.sentinelStatus !== 'sent') {
throw new Error(
`journey-renderer-probe-start-sentinel-${state.start.sentinelStatus}`
);
}
// A zero from an observer that never ran is not a measurement; a build
// without these entry types must fail the iteration, never lower a
// baseline.
@@ -1,18 +1,22 @@
import assert from 'node:assert/strict';
import { mkdtemp, readFile, rm } from 'node:fs/promises';
import { mkdtemp, readdir, readFile, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import test from 'node:test';
import {
formatJourneyOutputTimestamp,
JOURNEY_RUN_STARTED_AT_ENV,
JOURNEY_SUMMARY_SCHEMA_VERSION,
percentile,
recordJourneySummaryEntry,
resolveJourneyRunSummaryPath,
resolveJourneySummaryPath,
summarizeJourneyIterations,
writeJourneySummary,
type JourneyIterationRecord,
type JourneySummary,
type JourneySummaryHarness,
} from './journey-summary';
function iteration(
@@ -196,3 +200,99 @@ test('writes the summary below dist/performance/journeys/<timestamp> and never o
await rm(root, { force: true, recursive: true });
}
});
const HARNESS: JourneySummaryHarness = {
arch: 'arm64',
ci: false,
electron: '43.0.0',
electronMain: 'dist/apps/electron-backend/main.js',
measuredIterations: 1,
node: 'v22',
platform: 'darwin',
rendererIndex: 'dist/apps/web/index.html',
warmupIterations: 0,
};
test('one journey run resolves to one summary path from its start time', () => {
const root = '/repo';
assert.equal(
resolveJourneyRunSummaryPath(root, {
[JOURNEY_RUN_STARTED_AT_ENV]: '2026-09-27T09:14:01.999Z',
}),
join(
root,
'dist',
'performance',
'journeys',
'20260927T091401Z',
'summary.json'
)
);
assert.throws(
() =>
resolveJourneyRunSummaryPath(root, {
[JOURNEY_RUN_STARTED_AT_ENV]: 'not a date',
}),
/invalid-date/
);
});
test('journeys of one run are merged into one summary and never replaced', async () => {
const root = await mkdtemp(join(tmpdir(), 'iptvnator-journey-run-'));
try {
const summaryPath = resolveJourneySummaryPath(
root,
new Date('2026-09-27T09:14:01Z')
);
const launch = summarizeJourneyIterations(
[iteration(0, { a: 1 }, { w: 1 })],
{}
);
const openSource = summarizeJourneyIterations(
[iteration(0, { b: 2 }, { v: 3 })],
{ c: 'reason' }
);
await recordJourneySummaryEntry(summaryPath, HARNESS, 'launch', launch);
const merged = await recordJourneySummaryEntry(
summaryPath,
HARNESS,
'open-source',
openSource
);
const written = JSON.parse(
await readFile(summaryPath, 'utf8')
) as JourneySummary;
assert.deepEqual(Object.keys(written.journeys), [
'launch',
'open-source',
]);
assert.deepEqual(written, JSON.parse(JSON.stringify(merged)));
assert.equal(written.journeys['open-source']?.counters['b'], 2);
assert.equal(written.journeys['launch']?.counters['a'], 1);
assert.deepEqual(written.harness, HARNESS);
// No temporary file is left next to the summary.
assert.deepEqual(await readdir(join(summaryPath, '..')), [
'summary.json',
]);
await assert.rejects(
recordJourneySummaryEntry(summaryPath, HARNESS, 'launch', launch),
/merge-duplicate-launch/
);
await assert.rejects(
recordJourneySummaryEntry(
summaryPath,
{ ...HARNESS, electron: '44.0.0' },
'playback',
launch
),
/merge-harness-mismatch/
);
const unchanged = JSON.parse(
await readFile(summaryPath, 'utf8')
) as JourneySummary;
assert.deepEqual(unchanged, written);
} finally {
await rm(root, { force: true, recursive: true });
}
});
@@ -1,4 +1,4 @@
import { mkdir, writeFile } from 'node:fs/promises';
import { mkdir, readFile, rename, writeFile } from 'node:fs/promises';
import { dirname, join } from 'node:path';
/**
@@ -192,6 +192,81 @@ export function resolveJourneySummaryPath(
);
}
/**
* Environment variable that pins one `perf:journeys` invocation to one
* summary directory. `playwright.journeys.config.ts` sets it in the runner
* before the worker starts, so every journey spec of the run writes to the
* same file.
*/
export const JOURNEY_RUN_STARTED_AT_ENV = 'IPTVNATOR_JOURNEY_RUN_STARTED_AT';
export function resolveJourneyRunSummaryPath(
repositoryRoot: string,
environment: NodeJS.ProcessEnv = process.env
): string {
const raw = environment[JOURNEY_RUN_STARTED_AT_ENV];
const startedAt = raw ? new Date(raw) : new Date();
return resolveJourneySummaryPath(repositoryRoot, startedAt);
}
function isFileExistsError(error: unknown): boolean {
return (
typeof error === 'object' &&
error !== null &&
(error as { code?: unknown }).code === 'EEXIST'
);
}
/**
* Adds one journey to the run's summary. The first journey creates the
* file; later journeys of the same run merge into it when the harness is
* identical. A journey that is already present fails, so a second run in
* the same second can never overwrite a measurement.
*/
export async function recordJourneySummaryEntry(
summaryPath: string,
harness: JourneySummaryHarness,
journeyId: string,
entry: JourneySummaryEntry
): Promise<JourneySummary> {
const created: JourneySummary = {
generatedAt: new Date().toISOString(),
harness,
journeys: { [journeyId]: entry },
schemaVersion: JOURNEY_SUMMARY_SCHEMA_VERSION,
};
try {
await writeJourneySummary(summaryPath, created);
return created;
} catch (error) {
if (!isFileExistsError(error)) throw error;
}
const existing = JSON.parse(
await readFile(summaryPath, 'utf8')
) as JourneySummary;
if (existing.schemaVersion !== JOURNEY_SUMMARY_SCHEMA_VERSION) {
throw new Error('journey-summary-merge-schema-mismatch');
}
if (JSON.stringify(existing.harness) !== JSON.stringify(harness)) {
throw new Error('journey-summary-merge-harness-mismatch');
}
if (Object.prototype.hasOwnProperty.call(existing.journeys, journeyId)) {
throw new Error(`journey-summary-merge-duplicate-${journeyId}`);
}
const merged: JourneySummary = {
...existing,
generatedAt: created.generatedAt,
journeys: { ...existing.journeys, [journeyId]: entry },
};
const temporaryPath = `${summaryPath}.${process.pid}.tmp`;
await writeFile(temporaryPath, `${JSON.stringify(merged, null, 2)}\n`, {
encoding: 'utf8',
flag: 'wx',
});
await rename(temporaryPath, summaryPath);
return merged;
}
export async function writeJourneySummary(
summaryPath: string,
summary: JourneySummary
@@ -23,6 +23,7 @@ function measurement(
domMutations: 480,
layoutShiftScore: 0.123456789,
longTasks: 2,
recentInputLayoutShiftScore: 0,
},
final: true,
firstCardPaintEpochMs: 2_650,
@@ -40,17 +41,22 @@ function measurement(
domContentLoadedEpochMs: 1_300,
loadEventEndEpochMs: 1_400.26,
},
preStart: { domMutations: 0, lastMutationEpochMs: null },
schemaVersion: 1,
sentinel: { epochMs: 2_601, status: 'sent' },
start: null,
terminal: {
cardCount: 2,
cardTag: 'div',
cardTestId: 'dashboard-recent-sources-rail-card',
companionCounts: [],
epochMs: 2_600.04,
pathname: '/dist/apps/web/workspace/dashboard',
},
};
const ipc: JourneyMainIpcCaptureState = {
callsAfterSentinel: 3,
callsBeforeStart: 0,
callsBeforeSentinel: 14,
callsByMethod: { dbGetAppPlaylists: 1, getSettings: 13 },
installedEpochMs: 1_100,
@@ -58,6 +64,7 @@ function measurement(
processStartEpochMs: 900,
senderIds: [1],
sentinel: { occurrences: 1, receivedEpochMs: 2_602 },
start: null,
};
return {
electronVersion: '43.3.0',
@@ -0,0 +1,258 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import type { JourneyMainIpcCaptureState } from './journey-main-ipc-capture';
import type { JourneyMockRequest } from './journey-mock-request-ledger';
import type { JourneyRendererProbeState } from './journey-renderer-probe';
import { summarizeJourneyIterations } from './journey-summary';
import {
OPEN_SOURCE_JOURNEY_UNAVAILABLE_COUNTERS,
toOpenSourceIterationRecord,
type OpenSourceJourneyMeasurement,
} from './open-source-journey-record';
const SOURCE_PATH = '/dist/apps/web/workspace/xtreams/playlist-1/vod';
function request(sequence: number, route: string, epochMs: number) {
return { epochMs, method: 'GET', route, sequence } as JourneyMockRequest;
}
function measurement(
overrides: Partial<OpenSourceJourneyMeasurement> = {}
): OpenSourceJourneyMeasurement {
const renderer: JourneyRendererProbeState = {
capabilities: {
changeDetectionTicks: 'unavailable-ng-global-not-published',
layoutShift: true,
longTask: true,
observedTarget: 'documentElement',
},
counters: {
domMutations: 1_596,
layoutShiftScore: 0.0004,
longTasks: 1,
recentInputLayoutShiftScore: 0.22061,
},
final: true,
firstCardPaintEpochMs: 10_090,
installed: {
bridgePresent: true,
documentElementPresent: true,
epochMs: 9_000,
readyState: 'complete',
scriptCount: 12,
},
invalidReasons: [],
journey: 'open-source',
longTaskDurationsMs: [61.26],
navigation: null,
preStart: { domMutations: 4, lastMutationEpochMs: 9_100 },
schemaVersion: 1,
sentinel: { epochMs: 10_080.5, status: 'sent' },
start: {
epochMs: 10_000.2,
listenerEpochMs: 10_001,
pathname: '/dist/apps/web/workspace/dashboard',
sentinelStatus: 'sent',
targetTag: 'div',
targetTestId: 'dashboard-recent-sources-rail-card',
},
terminal: {
cardCount: 50,
cardTag: 'mat-card',
cardTestId: null,
companionCounts: [8],
epochMs: 10_078.54,
pathname: SOURCE_PATH,
},
};
const ipc: JourneyMainIpcCaptureState = {
callsAfterSentinel: 2,
callsBeforeStart: 0,
callsBeforeSentinel: 17,
callsByMethod: { dbGetAppState: 6, dbGetContent: 2, xtreamRequest: 1 },
installedEpochMs: 9_500,
malformedEvents: 0,
processStartEpochMs: 1_000,
senderIds: [1],
sentinel: { occurrences: 1, receivedEpochMs: 10_081 },
start: { occurrences: 1, receivedEpochMs: 10_002 },
};
return {
http: {
beforeClick: [
request(0, '/player_api.php?action=get_account_info', 2_000),
],
requests: [
request(1, '/player_api.php?action=get_account_info', 10_040),
request(2, '/assets/marketing/poster/a', 10_200),
],
},
ipc,
pid: 4343,
renderer,
settle: {
preStartDomMutations: 4,
preStartHttpRequests: 0,
preStartIpcCalls: 0,
quietMs: 1_000,
waitedMs: 1_048,
},
...overrides,
};
}
test('maps the click-started probe, IPC window and mock ledger to exact counters', () => {
const record = toOpenSourceIterationRecord(3, false, measurement());
assert.equal(record.index, 3);
assert.equal(record.warmup, false);
assert.equal(record.pid, 4343);
assert.deepEqual(record.counters, {
'main.mockHttpRequestsToSettled': 2,
'renderer.domMutationsToFirstPage': 1_596,
'renderer.ipcCallsToFirstPage': 17,
'renderer.layoutShiftScore': 0.221,
'renderer.longTasks': 1,
});
assert.deepEqual(record.wallClock, { clickToFirstPageMs: 78.3 });
assert.deepEqual(record.evidence['layoutShift'], {
recentInput: 0.221,
withoutRecentInput: 0,
});
assert.deepEqual(record.evidence['httpRequestsByRoute'], {
'/assets/marketing/poster/a': 1,
'/player_api.php?action=get_account_info': 1,
});
assert.deepEqual(record.evidence['httpRequestsBeforeClickByRoute'], {
'/player_api.php?action=get_account_info': 1,
});
assert.equal(record.evidence['httpRequestsToFirstPage'], 1);
assert.equal(record.evidence['ipcCallsAfterFirstPage'], 2);
assert.deepEqual(record.evidence['firstPage'], {
cardCount: 50,
cardTag: 'mat-card',
cardTestId: null,
categoryCount: 8,
pathname: SOURCE_PATH,
section: 'vod',
});
assert.deepEqual(record.evidence['start'], {
pathname: '/dist/apps/web/workspace/dashboard',
targetTag: 'div',
targetTestId: 'dashboard-recent-sources-rail-card',
});
assert.deepEqual(record.evidence['longTaskDurationsMs'], [61.3]);
assert.deepEqual(record.evidence['epochs'], {
click: 10_000.2,
clickListener: 10_001,
firstPage: 10_078.54,
firstPagePaint: 10_090,
mainIpcSentinel: 10_081,
mainIpcStart: 10_002,
});
});
test('rejects measurements that did not start at the click or did not open the source', () => {
const base = measurement();
const renderer = base.renderer;
assert.throws(
() =>
toOpenSourceIterationRecord(0, false, {
...base,
renderer: { ...renderer, start: null },
}),
/incomplete-probe/
);
assert.throws(
() =>
toOpenSourceIterationRecord(0, false, {
...base,
renderer: { ...renderer, terminal: null },
}),
/incomplete-probe/
);
assert.throws(
() =>
toOpenSourceIterationRecord(0, false, {
...base,
ipc: { ...base.ipc, start: null },
}),
/ipc-without-start/
);
const terminal = renderer.terminal as NonNullable<
JourneyRendererProbeState['terminal']
>;
const start = renderer.start as NonNullable<
JourneyRendererProbeState['start']
>;
assert.throws(
() =>
toOpenSourceIterationRecord(0, false, {
...base,
renderer: {
...renderer,
terminal: { ...terminal, pathname: '/workspace/dashboard' },
},
}),
/record-route/
);
assert.throws(
() =>
toOpenSourceIterationRecord(0, false, {
...base,
renderer: {
...renderer,
start: { ...start, pathname: SOURCE_PATH },
},
}),
/started-inside-source/
);
assert.throws(
() =>
toOpenSourceIterationRecord(0, false, {
...base,
renderer: {
...renderer,
terminal: { ...terminal, epochMs: start.epochMs },
},
}),
/clock-order/
);
assert.throws(
() =>
toOpenSourceIterationRecord(0, false, {
...base,
renderer: {
...renderer,
capabilities: {
...renderer.capabilities,
changeDetectionTicks: 'hook-present-not-counted',
},
},
}),
/cd-hook-hook-present-not-counted/
);
});
test('summarizes under the J2 counters with the unmeasurable ones listed', () => {
const entry = summarizeJourneyIterations(
[0, 1, 2].map((index) =>
toOpenSourceIterationRecord(
index,
index === 0,
measurement({ pid: 5_000 + index })
)
),
OPEN_SOURCE_JOURNEY_UNAVAILABLE_COUNTERS
);
assert.equal(entry.counters['renderer.ipcCallsToFirstPage'], 17);
assert.equal(
entry.counterStability['main.mockHttpRequestsToSettled']?.stable,
true
);
assert.equal(entry.wallClock['clickToFirstPageMs.p50'], 78.3);
assert.deepEqual(Object.keys(entry.unavailable).sort(), [
'main.sqlStatementsToFirstPage',
'renderer.cdTicksToFirstPage',
]);
});
@@ -0,0 +1,175 @@
import type { JourneyMainIpcCaptureState } from './journey-main-ipc-capture';
import {
countJourneyMockRoutes,
type JourneyMockRequest,
} from './journey-mock-request-ledger';
import type { JourneyRendererProbeState } from './journey-renderer-probe';
import type { JourneyIterationRecord } from './journey-summary';
/**
* Maps one measured "open a source" (renderer probe armed at the click, main
* IPC capture between the start and end sentinels, mock request ledger) to
* the journey summary's iteration record for J2.
*/
export const OPEN_SOURCE_JOURNEY_ID = 'open-source';
export const OPEN_SOURCE_JOURNEY_COUNTER = {
DOM_MUTATIONS: 'renderer.domMutationsToFirstPage',
IPC_CALLS: 'renderer.ipcCallsToFirstPage',
LAYOUT_SHIFT_SCORE: 'renderer.layoutShiftScore',
LONG_TASKS: 'renderer.longTasks',
MOCK_HTTP_REQUESTS: 'main.mockHttpRequestsToSettled',
} as const;
export const OPEN_SOURCE_JOURNEY_WALL_CLOCK = {
CLICK_TO_FIRST_PAGE: 'clickToFirstPageMs',
} as const;
export const OPEN_SOURCE_JOURNEY_UNAVAILABLE_COUNTERS: Readonly<
Record<string, string>
> = Object.freeze({
'main.sqlStatementsToFirstPage':
'SQL statements are only visible as worker stdout trace lines, which are forwarded asynchronously; plan item A2 adds a countable channel.',
'renderer.cdTicksToFirstPage':
'The electron-performance build optimizes scripts (ngDevMode=false), so Angular does not publish window.ng and ɵsetProfiler is unavailable.',
});
/** How long the app was left alone before the click, and what it did. */
export interface OpenSourceJourneySettle {
readonly preStartDomMutations: number;
readonly preStartHttpRequests: number;
readonly preStartIpcCalls: number;
readonly quietMs: number;
readonly waitedMs: number;
}
export interface OpenSourceJourneyMeasurement {
readonly http: {
/** Mock requests from the spawn (J1 and settling) until the click. */
readonly beforeClick: readonly JourneyMockRequest[];
/** Mock requests from the click until the mock was quiet again. */
readonly requests: readonly JourneyMockRequest[];
};
readonly ipc: JourneyMainIpcCaptureState;
readonly pid: number;
readonly renderer: JourneyRendererProbeState;
readonly settle: OpenSourceJourneySettle;
}
const ROUTE_FRAGMENT = '/workspace/xtreams/';
function roundTenth(value: number): number {
return Math.round(value * 10) / 10;
}
function roundThousandth(value: number): number {
return Math.round(value * 1_000) / 1_000;
}
export function toOpenSourceIterationRecord(
index: number,
warmup: boolean,
measurement: OpenSourceJourneyMeasurement
): JourneyIterationRecord {
const { http, ipc, renderer, settle } = measurement;
const { start, terminal } = renderer;
if (start === null || terminal === null) {
throw new Error('open-source-journey-record-incomplete-probe');
}
if (ipc.start === null) {
throw new Error('open-source-journey-record-ipc-without-start');
}
if (!terminal.pathname.includes(ROUTE_FRAGMENT)) {
throw new Error('open-source-journey-record-route');
}
if (start.pathname.includes(ROUTE_FRAGMENT)) {
throw new Error('open-source-journey-record-started-inside-source');
}
const clickToFirstPageMs = terminal.epochMs - start.epochMs;
if (clickToFirstPageMs <= 0) {
throw new Error('open-source-journey-record-clock-order');
}
if (
renderer.capabilities.changeDetectionTicks !==
'unavailable-ng-global-not-published'
) {
throw new Error(
`open-source-journey-record-cd-hook-${renderer.capabilities.changeDetectionTicks}`
);
}
// The ledger's clock is the test process's, the terminal's the
// renderer's; the split is evidence only.
const requestsToFirstPage = http.requests.filter(
(entry) => entry.epochMs <= terminal.epochMs
).length;
const section = terminal.pathname
.slice(
terminal.pathname.indexOf(ROUTE_FRAGMENT) + ROUTE_FRAGMENT.length
)
.split('/')[1];
return Object.freeze({
counters: Object.freeze({
[OPEN_SOURCE_JOURNEY_COUNTER.DOM_MUTATIONS]:
renderer.counters.domMutations,
[OPEN_SOURCE_JOURNEY_COUNTER.IPC_CALLS]: ipc.callsBeforeSentinel,
// The whole journey runs inside the 500 ms window after the
// click, so shifts flagged hadRecentInput are included.
[OPEN_SOURCE_JOURNEY_COUNTER.LAYOUT_SHIFT_SCORE]: roundThousandth(
renderer.counters.layoutShiftScore +
renderer.counters.recentInputLayoutShiftScore
),
[OPEN_SOURCE_JOURNEY_COUNTER.LONG_TASKS]:
renderer.counters.longTasks,
[OPEN_SOURCE_JOURNEY_COUNTER.MOCK_HTTP_REQUESTS]:
http.requests.length,
}),
evidence: Object.freeze({
capabilities: renderer.capabilities,
epochs: Object.freeze({
click: start.epochMs,
clickListener: start.listenerEpochMs,
firstPage: terminal.epochMs,
firstPagePaint: renderer.firstCardPaintEpochMs,
mainIpcStart: ipc.start.receivedEpochMs,
mainIpcSentinel: ipc.sentinel.receivedEpochMs,
}),
firstPage: Object.freeze({
cardCount: terminal.cardCount,
cardTag: terminal.cardTag,
cardTestId: terminal.cardTestId,
categoryCount: terminal.companionCounts[0] ?? null,
pathname: terminal.pathname,
section: section ?? null,
}),
httpRequestsBeforeClickByRoute: countJourneyMockRoutes(
http.beforeClick
),
httpRequestsByRoute: countJourneyMockRoutes(http.requests),
httpRequestsToFirstPage: requestsToFirstPage,
ipcCallsAfterFirstPage: ipc.callsAfterSentinel,
ipcCallsByMethod: ipc.callsByMethod,
layoutShift: Object.freeze({
recentInput: roundThousandth(
renderer.counters.recentInputLayoutShiftScore
),
withoutRecentInput: roundThousandth(
renderer.counters.layoutShiftScore
),
}),
longTaskDurationsMs: renderer.longTaskDurationsMs.map(roundTenth),
settle,
start: Object.freeze({
pathname: start.pathname,
targetTag: start.targetTag,
targetTestId: start.targetTestId,
}),
}),
index,
pid: measurement.pid,
wallClock: Object.freeze({
[OPEN_SOURCE_JOURNEY_WALL_CLOCK.CLICK_TO_FIRST_PAGE]:
roundTenth(clickToFirstPageMs),
}),
warmup,
});
}
+100 -13
View File
@@ -14,14 +14,14 @@ live in `tools/performance/`.
| Journey | Start | End |
| ---------------- | -------------------------------------------- | ----------------------------------------------------------------------------- |
| J1 `launch` | Electron process spawn | first playlist or portal card rendered on `/workspace`, inline splash removed |
| J2 `open-source` | click on a portal card | live category list and first channel page painted |
| J2 `open-source` | click on the Xtream portal card | category list and first page of the opened section painted |
| J3 `playback` | click on a channel | HTML5 `playing` event |
| J4 `search` | six-character query typed into global search | results list settled |
J1 is instrumented today: `renderer.initialBytes` from the built output, and
the runtime counters of the launch benchmark below. J2 to J4 follow the plan
in `.plans/` and are added one thread at a time; each thread names its journey
and counter in the PR description.
J1 is instrumented: `renderer.initialBytes` from the built output, and the
runtime counters of the launch benchmark below. J2 is instrumented by its own
spec (below). J3 and J4 follow the plan in `.plans/` and are added one thread
at a time; each thread names its journey and counter in the PR description.
## Running the journeys
@@ -39,7 +39,14 @@ first, starts the Xtream mock server on the dedicated loopback port
dist/performance/journeys/<YYYYMMDDTHHMMSSZ>/summary.json
```
The file is never overwritten; a second run in the same second fails instead.
Every journey spec (`src/journeys/*.journey.ts`) adds its own
`journeys.<id>` entry to that file. The config pins the timestamp to the start
of the invocation (`IPTVNATOR_JOURNEY_RUN_STARTED_AT`, set in the runner before
the worker forks), so all specs of one run share the directory. The first spec
creates the file; a later one merges into it only when the `harness` block is
identical, through a temporary file and a rename. A journey that is already
present fails, so no measurement is ever overwritten and a second run in the
same second fails instead.
`IPTVNATOR_JOURNEY_MEASURED_ITERATIONS` lowers the five measured iterations
for a quick local check; the warm-up iteration always runs. Numbers from a
laptop are previews: the Linux CI runner is the canonical measurer for
@@ -226,6 +233,81 @@ numbers so `tools/performance/check-journey-ratchet.mjs` can compare them with
once its counter is deterministic on the CI runner; the launch counters are
not yet (see [Ratchet](#ratchet)), so the summary is evidence only.
## J2 `open-source`: open a source to a browsable list
`open-source.journey.ts` reuses the J1 profile and process pattern: the
profile is seeded once through the "Add playlist" dialogs, and every
iteration copies it and spawns a fresh process through `runLaunchJourney`,
which measures J1 as usual (gate, probe, IPC capture) and then hands the
running app to `measureOpenSourceJourney` in
`src/journeys/open-source-journey-app.ts`. The click therefore happens after
J1's terminal condition and its counters are final, and the two journeys never
overlap. One warm-up and five measured iterations, as for J1; the J1 numbers
of these launches are not reported again.
**Start.** The click on the dashboard card of the Xtream portal
(`dashboard-recent-sources-rail-card` with the portal's name; the probe also
accepts an `app-playlist-item` row on `/workspace/sources`). Before the
click the test hovers the card and waits until the app has been quiet for
1 s: no DOM mutation, no bridge call and no request to the mock (30 s
timeout, which fails the iteration). The settle wait and what happened during
it are kept under `evidence.settle`. The renderer probe is armed in the
loaded document with `page.evaluate` (the same self-contained script as J1,
with `startClick` set). It registers a capture-phase `click` listener on
`window`, which runs before every listener of the app. On the first click
inside the start selector it stamps the start at the event's timestamp (or
the listener's time if that is earlier) and sends the start sentinel
`dbGetAppPlaylist('__iptvnator-journey-open-source-start__')`. Only then do
the counters start.
**End.** The first `MutationObserver` batch after the start in which the
path contains `/workspace/xtreams/`, an item of the first page is visible
(`app-grid-list mat-card, .content-card, [data-test-id="channel-item"]`;
skeleton cards do not match) and a category of the context panel is visible
(`app-workspace-context-panel .category-item`). The probe then sends the end
sentinel `dbGetAppPlaylist('__iptvnator-journey-open-source-end__')` and closes
the observers at the same post-paint cutoff as J1. A portal card opens the
source's default section, which is VOD (`getPlaylistLink` links to
`/workspace/xtreams/<id>/vod`): the category list is the movie category list
and the first page is the "All items" grid. The plan's "live category list"
would need a second click and is not measured; the landed section is
recorded under `evidence.firstPage.section`.
**HTTP requests to the mock.** The J2 profile is seeded with the origin of a
loopback proxy in the test process
(`src/performance/journey-mock-request-ledger.ts`) that forwards to the mock
and records every request, so requests from the main process (Xtream API,
M3U) and from the renderer (artwork served by the mock) are all counted. The
default fixture's posters point at `picsum.photos`, so they are neither
counted nor blocked: they load after the first page is painted, and on an
offline runner they fail instead. Blocking them with `page.route` would put
request interception on every renderer request, including the lazy chunks
the journey loads. The mock's own
`/__control/state` ledger is not used: it exists only in performance-control
mode, which disables `/playlist.m3u` and tracks only the 100k scenario, and a
Playwright request listener would see renderer traffic only. The ledger stores
the method, the path and, for `player_api.php`, the `action` parameter; query
strings and stream paths carry credentials and are never stored.
### Counters
| Counter | Source |
| ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `renderer.ipcCallsToFirstPage` | Bridge `start` trace events between the start and end sentinels, counted by a second `journey-main-ipc-capture.ts` instance installed with `startSentinelId`. Calls before the start marker are tallied separately (`callsBeforeStart`); a start marker that is missing, repeated or received after the end sentinel fails the iteration. |
| `renderer.domMutationsToFirstPage` | `MutationRecord`s from the click until the terminal batch. Records produced before the click (hover, settling) are taken from the observer at the start and counted under `evidence.settle` instead. |
| `renderer.layoutShiftScore` | Sum of all `layout-shift` entries from the click until the post-paint cutoff, rounded to three decimals. Unlike J1 it includes entries with `hadRecentInput === true`: the journey is a response to the click and runs inside the 500 ms input window, so the CLS filter would always read 0. The split is under `evidence.layoutShift`. |
| `renderer.longTasks` | `longtask` entries over 50 ms that started at or after the click (buffered entries from J1 are dropped) and before the cutoff. Evidence until it is shown to be stable on the CI runner, as for J1. |
| `main.mockHttpRequestsToSettled` | Requests the proxy received from the click until the mock had been quiet for 1 s after the terminal batch. Bounding by the terminal would compare the test process's clock with the renderer's, so the count up to the terminal epoch is evidence only (`evidence.httpRequestsToFirstPage`); `evidence.httpRequestsByRoute` names the requests. |
`renderer.cdTicksToFirstPage` and `main.sqlStatementsToFirstPage` are listed
under `unavailable` for the same reasons as their J1 counterparts.
### Wall-clock
| Entry | Derivation |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `clickToFirstPageMs.p50/.p90` | Terminal epoch minus start epoch, both taken in the renderer, so there is no cross-process clock. The post-paint cutoff is under `evidence.epochs`. |
## `renderer.initialBytes`
The bytes a browser fetches before Angular can bootstrap, read from the built
@@ -466,17 +548,22 @@ reports slow imports of non-Latin playlists.
1. Add `apps/electron-backend-e2e/src/journeys/<journey>.journey.ts`. Seed the
profile through the app's dialogs, spawn a fresh process per iteration
with `measureLaunchJourney` as the model, and drive the journey's start
action with Playwright.
2. Give the journey its own probe options (`cardSelector`, `routeFragment`,
terminal condition) or extend `journey-renderer-probe.ts` when the end
condition is not "an element became visible". Keep the probe
self-contained: Playwright serializes it with `toString()`.
action with Playwright. A journey that starts inside the running app
continues from J1 with `runLaunchJourney` and lets the app settle first,
as `open-source-journey-app.ts` does.
2. Give the journey its own probe options (`cardSelector`,
`companionSelectors`, `routeFragment`, `startClick` for a click start) or
extend `journey-renderer-probe.ts` when the end condition is not "elements
became visible". Use a state key and sentinel ids of its own. Keep the
probe self-contained: Playwright serializes it with `toString()`.
3. Map the measurement to a `JourneyIterationRecord` in a
`<journey>-journey-record.ts` under `src/performance/`; name counters
`renderer.*` or `main.*`, and list counters you cannot measure under
`unavailable` with the reason.
4. Add the journey under `journeys.<id>` in the summary through
`summarizeJourneyIterations`; the schema needs no change.
4. Add the journey under `journeys.<id>` in the run's summary with
`writeJourneyRunEntry` (`src/journeys/journey-run.ts`), which calls
`summarizeJourneyIterations` and merges the entry; the schema needs no
change.
5. Cover the probe with jsdom fixtures and the record and summary code with
`node:test` (`pnpm nx run electron-backend-e2e:test-performance-harness`).
6. Validate a counter before it becomes a guardrail: one PR must show that