mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-08 09:01:03 -08:00
test(performance): add the J1 launch-to-usable journey benchmark (#1698)
* test(performance): add the J1 launch-to-usable journey benchmark Implements plan items A1, A3 (J1 only) and the minimal A4 from .plans/2026-09-25-performance-journeys-ratchet.md. - journey-renderer-probe.ts: init-script probe counting DOM mutations, layout shifts and long tasks until the first source card is visible on /workspace with the splash removed; unit-tested with jsdom fixtures. - journey-main-ipc-capture.ts: counts bridge invocations from the preload's renderer-API trace channel up to a sentinel call the probe fires, so the IPC counter is exact without touching production code. - launch.journey.ts + playwright.journeys.config.ts: seeded profile (one M3U source, one Xtream portal on the loopback mock), one warm-up and five measured iterations, fresh process and data directory each, writing dist/performance/journeys/<timestamp>/summary.json with exact counters and P50/P90 wall-clock. - Nx target electron-backend-e2e:journeys and root script perf:journeys. - docs/architecture/performance-journeys.md, README, context and validation map entries. renderer.cdTicksToFirstCard and main.sqlStatementsBeforeReadyToShow are reported as unavailable with the reason instead of being faked. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(performance): gate the renderer load so the probes never race startup Review follow-up for #1698. - journey-renderer-gate.cjs: a main-process hook loaded with `-r` (the mechanism Playwright uses for its own loader) makes the first loadFile/loadURL navigate to about:blank and holds the real load until the test releases it. Playwright reports no page before a navigation commits, so this is what lets the renderer probe be registered on the page before the real document exists; the IPC capture is installed before the release too. A safety timeout releases the gate on its own and marks the iteration invalid. Unit-tested with a fake BrowserWindow. - launch-journey-app.ts: registers the probe on the parked page, releases the gate, waits for the real document to commit, fails fast when the probe is missing, and refuses an iteration whose gate timed out, saw a second load, or released before the probe was in place. - journey-renderer-probe.ts: entries delivered live after the terminal batch are buffered and filtered by the same cutoff as queued ones, and the cutoff is sampled in a timer queued from the first rAF, i.e. after the card's frame is painted, so the render task's long task and layout shift are consistently included. - launch-journey-record.ts: layoutShiftScore rounded to three decimals; a 0.0001 shift flipped in and out of the cutoff between iterations. - playwright.journeys.config.ts: reuse a mock server left on the journeys port locally (its fixtures are deterministic); CI still starts its own. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(performance): validate the journey gate after the probe completes Codex follow-up on #1698: the gate state returned by release() cannot see a reload or recovery navigation that happens before the first card. Re-read the live state once the renderer probe has finished and validate that instead, so an iteration spanning an extra navigation is rejected. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(performance): fail an iteration whose performance observers were unavailable Codex follow-up on #1698: a renderer that cannot observe layout-shift or longtask entries used to pass the probe with zero counters, which a ratchet could not tell apart from a genuine zero. The probe assertion now rejects such an iteration and names the missing observer. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: 4gray <fourgray@proton.me> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
23 files changed
+2717
-20
No files matched your search
@@ -387,6 +387,17 @@ $ pnpm run perf:initial-bytes
|
||||
The contract behind that number is in
|
||||
[docs/architecture/performance-journeys.md](docs/architecture/performance-journeys.md).
|
||||
|
||||
To benchmark the "launch to usable" journey (fresh Electron process on a
|
||||
seeded profile, exact renderer counters plus wall-clock), run:
|
||||
|
||||
```
|
||||
$ pnpm run perf:journeys
|
||||
```
|
||||
|
||||
The journeys, their counters and the summary written under
|
||||
`dist/performance/journeys/` are described in
|
||||
[docs/architecture/performance-journeys.md](docs/architecture/performance-journeys.md).
|
||||
|
||||
## Disclaimer
|
||||
|
||||
**IPTVnator doesn't provide any playlists or other digital content.**
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
import { workspaceRoot } from '@nx/devkit';
|
||||
import { defineConfig } from '@playwright/test';
|
||||
|
||||
/**
|
||||
* Performance journeys (docs/architecture/performance-journeys.md). One
|
||||
* worker, no retries: every journey spawns its own Electron processes and
|
||||
* writes one summary per run. The Xtream mock serves both the M3U playlist
|
||||
* and the portal on a dedicated loopback port so a normal E2E server on
|
||||
* 3211 cannot be reused by accident. Locally a server left behind by an
|
||||
* earlier run on that port is reused (its fixtures are deterministic); CI
|
||||
* always starts its own.
|
||||
*/
|
||||
const xtreamMockPort =
|
||||
process.env['IPTVNATOR_JOURNEY_XTREAM_MOCK_PORT'] ?? '3231';
|
||||
|
||||
export default defineConfig({
|
||||
fullyParallel: false,
|
||||
reporter: [['list']],
|
||||
retries: 0,
|
||||
testDir: './src/journeys',
|
||||
testMatch: '**/*.journey.ts',
|
||||
timeout: 30 * 60 * 1_000,
|
||||
use: {
|
||||
testIdAttribute: 'data-test-id',
|
||||
},
|
||||
webServer: {
|
||||
command: 'pnpm nx run xtream-mock-server:serve',
|
||||
cwd: workspaceRoot,
|
||||
env: {
|
||||
HOST: '127.0.0.1',
|
||||
PORT: xtreamMockPort,
|
||||
},
|
||||
reuseExistingServer: !process.env['CI'],
|
||||
url: `http://127.0.0.1:${xtreamMockPort}/health`,
|
||||
},
|
||||
workers: 1,
|
||||
});
|
||||
@@ -49,6 +49,16 @@
|
||||
"command": "pnpm exec playwright test --config=playwright.xtream-performance.config.ts src/xtream.performance.ts"
|
||||
}
|
||||
},
|
||||
"journeys": {
|
||||
"dependsOn": ["electron-backend:build-performance"],
|
||||
"executor": "nx:run-commands",
|
||||
"cache": false,
|
||||
"parallelism": false,
|
||||
"options": {
|
||||
"cwd": "apps/electron-backend-e2e",
|
||||
"command": "pnpm exec playwright test --config=playwright.journeys.config.ts"
|
||||
}
|
||||
},
|
||||
"packaged-frame-copy-smoke": {
|
||||
"dependsOn": ["test-packaged-frame-copy-fixtures"],
|
||||
"executor": "nx:run-commands",
|
||||
|
||||
@@ -30,6 +30,7 @@ import {
|
||||
import {
|
||||
captureElectronProcess,
|
||||
closeElectronApplicationAndConfirmExit,
|
||||
type ElectronExitConfirmationOptions,
|
||||
prepareElectronApplication,
|
||||
} from './electron-process-lifecycle';
|
||||
|
||||
@@ -200,7 +201,7 @@ export { expect };
|
||||
* helper that spawns the app itself has to use the same list. `appArgs` land
|
||||
* after the entry point, which is where the OS puts an opened file's path.
|
||||
*/
|
||||
function buildElectronLaunchArgs(
|
||||
export function buildElectronLaunchArgs(
|
||||
extraArgs: readonly string[] = [],
|
||||
appArgs: readonly string[] = [],
|
||||
entryPoint = electronMainPath
|
||||
@@ -577,10 +578,18 @@ export async function closeElectronApp(
|
||||
export async function closeElectronAppAndConfirmExit(
|
||||
app: LaunchedElectronApp
|
||||
): Promise<void> {
|
||||
await closeElectronApplicationAndConfirmExit(app.electronApp, {
|
||||
await closeElectronApplicationAndConfirmExit(
|
||||
app.electronApp,
|
||||
electronAppExitConfirmationOptions()
|
||||
);
|
||||
}
|
||||
|
||||
/** The close/exit timeouts the shared fixture applies to every launch. */
|
||||
export function electronAppExitConfirmationOptions(): ElectronExitConfirmationOptions {
|
||||
return {
|
||||
closeTimeoutMs: electronAppCloseTimeoutMs,
|
||||
exitTimeoutMs: electronAppKillWaitMs,
|
||||
});
|
||||
};
|
||||
}
|
||||
|
||||
function assertPackagedRendererBuildIsElectronSafe(): void {
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
import type { ElectronApplication } from '@playwright/test';
|
||||
|
||||
/**
|
||||
* Test-side client for the main-process gate in
|
||||
* `../performance/journey-renderer-gate.cjs`.
|
||||
*/
|
||||
export const JOURNEY_RENDERER_GATE_KEY = '__iptvnatorJourneyGate';
|
||||
|
||||
export interface JourneyRendererGateState {
|
||||
readonly blankLoadedEpochMs: number | null;
|
||||
readonly errors: readonly string[];
|
||||
readonly gatedEpochMs: number | null;
|
||||
readonly gatedMethod: string | null;
|
||||
readonly passThroughLoads: number;
|
||||
readonly releasedEpochMs: number | null;
|
||||
readonly timedOut: boolean;
|
||||
}
|
||||
|
||||
export async function readJourneyRendererGate(
|
||||
electronApp: ElectronApplication,
|
||||
gateKey: string,
|
||||
action: 'read' | 'release'
|
||||
): Promise<JourneyRendererGateState> {
|
||||
const state = await electronApp.evaluate(
|
||||
(_electron, input) => {
|
||||
const gate = (globalThis as unknown as Record<string, unknown>)[
|
||||
input.gateKey
|
||||
] as { release(): unknown; state: unknown } | undefined;
|
||||
if (!gate) {
|
||||
return null;
|
||||
}
|
||||
const result =
|
||||
input.action === 'release' ? gate.release() : gate.state;
|
||||
return JSON.parse(JSON.stringify(result)) as unknown;
|
||||
},
|
||||
{ action, gateKey }
|
||||
);
|
||||
if (state === null) {
|
||||
throw new Error('journey-renderer-gate-not-installed');
|
||||
}
|
||||
return state as JourneyRendererGateState;
|
||||
}
|
||||
|
||||
/**
|
||||
* The gate proves the ordering the counters rely on: the real document was
|
||||
* loaded once, only after the test released it, and the renderer probe ran
|
||||
* after the release (so it was registered before that document existed).
|
||||
*/
|
||||
export function assertJourneyRendererGate(
|
||||
gate: JourneyRendererGateState,
|
||||
rendererProbeInstalledEpochMs: number
|
||||
): JourneyRendererGateState {
|
||||
if (gate.timedOut) {
|
||||
throw new Error('journey-renderer-gate-timed-out');
|
||||
}
|
||||
if (gate.errors.length > 0) {
|
||||
throw new Error(
|
||||
`journey-renderer-gate-errors: ${gate.errors.join(', ')}`
|
||||
);
|
||||
}
|
||||
if (
|
||||
gate.gatedEpochMs === null ||
|
||||
gate.blankLoadedEpochMs === null ||
|
||||
gate.releasedEpochMs === null
|
||||
) {
|
||||
throw new Error('journey-renderer-gate-incomplete');
|
||||
}
|
||||
if (gate.passThroughLoads !== 0) {
|
||||
throw new Error(
|
||||
`journey-renderer-gate-extra-loads-${gate.passThroughLoads}`
|
||||
);
|
||||
}
|
||||
if (rendererProbeInstalledEpochMs < gate.releasedEpochMs) {
|
||||
throw new Error('journey-renderer-gate-probe-before-release');
|
||||
}
|
||||
return gate;
|
||||
}
|
||||
@@ -0,0 +1,229 @@
|
||||
import { cp, mkdtemp, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join, resolve } from 'node:path';
|
||||
|
||||
import { _electron as electron, type Page } from '@playwright/test';
|
||||
|
||||
import { captureElectronProcess } from '../electron-process-lifecycle';
|
||||
import {
|
||||
addXtreamPortal,
|
||||
buildElectronLaunchArgs,
|
||||
buildElectronLaunchEnvironment,
|
||||
closeElectronAppAndConfirmExit,
|
||||
electronAppExitConfirmationOptions,
|
||||
importM3uPlaylistFromUrl,
|
||||
launchElectronApp,
|
||||
waitForM3uCatalog,
|
||||
waitForXtreamCatalog,
|
||||
type LaunchElectronAppOptions,
|
||||
} from '../electron-test-fixtures';
|
||||
import { closeElectronApplicationAndConfirmExit } from '../electron-process-lifecycle';
|
||||
import {
|
||||
installJourneyMainIpcCapture,
|
||||
JOURNEY_MAIN_IPC_STATE_KEY,
|
||||
JOURNEY_RENDERER_API_TRACE_CHANNEL,
|
||||
readJourneyMainIpcCapture,
|
||||
} from '../performance/journey-main-ipc-capture';
|
||||
import {
|
||||
createLaunchJourneyProbeOptions,
|
||||
installJourneyRendererProbe,
|
||||
waitForJourneyRendererProbe,
|
||||
} from '../performance/journey-renderer-probe';
|
||||
import {
|
||||
assertJourneyRendererGate,
|
||||
JOURNEY_RENDERER_GATE_KEY,
|
||||
readJourneyRendererGate,
|
||||
} from './journey-renderer-gate-client';
|
||||
import type { LaunchJourneyMeasurement } from '../performance/launch-journey-record';
|
||||
|
||||
/**
|
||||
* Process lifecycle for J1 "Launch to usable": one seeded profile template,
|
||||
* then a fresh Electron process on a fresh copy of that template per
|
||||
* iteration. Mirrors `xtream-benchmark-app-startup.ts` without the import
|
||||
* scaffolding that journey does not need.
|
||||
*/
|
||||
export const LAUNCH_JOURNEY_XTREAM_MOCK_PORT =
|
||||
process.env['IPTVNATOR_JOURNEY_XTREAM_MOCK_PORT'] ?? '3231';
|
||||
export const LAUNCH_JOURNEY_MOCK_ORIGIN = `http://127.0.0.1:${LAUNCH_JOURNEY_XTREAM_MOCK_PORT}`;
|
||||
/** Main-process hook loaded with `-r`; see journey-renderer-gate.cjs. */
|
||||
export const JOURNEY_RENDERER_GATE_PATH = resolve(
|
||||
__dirname,
|
||||
'../performance/journey-renderer-gate.cjs'
|
||||
);
|
||||
|
||||
function launchOptions(
|
||||
env: Record<string, string> = {}
|
||||
): LaunchElectronAppOptions {
|
||||
return {
|
||||
env: { IPTVNATOR_TRACE_RENDERER_CONSOLE: '0', ...env },
|
||||
environmentInheritance: 'runtime-only',
|
||||
omitEnvKeys: [
|
||||
'IPTVNATOR_XTREAM_MOCK_CONTROL',
|
||||
'IPTVNATOR_XTREAM_MOCK_CONTROL_TOKEN',
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
function removeDirectory(directory: string): Promise<void> {
|
||||
return rm(directory, {
|
||||
force: true,
|
||||
maxRetries: 20,
|
||||
recursive: true,
|
||||
retryDelay: 250,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Seeds one M3U source and one Xtream portal through the app's own dialogs
|
||||
* and returns the data directory to copy for every measured launch.
|
||||
*/
|
||||
export async function seedLaunchJourneyProfile(
|
||||
mockOrigin: string
|
||||
): Promise<string> {
|
||||
const templateDirectory = await mkdtemp(
|
||||
join(tmpdir(), 'iptvnator-journey-launch-seed-')
|
||||
);
|
||||
try {
|
||||
const app = await launchElectronApp(templateDirectory, launchOptions());
|
||||
try {
|
||||
await importM3uPlaylistFromUrl(
|
||||
app.mainWindow,
|
||||
`${mockOrigin}/playlist.m3u`
|
||||
);
|
||||
await waitForM3uCatalog(app.mainWindow);
|
||||
await addXtreamPortal(app.mainWindow, { serverUrl: mockOrigin });
|
||||
await waitForXtreamCatalog(app.mainWindow);
|
||||
} finally {
|
||||
await closeElectronAppAndConfirmExit(app);
|
||||
}
|
||||
return templateDirectory;
|
||||
} catch (failure) {
|
||||
await removeDirectory(templateDirectory);
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
|
||||
export function removeLaunchJourneyProfile(directory: string): Promise<void> {
|
||||
return removeDirectory(directory);
|
||||
}
|
||||
|
||||
/**
|
||||
* Spawns a fresh Electron process on a copy of the seeded profile. The gate
|
||||
* hook parks the first renderer load on `about:blank`, which gives Playwright
|
||||
* a page to attach the renderer probe to; the main-process IPC capture is
|
||||
* installed next, and only then is the real load released. Both captures are
|
||||
* therefore in place before the renderer runs any script, and the probe,
|
||||
* capture and gate records still prove it.
|
||||
*/
|
||||
export async function measureLaunchJourney(
|
||||
templateDirectory: string,
|
||||
timeoutMs: number
|
||||
): Promise<LaunchJourneyMeasurement> {
|
||||
const dataDirectory = await mkdtemp(
|
||||
join(tmpdir(), 'iptvnator-journey-launch-')
|
||||
);
|
||||
try {
|
||||
await cp(templateDirectory, dataDirectory, { recursive: true });
|
||||
const env = buildElectronLaunchEnvironment(
|
||||
dataDirectory,
|
||||
launchOptions({ IPTVNATOR_TRACE_IPC: '1' })
|
||||
);
|
||||
const args = buildElectronLaunchArgs([
|
||||
'-r',
|
||||
JOURNEY_RENDERER_GATE_PATH,
|
||||
]);
|
||||
const spawnEpochMs = Date.now();
|
||||
const electronApp = await electron.launch({ args, env });
|
||||
captureElectronProcess(electronApp);
|
||||
try {
|
||||
const probeOptions = createLaunchJourneyProbeOptions();
|
||||
// The gate parks the window on about:blank, so this resolves
|
||||
// before the real document exists.
|
||||
const mainWindow = await electronApp.firstWindow();
|
||||
if (mainWindow.url() !== 'about:blank') {
|
||||
throw new Error(
|
||||
`journey-renderer-gate-missing: first document is ${mainWindow.url()}`
|
||||
);
|
||||
}
|
||||
await installJourneyRendererProbe(mainWindow, probeOptions);
|
||||
await installJourneyMainIpcCapture(electronApp, {
|
||||
channel: JOURNEY_RENDERER_API_TRACE_CHANNEL,
|
||||
sentinelId: probeOptions.sentinelId,
|
||||
sentinelMethod: probeOptions.sentinelMethod,
|
||||
stateKey: JOURNEY_MAIN_IPC_STATE_KEY,
|
||||
});
|
||||
await readJourneyRendererGate(
|
||||
electronApp,
|
||||
JOURNEY_RENDERER_GATE_KEY,
|
||||
'release'
|
||||
);
|
||||
// The page object is still on about:blank; wait for the real
|
||||
// document to commit before touching its execution context.
|
||||
await mainWindow.waitForURL((url) => url.href !== 'about:blank', {
|
||||
timeout: timeoutMs,
|
||||
waitUntil: 'commit',
|
||||
});
|
||||
await assertJourneyRendererProbeInstalled(
|
||||
mainWindow,
|
||||
probeOptions.stateKey
|
||||
);
|
||||
const renderer = await waitForJourneyRendererProbe(
|
||||
mainWindow,
|
||||
probeOptions.stateKey,
|
||||
timeoutMs
|
||||
);
|
||||
// Re-read after the probe finished: a reload or recovery
|
||||
// navigation during startup shows up as a pass-through load
|
||||
// only in the live state, and such an iteration is invalid.
|
||||
const gate = assertJourneyRendererGate(
|
||||
await readJourneyRendererGate(
|
||||
electronApp,
|
||||
JOURNEY_RENDERER_GATE_KEY,
|
||||
'read'
|
||||
),
|
||||
renderer.installed.epochMs
|
||||
);
|
||||
const ipc = await readJourneyMainIpcCapture(
|
||||
electronApp,
|
||||
JOURNEY_MAIN_IPC_STATE_KEY,
|
||||
10_000
|
||||
);
|
||||
if (ipc.installedEpochMs > renderer.installed.epochMs) {
|
||||
throw new Error('journey-main-ipc-capture-installed-late');
|
||||
}
|
||||
const electronVersion = await electronApp.evaluate(
|
||||
() => process.versions.electron
|
||||
);
|
||||
return {
|
||||
electronVersion,
|
||||
gate,
|
||||
ipc,
|
||||
pid: electronApp.process().pid ?? -1,
|
||||
renderer,
|
||||
spawnEpochMs,
|
||||
};
|
||||
} finally {
|
||||
await closeElectronApplicationAndConfirmExit(
|
||||
electronApp,
|
||||
electronAppExitConfirmationOptions()
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
await removeDirectory(dataDirectory);
|
||||
}
|
||||
}
|
||||
|
||||
async function assertJourneyRendererProbeInstalled(
|
||||
mainWindow: Page,
|
||||
stateKey: string
|
||||
): Promise<void> {
|
||||
const installed = await mainWindow.evaluate(
|
||||
(key) =>
|
||||
(globalThis as unknown as Record<string, unknown>)[key] !==
|
||||
undefined,
|
||||
stateKey
|
||||
);
|
||||
if (!installed) {
|
||||
throw new Error('journey-renderer-probe-not-installed');
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
import { relative } from 'node:path';
|
||||
|
||||
import { test } from '@playwright/test';
|
||||
|
||||
import {
|
||||
electronMainPath,
|
||||
packagedRendererIndexPath,
|
||||
workspaceRoot,
|
||||
} from '../electron-test-fixtures';
|
||||
import {
|
||||
JOURNEY_SUMMARY_SCHEMA_VERSION,
|
||||
resolveJourneySummaryPath,
|
||||
summarizeJourneyIterations,
|
||||
writeJourneySummary,
|
||||
type JourneyIterationRecord,
|
||||
type JourneySummary,
|
||||
} from '../performance/journey-summary';
|
||||
import {
|
||||
LAUNCH_JOURNEY_ID,
|
||||
LAUNCH_JOURNEY_UNAVAILABLE_COUNTERS,
|
||||
toLaunchIterationRecord,
|
||||
} from '../performance/launch-journey-record';
|
||||
import {
|
||||
LAUNCH_JOURNEY_MOCK_ORIGIN,
|
||||
measureLaunchJourney,
|
||||
removeLaunchJourneyProfile,
|
||||
seedLaunchJourneyProfile,
|
||||
} from './launch-journey-app';
|
||||
|
||||
/**
|
||||
* J1 "Launch to usable": Electron process spawn until the first playlist or
|
||||
* portal card is visible on /workspace with the inline splash removed.
|
||||
* Contract: docs/architecture/performance-journeys.md.
|
||||
*/
|
||||
const WARMUP_ITERATIONS = 1;
|
||||
const MEASURED_ITERATIONS = readPositiveInteger(
|
||||
'IPTVNATOR_JOURNEY_MEASURED_ITERATIONS',
|
||||
5
|
||||
);
|
||||
const ITERATION_TIMEOUT_MS = 120_000;
|
||||
|
||||
function readPositiveInteger(name: string, fallback: number): number {
|
||||
const raw = process.env[name];
|
||||
if (raw === undefined || raw === '') {
|
||||
return fallback;
|
||||
}
|
||||
const value = Number(raw);
|
||||
if (!Number.isSafeInteger(value) || value < 1) {
|
||||
throw new Error(`${name} must be a positive integer`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
test.describe.configure({ mode: 'serial' });
|
||||
|
||||
test('J1 launch to usable', async () => {
|
||||
const templateDirectory = await seedLaunchJourneyProfile(
|
||||
LAUNCH_JOURNEY_MOCK_ORIGIN
|
||||
);
|
||||
const iterations: JourneyIterationRecord[] = [];
|
||||
let electronVersion = 'unknown';
|
||||
try {
|
||||
const total = WARMUP_ITERATIONS + MEASURED_ITERATIONS;
|
||||
for (let index = 0; index < total; index += 1) {
|
||||
const warmup = index < WARMUP_ITERATIONS;
|
||||
const measurement = await measureLaunchJourney(
|
||||
templateDirectory,
|
||||
ITERATION_TIMEOUT_MS
|
||||
);
|
||||
electronVersion = measurement.electronVersion;
|
||||
const record = toLaunchIterationRecord(index, warmup, measurement);
|
||||
iterations.push(record);
|
||||
console.log(
|
||||
`[journey:launch] iteration ${index}${warmup ? ' (warm-up)' : ''} pid=${record.pid} ${JSON.stringify(
|
||||
{ ...record.counters, ...record.wallClock }
|
||||
)}`
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
await removeLaunchJourneyProfile(templateDirectory);
|
||||
}
|
||||
|
||||
const entry = summarizeJourneyIterations(
|
||||
iterations,
|
||||
LAUNCH_JOURNEY_UNAVAILABLE_COUNTERS
|
||||
);
|
||||
const summary: JourneySummary = {
|
||||
generatedAt: new Date().toISOString(),
|
||||
harness: {
|
||||
arch: process.arch,
|
||||
ci: Boolean(process.env['CI']),
|
||||
electron: electronVersion,
|
||||
electronMain: relative(workspaceRoot, electronMainPath),
|
||||
measuredIterations: MEASURED_ITERATIONS,
|
||||
node: process.version,
|
||||
platform: process.platform,
|
||||
rendererIndex: relative(workspaceRoot, packagedRendererIndexPath),
|
||||
warmupIterations: WARMUP_ITERATIONS,
|
||||
},
|
||||
journeys: { [LAUNCH_JOURNEY_ID]: entry },
|
||||
schemaVersion: JOURNEY_SUMMARY_SCHEMA_VERSION,
|
||||
};
|
||||
const summaryPath = resolveJourneySummaryPath(workspaceRoot);
|
||||
await writeJourneySummary(summaryPath, summary);
|
||||
await test.info().attach('journey-summary', {
|
||||
contentType: 'application/json',
|
||||
path: summaryPath,
|
||||
});
|
||||
console.log(
|
||||
`[journey:launch] summary ${relative(workspaceRoot, summaryPath)}\n${JSON.stringify(
|
||||
{
|
||||
counters: entry.counters,
|
||||
counterStability: entry.counterStability,
|
||||
unavailable: entry.unavailable,
|
||||
wallClock: entry.wallClock,
|
||||
},
|
||||
null,
|
||||
2
|
||||
)}`
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,104 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { resolve } from 'node:path';
|
||||
import test from 'node:test';
|
||||
|
||||
import {
|
||||
assertJourneyMainIpcCapture,
|
||||
JOURNEY_RENDERER_API_TRACE_CHANNEL,
|
||||
type JourneyMainIpcCaptureState,
|
||||
} from './journey-main-ipc-capture';
|
||||
import { JOURNEY_IPC_SENTINEL_METHOD } from './journey-renderer-probe';
|
||||
|
||||
const electronBackendSource = resolve(
|
||||
__dirname,
|
||||
'../../../electron-backend/src/app'
|
||||
);
|
||||
|
||||
function validCapture(
|
||||
overrides: Partial<JourneyMainIpcCaptureState> = {}
|
||||
): JourneyMainIpcCaptureState {
|
||||
return {
|
||||
callsAfterSentinel: 2,
|
||||
callsBeforeSentinel: 7,
|
||||
callsByMethod: { dbGetAppPlaylists: 1, getSettings: 6 },
|
||||
installedEpochMs: 1,
|
||||
malformedEvents: 0,
|
||||
processStartEpochMs: 0,
|
||||
senderIds: [1],
|
||||
sentinel: { occurrences: 1, receivedEpochMs: 2 },
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
test('the trace channel literal matches the main-process constant', () => {
|
||||
const source = readFileSync(
|
||||
resolve(electronBackendSource, 'services/debug-trace.ts'),
|
||||
'utf8'
|
||||
);
|
||||
assert.match(
|
||||
source,
|
||||
new RegExp(
|
||||
`DEBUG_TRACE_EVENT_CHANNEL = '${JOURNEY_RENDERER_API_TRACE_CHANNEL}'`
|
||||
)
|
||||
);
|
||||
});
|
||||
|
||||
test('the preload traces every bridge invocation on that channel when IPC tracing is on', () => {
|
||||
const preload = readFileSync(
|
||||
resolve(electronBackendSource, 'api/main.preload.ts'),
|
||||
'utf8'
|
||||
);
|
||||
assert.match(
|
||||
preload,
|
||||
/ipcRenderer\.send\(DEBUG_TRACE_EVENT_CHANNEL, payload\)/
|
||||
);
|
||||
assert.match(preload, /name\.startsWith\('on'\)/);
|
||||
assert.match(preload, /name\.startsWith\('remove'\)/);
|
||||
assert.match(
|
||||
preload,
|
||||
new RegExp(`${JOURNEY_IPC_SENTINEL_METHOD}: \\(playlistId: string`)
|
||||
);
|
||||
const debugTrace = readFileSync(
|
||||
resolve(electronBackendSource, 'services/debug-trace.ts'),
|
||||
'utf8'
|
||||
);
|
||||
assert.match(debugTrace, /readFlag\('IPTVNATOR_TRACE_IPC'\)/);
|
||||
});
|
||||
|
||||
test('accepts a capture with exactly one sentinel from one renderer', () => {
|
||||
assert.equal(
|
||||
assertJourneyMainIpcCapture(validCapture()).callsBeforeSentinel,
|
||||
7
|
||||
);
|
||||
});
|
||||
|
||||
test('rejects captures that cannot bound the counter exactly', () => {
|
||||
assert.throws(() => assertJourneyMainIpcCapture(null), /missing/);
|
||||
assert.throws(
|
||||
() =>
|
||||
assertJourneyMainIpcCapture(
|
||||
validCapture({
|
||||
sentinel: { occurrences: 0, receivedEpochMs: null },
|
||||
})
|
||||
),
|
||||
/sentinel-count-0/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
assertJourneyMainIpcCapture(
|
||||
validCapture({
|
||||
sentinel: { occurrences: 2, receivedEpochMs: 2 },
|
||||
})
|
||||
),
|
||||
/sentinel-count-2/
|
||||
);
|
||||
assert.throws(
|
||||
() => assertJourneyMainIpcCapture(validCapture({ senderIds: [1, 2] })),
|
||||
/senders-2/
|
||||
);
|
||||
assert.throws(
|
||||
() => assertJourneyMainIpcCapture(validCapture({ malformedEvents: 1 })),
|
||||
/malformed/
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,155 @@
|
||||
import type { ElectronApplication } from '@playwright/test';
|
||||
|
||||
/**
|
||||
* Main-process side of the journey IPC counter.
|
||||
*
|
||||
* With `IPTVNATOR_TRACE_IPC=1` the preload wraps every bridge method (except
|
||||
* `on*` / `remove*` listener registrations) and sends one trace event per
|
||||
* invocation on the renderer-API trace channel before forwarding the call
|
||||
* (see `apps/electron-backend/src/app/api/main.preload.ts`). This capture
|
||||
* subscribes to that channel from the test side and counts `start` events
|
||||
* until the renderer probe's sentinel call arrives. Renderer-to-main IPC is
|
||||
* delivered in order, so every call started before the sentinel is counted
|
||||
* and nothing after it is.
|
||||
*/
|
||||
|
||||
/** Literal of `DEBUG_TRACE_EVENT_CHANNEL` in `services/debug-trace.ts`. */
|
||||
export const JOURNEY_RENDERER_API_TRACE_CHANNEL = 'IPTVNATOR_DEBUG_TRACE_EVENT';
|
||||
export const JOURNEY_MAIN_IPC_STATE_KEY = '__iptvnatorJourneyMainIpcCapture';
|
||||
|
||||
export interface JourneyMainIpcCaptureOptions {
|
||||
readonly channel: string;
|
||||
readonly sentinelId: string;
|
||||
readonly sentinelMethod: string;
|
||||
readonly stateKey: string;
|
||||
}
|
||||
|
||||
export interface JourneyMainIpcCaptureState {
|
||||
readonly callsAfterSentinel: number;
|
||||
readonly callsBeforeSentinel: number;
|
||||
readonly callsByMethod: Record<string, number>;
|
||||
readonly installedEpochMs: number;
|
||||
readonly malformedEvents: number;
|
||||
readonly processStartEpochMs: number;
|
||||
readonly senderIds: number[];
|
||||
readonly sentinel: {
|
||||
readonly occurrences: number;
|
||||
readonly receivedEpochMs: number | null;
|
||||
};
|
||||
}
|
||||
|
||||
export async function installJourneyMainIpcCapture(
|
||||
electronApp: ElectronApplication,
|
||||
options: JourneyMainIpcCaptureOptions
|
||||
): Promise<void> {
|
||||
await electronApp.evaluate(({ ipcMain }, input) => {
|
||||
const target = globalThis as unknown as Record<string, unknown>;
|
||||
if (target[input.stateKey] !== undefined) {
|
||||
throw new Error('journey-main-ipc-capture-already-installed');
|
||||
}
|
||||
const state = {
|
||||
callsAfterSentinel: 0,
|
||||
callsBeforeSentinel: 0,
|
||||
callsByMethod: {} as Record<string, number>,
|
||||
installedEpochMs: Date.now(),
|
||||
malformedEvents: 0,
|
||||
processStartEpochMs: Date.now() - process.uptime() * 1000,
|
||||
senderIds: [] as number[],
|
||||
sentinel: {
|
||||
occurrences: 0,
|
||||
receivedEpochMs: null as number | null,
|
||||
},
|
||||
};
|
||||
target[input.stateKey] = state;
|
||||
ipcMain.on(input.channel, (event, payload: unknown) => {
|
||||
const record =
|
||||
typeof payload === 'object' && payload !== null
|
||||
? (payload as Record<string, unknown>)
|
||||
: null;
|
||||
if (!record || typeof record['method'] !== 'string') {
|
||||
state.malformedEvents += 1;
|
||||
return;
|
||||
}
|
||||
if (record['phase'] !== 'start') {
|
||||
return;
|
||||
}
|
||||
const senderId = event.sender.id;
|
||||
if (!state.senderIds.includes(senderId)) {
|
||||
state.senderIds.push(senderId);
|
||||
}
|
||||
const method = record['method'];
|
||||
let isSentinel = false;
|
||||
if (method === input.sentinelMethod) {
|
||||
try {
|
||||
isSentinel = JSON.stringify(
|
||||
record['args'] ?? null
|
||||
).includes(input.sentinelId);
|
||||
} catch {
|
||||
isSentinel = false;
|
||||
}
|
||||
}
|
||||
if (isSentinel) {
|
||||
state.sentinel.occurrences += 1;
|
||||
state.sentinel.receivedEpochMs ??= Date.now();
|
||||
return;
|
||||
}
|
||||
if (state.sentinel.receivedEpochMs !== null) {
|
||||
state.callsAfterSentinel += 1;
|
||||
return;
|
||||
}
|
||||
state.callsBeforeSentinel += 1;
|
||||
state.callsByMethod[method] =
|
||||
(state.callsByMethod[method] ?? 0) + 1;
|
||||
});
|
||||
}, options);
|
||||
}
|
||||
|
||||
export async function readJourneyMainIpcCapture(
|
||||
electronApp: ElectronApplication,
|
||||
stateKey: string,
|
||||
timeoutMs: number
|
||||
): Promise<JourneyMainIpcCaptureState> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
for (;;) {
|
||||
const state = await electronApp.evaluate(
|
||||
(_electron, key) =>
|
||||
JSON.parse(
|
||||
JSON.stringify(
|
||||
(globalThis as unknown as Record<string, unknown>)[key]
|
||||
)
|
||||
) as unknown,
|
||||
stateKey
|
||||
);
|
||||
const capture = state as JourneyMainIpcCaptureState | null;
|
||||
if (capture?.sentinel.receivedEpochMs !== null) {
|
||||
return assertJourneyMainIpcCapture(capture);
|
||||
}
|
||||
if (Date.now() >= deadline) {
|
||||
throw new Error('journey-main-ipc-capture-sentinel-timeout');
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 25));
|
||||
}
|
||||
}
|
||||
|
||||
export function assertJourneyMainIpcCapture(
|
||||
value: unknown
|
||||
): JourneyMainIpcCaptureState {
|
||||
const state = value as JourneyMainIpcCaptureState | null | undefined;
|
||||
if (!state || typeof state.callsBeforeSentinel !== 'number') {
|
||||
throw new Error('journey-main-ipc-capture-missing');
|
||||
}
|
||||
if (state.sentinel.occurrences !== 1) {
|
||||
throw new Error(
|
||||
`journey-main-ipc-capture-sentinel-count-${state.sentinel.occurrences}`
|
||||
);
|
||||
}
|
||||
if (state.senderIds.length !== 1) {
|
||||
throw new Error(
|
||||
`journey-main-ipc-capture-senders-${state.senderIds.length}`
|
||||
);
|
||||
}
|
||||
if (state.malformedEvents > 0) {
|
||||
throw new Error('journey-main-ipc-capture-malformed-events');
|
||||
}
|
||||
return state;
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import {
|
||||
assertJourneyRendererGate,
|
||||
JOURNEY_RENDERER_GATE_KEY,
|
||||
type JourneyRendererGateState,
|
||||
} from '../journeys/journey-renderer-gate-client';
|
||||
|
||||
function gate(
|
||||
overrides: Partial<JourneyRendererGateState> = {}
|
||||
): JourneyRendererGateState {
|
||||
return {
|
||||
blankLoadedEpochMs: 1_050,
|
||||
errors: [],
|
||||
gatedEpochMs: 1_020,
|
||||
gatedMethod: 'loadFile',
|
||||
passThroughLoads: 0,
|
||||
releasedEpochMs: 1_150,
|
||||
timedOut: false,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
test('the client and the hook agree on the global key', () => {
|
||||
assert.equal(JOURNEY_RENDERER_GATE_KEY, '__iptvnatorJourneyGate');
|
||||
});
|
||||
|
||||
test('accepts a gate that held the load until the test released it', () => {
|
||||
assert.equal(
|
||||
assertJourneyRendererGate(gate(), 1_200).releasedEpochMs,
|
||||
1_150
|
||||
);
|
||||
});
|
||||
|
||||
test('rejects gates that cannot prove the probe preceded the document', () => {
|
||||
assert.throws(
|
||||
() => assertJourneyRendererGate(gate({ timedOut: true }), 1_200),
|
||||
/timed-out/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
assertJourneyRendererGate(
|
||||
gate({ errors: ['blank-failed'] }),
|
||||
1_200
|
||||
),
|
||||
/errors: blank-failed/
|
||||
);
|
||||
assert.throws(
|
||||
() => assertJourneyRendererGate(gate({ releasedEpochMs: null }), 1_200),
|
||||
/incomplete/
|
||||
);
|
||||
assert.throws(
|
||||
() => assertJourneyRendererGate(gate({ passThroughLoads: 1 }), 1_200),
|
||||
/extra-loads-1/
|
||||
);
|
||||
assert.throws(
|
||||
() => assertJourneyRendererGate(gate(), 1_100),
|
||||
/probe-before-release/
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,96 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* Main-process gate for the performance journeys, loaded into Electron with
|
||||
* `-r` from the test side (the same mechanism Playwright uses for its own
|
||||
* loader). It is not part of the application build.
|
||||
*
|
||||
* Problem: Playwright resolves `electron.launch()` while the app is already
|
||||
* creating its window, and an init script registered afterwards races the
|
||||
* renderer's first document. Electron reports no page to Playwright until a
|
||||
* navigation commits, so the gate makes the first `loadFile`/`loadURL`
|
||||
* navigate to `about:blank` first. That gives Playwright a page object the
|
||||
* test can attach `addInitScript` to; the real load proceeds only after the
|
||||
* test calls `globalThis.__iptvnatorJourneyGate.release()`. A safety timeout
|
||||
* releases the gate on its own and records that it did, so a broken test
|
||||
* cannot hang the app; the journey treats a timed-out gate as invalid.
|
||||
*/
|
||||
const GATE_KEY = '__iptvnatorJourneyGate';
|
||||
const DEFAULT_TIMEOUT_MS = 15000;
|
||||
|
||||
function installJourneyRendererGate(BrowserWindow, target, options = {}) {
|
||||
const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
|
||||
const now = options.now ?? (() => Date.now());
|
||||
const state = {
|
||||
blankLoadedEpochMs: null,
|
||||
errors: [],
|
||||
gatedEpochMs: null,
|
||||
gatedMethod: null,
|
||||
passThroughLoads: 0,
|
||||
releasedEpochMs: null,
|
||||
timedOut: false,
|
||||
};
|
||||
let releaseGate = null;
|
||||
const gate = new Promise((resolve) => {
|
||||
releaseGate = resolve;
|
||||
});
|
||||
const timer = setTimeout(() => {
|
||||
if (state.releasedEpochMs === null) {
|
||||
state.timedOut = true;
|
||||
state.releasedEpochMs = now();
|
||||
releaseGate();
|
||||
}
|
||||
}, timeoutMs);
|
||||
const api = {
|
||||
release() {
|
||||
if (state.releasedEpochMs === null) {
|
||||
state.releasedEpochMs = now();
|
||||
clearTimeout(timer);
|
||||
releaseGate();
|
||||
}
|
||||
return state;
|
||||
},
|
||||
state,
|
||||
};
|
||||
Object.defineProperty(target, GATE_KEY, {
|
||||
configurable: false,
|
||||
enumerable: false,
|
||||
value: api,
|
||||
writable: false,
|
||||
});
|
||||
for (const method of ['loadFile', 'loadURL']) {
|
||||
const original = BrowserWindow.prototype[method];
|
||||
if (typeof original !== 'function') continue;
|
||||
BrowserWindow.prototype[method] = async function gatedLoad(...args) {
|
||||
if (state.gatedEpochMs !== null) {
|
||||
state.passThroughLoads += 1;
|
||||
return original.apply(this, args);
|
||||
}
|
||||
state.gatedEpochMs = now();
|
||||
state.gatedMethod = method;
|
||||
try {
|
||||
await this.webContents.loadURL('about:blank');
|
||||
state.blankLoadedEpochMs = now();
|
||||
} catch (error) {
|
||||
state.errors.push(
|
||||
error instanceof Error ? error.message : String(error)
|
||||
);
|
||||
}
|
||||
await gate;
|
||||
return original.apply(this, args);
|
||||
};
|
||||
}
|
||||
return api;
|
||||
}
|
||||
|
||||
module.exports = { GATE_KEY, installJourneyRendererGate };
|
||||
|
||||
if (
|
||||
process.versions &&
|
||||
process.versions.electron &&
|
||||
!process.env['IPTVNATOR_JOURNEY_GATE_MANUAL']
|
||||
) {
|
||||
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
||||
const { BrowserWindow } = require('electron');
|
||||
installJourneyRendererGate(BrowserWindow, globalThis);
|
||||
}
|
||||
@@ -0,0 +1,141 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
interface GateState {
|
||||
blankLoadedEpochMs: number | null;
|
||||
errors: string[];
|
||||
gatedEpochMs: number | null;
|
||||
gatedMethod: string | null;
|
||||
passThroughLoads: number;
|
||||
releasedEpochMs: number | null;
|
||||
timedOut: boolean;
|
||||
}
|
||||
|
||||
interface GateApi {
|
||||
release(): GateState;
|
||||
state: GateState;
|
||||
}
|
||||
|
||||
interface GateModule {
|
||||
GATE_KEY: string;
|
||||
installJourneyRendererGate(
|
||||
browserWindow: { prototype: Record<string, unknown> },
|
||||
target: Record<string, unknown>,
|
||||
options?: { now?: () => number; timeoutMs?: number }
|
||||
): GateApi;
|
||||
}
|
||||
|
||||
// The e2e project compiles to CommonJS, so the hook is loaded with require.
|
||||
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
||||
const gateModule = require('./journey-renderer-gate.cjs') as GateModule;
|
||||
|
||||
function createFakeBrowserWindow(log: string[]) {
|
||||
class FakeBrowserWindow {
|
||||
webContents = {
|
||||
loadURL: async (url: string) => {
|
||||
log.push(`webContents.loadURL:${url}`);
|
||||
},
|
||||
};
|
||||
async loadFile(file: string): Promise<string> {
|
||||
log.push(`loadFile:${file}`);
|
||||
return `loaded:${file}`;
|
||||
}
|
||||
async loadURL(url: string): Promise<string> {
|
||||
log.push(`loadURL:${url}`);
|
||||
return `loaded:${url}`;
|
||||
}
|
||||
}
|
||||
return FakeBrowserWindow;
|
||||
}
|
||||
|
||||
function settle(): Promise<void> {
|
||||
return new Promise((resolve) => setTimeout(resolve, 5));
|
||||
}
|
||||
|
||||
test('the module does not touch Electron when loaded outside it', () => {
|
||||
assert.equal(typeof gateModule.installJourneyRendererGate, 'function');
|
||||
assert.equal(gateModule.GATE_KEY, '__iptvnatorJourneyGate');
|
||||
assert.equal(
|
||||
(globalThis as Record<string, unknown>)[gateModule.GATE_KEY],
|
||||
undefined
|
||||
);
|
||||
});
|
||||
|
||||
test('holds the first load behind about:blank until released, then passes later loads through', async () => {
|
||||
const log: string[] = [];
|
||||
const FakeBrowserWindow = createFakeBrowserWindow(log);
|
||||
const target: Record<string, unknown> = {};
|
||||
let clock = 100;
|
||||
const api = gateModule.installJourneyRendererGate(
|
||||
FakeBrowserWindow as unknown as { prototype: Record<string, unknown> },
|
||||
target,
|
||||
{ now: () => clock++, timeoutMs: 60_000 }
|
||||
);
|
||||
assert.equal(target[gateModule.GATE_KEY], api);
|
||||
const window = new FakeBrowserWindow();
|
||||
const load = window.loadFile('index.html');
|
||||
await settle();
|
||||
assert.deepEqual(log, ['webContents.loadURL:about:blank']);
|
||||
assert.equal(api.state.gatedMethod, 'loadFile');
|
||||
assert.equal(api.state.gatedEpochMs, 100);
|
||||
assert.equal(api.state.blankLoadedEpochMs, 101);
|
||||
assert.equal(api.state.releasedEpochMs, null);
|
||||
|
||||
api.release();
|
||||
assert.equal(await load, 'loaded:index.html');
|
||||
assert.deepEqual(log, [
|
||||
'webContents.loadURL:about:blank',
|
||||
'loadFile:index.html',
|
||||
]);
|
||||
assert.equal(api.state.releasedEpochMs, 102);
|
||||
assert.equal(api.state.timedOut, false);
|
||||
|
||||
assert.equal(
|
||||
await window.loadURL('http://localhost/'),
|
||||
'loaded:http://localhost/'
|
||||
);
|
||||
assert.equal(api.state.passThroughLoads, 1);
|
||||
assert.equal(api.release().releasedEpochMs, 102);
|
||||
});
|
||||
|
||||
test('releases itself after the timeout and records it', async () => {
|
||||
const log: string[] = [];
|
||||
const FakeBrowserWindow = createFakeBrowserWindow(log);
|
||||
const api = gateModule.installJourneyRendererGate(
|
||||
FakeBrowserWindow as unknown as { prototype: Record<string, unknown> },
|
||||
{},
|
||||
{ timeoutMs: 10 }
|
||||
);
|
||||
const window = new FakeBrowserWindow();
|
||||
assert.equal(
|
||||
await window.loadURL('http://localhost/'),
|
||||
'loaded:http://localhost/'
|
||||
);
|
||||
assert.equal(api.state.timedOut, true);
|
||||
assert.equal(typeof api.state.releasedEpochMs, 'number');
|
||||
});
|
||||
|
||||
test('records a failed about:blank navigation and still loads after release', async () => {
|
||||
class BrokenBrowserWindow {
|
||||
webContents = {
|
||||
loadURL: async () => {
|
||||
throw new Error('blank-failed');
|
||||
},
|
||||
};
|
||||
async loadFile(file: string): Promise<string> {
|
||||
return `loaded:${file}`;
|
||||
}
|
||||
}
|
||||
const api = gateModule.installJourneyRendererGate(
|
||||
BrokenBrowserWindow as unknown as {
|
||||
prototype: Record<string, unknown>;
|
||||
},
|
||||
{},
|
||||
{ timeoutMs: 60_000 }
|
||||
);
|
||||
const load = new BrokenBrowserWindow().loadFile('index.html');
|
||||
await settle();
|
||||
assert.deepEqual(api.state.errors, ['blank-failed']);
|
||||
api.release();
|
||||
assert.equal(await load, 'loaded:index.html');
|
||||
});
|
||||
@@ -0,0 +1,409 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import { JSDOM } from 'jsdom';
|
||||
|
||||
import {
|
||||
assertJourneyRendererProbeState,
|
||||
createLaunchJourneyProbeOptions,
|
||||
JOURNEY_IPC_SENTINEL_ID,
|
||||
JOURNEY_IPC_SENTINEL_METHOD,
|
||||
JOURNEY_PROBE_STATE_KEY,
|
||||
journeyRendererProbeScript,
|
||||
type JourneyRendererProbeOptions,
|
||||
type JourneyRendererProbeState,
|
||||
} from './journey-renderer-probe';
|
||||
|
||||
interface FakeEntry {
|
||||
duration?: number;
|
||||
entryType: string;
|
||||
hadRecentInput?: boolean;
|
||||
startTime: number;
|
||||
value?: number;
|
||||
}
|
||||
|
||||
interface FakeObserver {
|
||||
disconnected: boolean;
|
||||
emit(entries: FakeEntry[]): void;
|
||||
queue: FakeEntry[];
|
||||
type: string | null;
|
||||
}
|
||||
|
||||
interface Fixture {
|
||||
readonly bridgeCalls: unknown[];
|
||||
readonly observers: FakeObserver[];
|
||||
/** The live state object inside the jsdom realm. */
|
||||
readonly rawState: () => JourneyRendererProbeState;
|
||||
/** A JSON clone, so assertions compare values across realms. */
|
||||
readonly state: () => JourneyRendererProbeState;
|
||||
readonly window: JSDOM['window'];
|
||||
}
|
||||
|
||||
const PAGE = `<!doctype html><html><head></head><body class="mat-app-background">
|
||||
<div id="initial-splash" role="status"><span>IPTVnator</span></div>
|
||||
<app-root></app-root></body></html>`;
|
||||
|
||||
function installFakePerformance(
|
||||
window: JSDOM['window'],
|
||||
observers: FakeObserver[]
|
||||
): void {
|
||||
class FakePerformanceObserver implements FakeObserver {
|
||||
disconnected = false;
|
||||
queue: FakeEntry[] = [];
|
||||
type: string | null = null;
|
||||
constructor(
|
||||
private readonly callback: (list: {
|
||||
getEntries(): FakeEntry[];
|
||||
}) => void
|
||||
) {
|
||||
observers.push(this);
|
||||
}
|
||||
observe(options: { type: string }): void {
|
||||
this.type = options.type;
|
||||
}
|
||||
takeRecords(): FakeEntry[] {
|
||||
const queued = this.queue;
|
||||
this.queue = [];
|
||||
return queued;
|
||||
}
|
||||
disconnect(): void {
|
||||
this.disconnected = true;
|
||||
}
|
||||
emit(entries: FakeEntry[]): void {
|
||||
this.callback({ getEntries: () => entries });
|
||||
}
|
||||
}
|
||||
Object.defineProperty(window, 'PerformanceObserver', {
|
||||
configurable: true,
|
||||
value: FakePerformanceObserver,
|
||||
});
|
||||
Object.defineProperty(window.performance, 'getEntriesByType', {
|
||||
configurable: true,
|
||||
value: (type: string) =>
|
||||
type === 'navigation'
|
||||
? [{ domContentLoadedEventEnd: 100, loadEventEnd: 120 }]
|
||||
: [],
|
||||
});
|
||||
// jsdom never lays out, so visibility is "connected to the document".
|
||||
window.HTMLElement.prototype.getClientRects = function getClientRects(
|
||||
this: HTMLElement
|
||||
) {
|
||||
return (this.isConnected ? [{}] : []) as unknown as DOMRectList;
|
||||
};
|
||||
}
|
||||
|
||||
function createFixture(
|
||||
overrides: Partial<JourneyRendererProbeOptions> & {
|
||||
bridge?: boolean;
|
||||
url?: string;
|
||||
} = {}
|
||||
): Fixture {
|
||||
const {
|
||||
bridge = true,
|
||||
url = 'file:///dist/apps/web/workspace/dashboard',
|
||||
...optionOverrides
|
||||
} = overrides;
|
||||
const dom = new JSDOM(PAGE, {
|
||||
pretendToBeVisual: true,
|
||||
runScripts: 'outside-only',
|
||||
url,
|
||||
});
|
||||
const { window } = dom;
|
||||
const observers: FakeObserver[] = [];
|
||||
const bridgeCalls: unknown[] = [];
|
||||
installFakePerformance(window, observers);
|
||||
// tsx (esbuild keepNames) rewrites named inner functions as
|
||||
// `__name(fn, 'name')` when it transpiles the probe for this test runner.
|
||||
// Playwright's Babel transform, which serializes the probe for the real
|
||||
// browser, does not, so the shim is a test-runner concern only.
|
||||
Object.defineProperty(window, '__name', {
|
||||
configurable: true,
|
||||
value: (target: unknown) => target,
|
||||
});
|
||||
if (bridge) {
|
||||
Object.defineProperty(window, 'electron', {
|
||||
configurable: true,
|
||||
value: Object.freeze({
|
||||
[JOURNEY_IPC_SENTINEL_METHOD]: (id: unknown) => {
|
||||
bridgeCalls.push(id);
|
||||
return Promise.resolve(null);
|
||||
},
|
||||
onSomething: () => undefined,
|
||||
}),
|
||||
});
|
||||
}
|
||||
const options = {
|
||||
...createLaunchJourneyProbeOptions(),
|
||||
...optionOverrides,
|
||||
};
|
||||
window.eval(
|
||||
`(${journeyRendererProbeScript.toString()})(${JSON.stringify(options)})`
|
||||
);
|
||||
const rawState = (): JourneyRendererProbeState =>
|
||||
(window as unknown as Record<string, JourneyRendererProbeState>)[
|
||||
options.stateKey
|
||||
] as JourneyRendererProbeState;
|
||||
return {
|
||||
bridgeCalls,
|
||||
observers,
|
||||
rawState,
|
||||
state: () =>
|
||||
JSON.parse(JSON.stringify(rawState())) as JourneyRendererProbeState,
|
||||
window,
|
||||
};
|
||||
}
|
||||
|
||||
function settle(ms = 40): Promise<void> {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
function renderFirstCard(fixture: Fixture): void {
|
||||
const { document } = fixture.window;
|
||||
document.getElementById('initial-splash')?.remove();
|
||||
const rail = document.createElement('section');
|
||||
rail.setAttribute('data-test-id', 'dashboard-recent-sources-rail');
|
||||
document.querySelector('app-root')?.append(rail);
|
||||
const card = document.createElement('div');
|
||||
card.setAttribute('data-test-id', 'dashboard-recent-sources-rail-card');
|
||||
rail.append(card);
|
||||
}
|
||||
|
||||
test('records install facts before any renderer script ran', () => {
|
||||
const fixture = createFixture();
|
||||
const state = fixture.state();
|
||||
assert.equal(state.schemaVersion, 1);
|
||||
assert.equal(state.journey, 'launch');
|
||||
assert.equal(state.installed.readyState, 'loading');
|
||||
assert.equal(state.installed.scriptCount, 0);
|
||||
assert.equal(state.installed.bridgePresent, true);
|
||||
assert.equal(state.capabilities.observedTarget, 'documentElement');
|
||||
assert.equal(state.capabilities.layoutShift, true);
|
||||
assert.equal(state.capabilities.longTask, true);
|
||||
assert.deepEqual(state.invalidReasons, []);
|
||||
assert.equal(state.terminal, null);
|
||||
assert.deepEqual(
|
||||
fixture.observers.map((observer) => observer.type),
|
||||
['layout-shift', 'longtask']
|
||||
);
|
||||
});
|
||||
|
||||
test('installs once per document', () => {
|
||||
const fixture = createFixture();
|
||||
const first = fixture.rawState();
|
||||
fixture.window.eval(
|
||||
`(${journeyRendererProbeScript.toString()})(${JSON.stringify(createLaunchJourneyProbeOptions())})`
|
||||
);
|
||||
assert.equal(fixture.rawState(), first);
|
||||
});
|
||||
|
||||
test('counts mutation records until the first card is visible after the splash is gone', async () => {
|
||||
const fixture = createFixture();
|
||||
const { document } = fixture.window;
|
||||
const appRoot = document.querySelector('app-root') as HTMLElement;
|
||||
appRoot.append(document.createElement('div'));
|
||||
appRoot.append(document.createElement('div'));
|
||||
appRoot.setAttribute('data-ready', '1');
|
||||
await settle();
|
||||
assert.equal(fixture.state().terminal, null);
|
||||
|
||||
renderFirstCard(fixture);
|
||||
await settle();
|
||||
const state = fixture.state();
|
||||
assert.ok(state.terminal, 'terminal must be recorded');
|
||||
assert.equal(
|
||||
state.terminal.cardTestId,
|
||||
'dashboard-recent-sources-rail-card'
|
||||
);
|
||||
assert.equal(state.terminal.cardTag, 'div');
|
||||
assert.match(state.terminal.pathname, /\/workspace\/dashboard$/);
|
||||
// 2 appends + 1 attribute + splash removal + rail append + card append
|
||||
assert.equal(state.counters.domMutations, 6);
|
||||
assert.equal(state.final, true);
|
||||
assert.equal(typeof state.firstCardPaintEpochMs, 'number');
|
||||
assert.deepEqual(fixture.bridgeCalls, [JOURNEY_IPC_SENTINEL_ID]);
|
||||
assert.equal(state.sentinel.status, 'sent');
|
||||
assert.equal(
|
||||
state.navigation?.loadEventEndEpochMs,
|
||||
fixture.window.performance.timeOrigin + 120
|
||||
);
|
||||
assert.equal(
|
||||
state.capabilities.changeDetectionTicks,
|
||||
'unavailable-ng-global-not-published'
|
||||
);
|
||||
assert.ok(fixture.observers.every((observer) => observer.disconnected));
|
||||
|
||||
appRoot.append(document.createElement('div'));
|
||||
await settle();
|
||||
assert.equal(fixture.state().counters.domMutations, 6);
|
||||
assert.doesNotThrow(() => assertJourneyRendererProbeState(fixture.state()));
|
||||
});
|
||||
|
||||
test('sums layout shifts without recent input and counts long tasks over 50 ms up to the post-paint cutoff', async () => {
|
||||
const fixture = createFixture();
|
||||
const [layoutShift, longTask] = fixture.observers as [
|
||||
FakeObserver,
|
||||
FakeObserver,
|
||||
];
|
||||
const now = () => fixture.window.performance.now();
|
||||
layoutShift.emit([
|
||||
{
|
||||
entryType: 'layout-shift',
|
||||
hadRecentInput: false,
|
||||
startTime: now(),
|
||||
value: 0.25,
|
||||
},
|
||||
{
|
||||
entryType: 'layout-shift',
|
||||
hadRecentInput: true,
|
||||
startTime: now(),
|
||||
value: 5,
|
||||
},
|
||||
]);
|
||||
longTask.emit([
|
||||
{ duration: 80, entryType: 'longtask', startTime: now() },
|
||||
{ duration: 50, entryType: 'longtask', startTime: now() },
|
||||
]);
|
||||
// Entries still queued when the terminal frame closes the observers.
|
||||
layoutShift.queue.push(
|
||||
{
|
||||
entryType: 'layout-shift',
|
||||
hadRecentInput: false,
|
||||
startTime: now(),
|
||||
value: 0.5,
|
||||
},
|
||||
{
|
||||
entryType: 'layout-shift',
|
||||
hadRecentInput: false,
|
||||
startTime: now() + 60_000,
|
||||
value: 9,
|
||||
}
|
||||
);
|
||||
longTask.queue.push(
|
||||
{ duration: 120, entryType: 'longtask', startTime: now() },
|
||||
{ duration: 300, entryType: 'longtask', startTime: now() + 60_000 }
|
||||
);
|
||||
renderFirstCard(fixture);
|
||||
await new Promise((resolve) => queueMicrotask(() => resolve(undefined)));
|
||||
// Delivered after the terminal batch, before the post-paint cutoff.
|
||||
assert.ok(fixture.rawState().terminal, 'terminal must be set');
|
||||
assert.equal(fixture.rawState().final, false);
|
||||
layoutShift.emit([
|
||||
{
|
||||
entryType: 'layout-shift',
|
||||
hadRecentInput: false,
|
||||
startTime: now(),
|
||||
value: 0.125,
|
||||
},
|
||||
{
|
||||
entryType: 'layout-shift',
|
||||
hadRecentInput: false,
|
||||
startTime: now() + 60_000,
|
||||
value: 7,
|
||||
},
|
||||
]);
|
||||
longTask.emit([
|
||||
{ duration: 64, entryType: 'longtask', startTime: now() },
|
||||
{ duration: 500, entryType: 'longtask', startTime: now() + 60_000 },
|
||||
]);
|
||||
await settle();
|
||||
const state = fixture.state();
|
||||
assert.equal(state.final, true);
|
||||
assert.ok(
|
||||
(state.firstCardPaintEpochMs ?? 0) >
|
||||
(state.terminal?.epochMs ?? Number.POSITIVE_INFINITY),
|
||||
'the cutoff is sampled after the terminal batch'
|
||||
);
|
||||
assert.equal(state.counters.layoutShiftScore, 0.875);
|
||||
assert.equal(state.counters.longTasks, 3);
|
||||
assert.deepEqual(state.longTaskDurationsMs, [80, 64, 120]);
|
||||
|
||||
layoutShift.emit([
|
||||
{
|
||||
entryType: 'layout-shift',
|
||||
hadRecentInput: false,
|
||||
startTime: now(),
|
||||
value: 1,
|
||||
},
|
||||
]);
|
||||
longTask.emit([{ duration: 99, entryType: 'longtask', startTime: now() }]);
|
||||
assert.equal(fixture.state().counters.layoutShiftScore, 0.875);
|
||||
assert.equal(fixture.state().counters.longTasks, 3);
|
||||
});
|
||||
|
||||
test('does not end while the splash is present, off the workspace route, or before a card is visible', async () => {
|
||||
const withSplash = createFixture();
|
||||
const rail = withSplash.window.document.createElement('div');
|
||||
rail.setAttribute('data-test-id', 'dashboard-recent-sources-rail-card');
|
||||
withSplash.window.document.querySelector('app-root')?.append(rail);
|
||||
await settle();
|
||||
assert.equal(withSplash.state().terminal, null);
|
||||
assert.deepEqual(withSplash.bridgeCalls, []);
|
||||
|
||||
const offRoute = createFixture({ url: 'file:///dist/apps/web/index.html' });
|
||||
renderFirstCard(offRoute);
|
||||
await settle();
|
||||
assert.equal(offRoute.state().terminal, null);
|
||||
|
||||
const noCard = createFixture();
|
||||
noCard.window.document.getElementById('initial-splash')?.remove();
|
||||
await settle();
|
||||
assert.equal(noCard.state().terminal, null);
|
||||
assert.throws(
|
||||
() => assertJourneyRendererProbeState(noCard.state()),
|
||||
/incomplete/
|
||||
);
|
||||
});
|
||||
|
||||
test('reports a missing bridge instead of guessing the IPC boundary', async () => {
|
||||
const fixture = createFixture({ bridge: false });
|
||||
assert.equal(fixture.state().installed.bridgePresent, false);
|
||||
renderFirstCard(fixture);
|
||||
await settle();
|
||||
assert.equal(fixture.state().sentinel.status, 'bridge-missing');
|
||||
assert.throws(
|
||||
() => assertJourneyRendererProbeState(fixture.state()),
|
||||
/sentinel-bridge-missing/
|
||||
);
|
||||
});
|
||||
|
||||
test('rejects a probe that was installed after the document started', async () => {
|
||||
const fixture = createFixture();
|
||||
const state = fixture.rawState() as { invalidReasons: string[] };
|
||||
state.invalidReasons.push('probe-installed-after-document-start');
|
||||
renderFirstCard(fixture);
|
||||
await settle();
|
||||
assert.throws(
|
||||
() => assertJourneyRendererProbeState(fixture.state()),
|
||||
/probe-installed-after-document-start/
|
||||
);
|
||||
});
|
||||
|
||||
test('rejects a probe whose performance observers were unavailable instead of reporting zeros', async () => {
|
||||
const fixture = createFixture();
|
||||
const state = fixture.rawState() as {
|
||||
capabilities: { layoutShift: boolean; longTask: boolean };
|
||||
};
|
||||
state.capabilities.longTask = false;
|
||||
renderFirstCard(fixture);
|
||||
await settle();
|
||||
assert.equal(fixture.state().counters.longTasks, 0);
|
||||
assert.throws(
|
||||
() => assertJourneyRendererProbeState(fixture.state()),
|
||||
/observer-unavailable: longTask/
|
||||
);
|
||||
state.capabilities.layoutShift = false;
|
||||
assert.throws(
|
||||
() => assertJourneyRendererProbeState(fixture.state()),
|
||||
/observer-unavailable: layoutShift, longTask/
|
||||
);
|
||||
});
|
||||
|
||||
test('launch options target the workspace source cards and the shared sentinel', () => {
|
||||
const options = createLaunchJourneyProbeOptions();
|
||||
assert.equal(options.stateKey, JOURNEY_PROBE_STATE_KEY);
|
||||
assert.equal(options.sentinelMethod, 'dbGetAppPlaylist');
|
||||
assert.equal(options.splashId, 'initial-splash');
|
||||
assert.equal(options.routeFragment, '/workspace');
|
||||
assert.match(options.cardSelector, /dashboard-recent-sources-rail-card/);
|
||||
assert.match(options.cardSelector, /app-playlist-item/);
|
||||
});
|
||||
@@ -0,0 +1,378 @@
|
||||
import type { Page } from '@playwright/test';
|
||||
|
||||
/**
|
||||
* Renderer-side probe for the performance journeys (J1 "Launch to usable").
|
||||
*
|
||||
* The probe is injected from the test side through `addInitScript` while the
|
||||
* journey gate (`journey-renderer-gate.cjs`) parks the window on
|
||||
* `about:blank`, so it runs before any renderer script and never touches
|
||||
* production code. It
|
||||
* counts DOM mutations, layout shifts and long tasks until the journey's
|
||||
* terminal condition and then emits one JSON blob under
|
||||
* `window.__iptvnatorJourneyProbe`.
|
||||
*
|
||||
* IPC invocations are not counted here: the bridge object exposed by
|
||||
* `contextBridge` is frozen, so the probe cannot wrap it. Instead the probe
|
||||
* fires one sentinel bridge call at the terminal moment; the main-process
|
||||
* capture (`journey-main-ipc-capture.ts`) counts the preload's renderer-API
|
||||
* trace events received before that sentinel. Renderer-to-main IPC is
|
||||
* ordered, so the count is exact regardless of clock skew.
|
||||
*/
|
||||
export const JOURNEY_PROBE_STATE_KEY = '__iptvnatorJourneyProbe';
|
||||
export const JOURNEY_PROBE_SCHEMA_VERSION = 1;
|
||||
export const JOURNEY_IPC_SENTINEL_ID = '__iptvnator-journey-sentinel__';
|
||||
/** Bridge method used for the sentinel: a read-only lookup by id. */
|
||||
export const JOURNEY_IPC_SENTINEL_METHOD = 'dbGetAppPlaylist';
|
||||
|
||||
export interface JourneyRendererProbeOptions {
|
||||
/** Selector for the element whose visibility ends the journey. */
|
||||
readonly cardSelector: string;
|
||||
readonly journey: string;
|
||||
/** Pathname fragment the terminal route must contain. */
|
||||
readonly routeFragment: string;
|
||||
readonly sentinelId: string;
|
||||
readonly sentinelMethod: string;
|
||||
/** Element id of the inline splash that must be gone at the end. */
|
||||
readonly splashId: string;
|
||||
readonly stateKey: string;
|
||||
}
|
||||
|
||||
export interface JourneyRendererProbeCounters {
|
||||
domMutations: number;
|
||||
layoutShiftScore: number;
|
||||
longTasks: number;
|
||||
}
|
||||
|
||||
export interface JourneyRendererProbeState {
|
||||
readonly capabilities: {
|
||||
changeDetectionTicks: string;
|
||||
layoutShift: boolean;
|
||||
longTask: boolean;
|
||||
observedTarget: 'document' | 'documentElement';
|
||||
};
|
||||
readonly counters: JourneyRendererProbeCounters;
|
||||
final: boolean;
|
||||
firstCardPaintEpochMs: number | null;
|
||||
readonly installed: {
|
||||
readonly bridgePresent: boolean;
|
||||
readonly documentElementPresent: boolean;
|
||||
readonly epochMs: number;
|
||||
readonly readyState: string;
|
||||
readonly scriptCount: number;
|
||||
};
|
||||
readonly invalidReasons: string[];
|
||||
readonly journey: string;
|
||||
readonly longTaskDurationsMs: number[];
|
||||
navigation: {
|
||||
readonly domContentLoadedEpochMs: number;
|
||||
readonly loadEventEndEpochMs: number;
|
||||
} | null;
|
||||
readonly schemaVersion: number;
|
||||
sentinel: {
|
||||
readonly epochMs: number | null;
|
||||
readonly status: 'bridge-missing' | 'failed' | 'not-sent' | 'sent';
|
||||
};
|
||||
terminal: {
|
||||
readonly cardTag: string;
|
||||
readonly cardTestId: string | null;
|
||||
readonly epochMs: number;
|
||||
readonly pathname: string;
|
||||
} | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Page-side script. It must stay self-contained: Playwright serializes it
|
||||
* with `toString()`, so it may only use its argument and browser globals.
|
||||
*/
|
||||
export function journeyRendererProbeScript(
|
||||
options: JourneyRendererProbeOptions
|
||||
): void {
|
||||
const target = globalThis as unknown as Record<string, unknown>;
|
||||
if (target[options.stateKey] !== undefined) {
|
||||
return;
|
||||
}
|
||||
const epoch = (): number => performance.timeOrigin + performance.now();
|
||||
const bridge = target['electron'] as Record<string, unknown> | undefined;
|
||||
const state: JourneyRendererProbeState = {
|
||||
capabilities: {
|
||||
changeDetectionTicks: 'pending',
|
||||
layoutShift: false,
|
||||
longTask: false,
|
||||
observedTarget: document.documentElement
|
||||
? 'documentElement'
|
||||
: 'document',
|
||||
},
|
||||
counters: { domMutations: 0, layoutShiftScore: 0, longTasks: 0 },
|
||||
final: false,
|
||||
firstCardPaintEpochMs: null,
|
||||
installed: {
|
||||
bridgePresent: typeof bridge === 'object' && bridge !== null,
|
||||
documentElementPresent: document.documentElement !== null,
|
||||
epochMs: epoch(),
|
||||
readyState: document.readyState,
|
||||
scriptCount: document.scripts.length,
|
||||
},
|
||||
invalidReasons: [],
|
||||
journey: options.journey,
|
||||
longTaskDurationsMs: [],
|
||||
navigation: null,
|
||||
schemaVersion: 1,
|
||||
sentinel: { epochMs: null, status: 'not-sent' },
|
||||
terminal: null,
|
||||
};
|
||||
target[options.stateKey] = state;
|
||||
if (
|
||||
state.installed.scriptCount > 0 ||
|
||||
state.installed.readyState !== 'loading'
|
||||
) {
|
||||
state.invalidReasons.push('probe-installed-after-document-start');
|
||||
}
|
||||
|
||||
const acceptLayoutShift = (
|
||||
entries: readonly PerformanceEntry[],
|
||||
untilEpochMs: number
|
||||
): void => {
|
||||
for (const entry of entries) {
|
||||
const shift = entry as PerformanceEntry & {
|
||||
hadRecentInput?: boolean;
|
||||
value?: number;
|
||||
};
|
||||
if (
|
||||
shift.hadRecentInput === true ||
|
||||
typeof shift.value !== 'number' ||
|
||||
performance.timeOrigin + shift.startTime > untilEpochMs
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
state.counters.layoutShiftScore += shift.value;
|
||||
}
|
||||
};
|
||||
const acceptLongTasks = (
|
||||
entries: readonly PerformanceEntry[],
|
||||
untilEpochMs: number
|
||||
): void => {
|
||||
for (const entry of entries) {
|
||||
if (
|
||||
entry.duration <= 50 ||
|
||||
performance.timeOrigin + entry.startTime > untilEpochMs
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
state.counters.longTasks += 1;
|
||||
state.longTaskDurationsMs.push(entry.duration);
|
||||
}
|
||||
};
|
||||
// Entries delivered between the terminal batch and the post-paint
|
||||
// cutoff wait here so the cutoff applies to them as well.
|
||||
const pendingLayoutShifts: PerformanceEntry[] = [];
|
||||
const pendingLongTasks: PerformanceEntry[] = [];
|
||||
const observe = (
|
||||
type: string,
|
||||
accept: (entries: readonly PerformanceEntry[], until: number) => void,
|
||||
pending: PerformanceEntry[]
|
||||
): PerformanceObserver | null => {
|
||||
try {
|
||||
const observer = new PerformanceObserver((list) => {
|
||||
if (state.final) return;
|
||||
if (state.terminal !== null) {
|
||||
pending.push(...list.getEntries());
|
||||
return;
|
||||
}
|
||||
accept(list.getEntries(), Number.POSITIVE_INFINITY);
|
||||
});
|
||||
observer.observe({ type, buffered: true });
|
||||
return observer;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
const layoutShiftObserver = observe(
|
||||
'layout-shift',
|
||||
acceptLayoutShift,
|
||||
pendingLayoutShifts
|
||||
);
|
||||
const longTaskObserver = observe(
|
||||
'longtask',
|
||||
acceptLongTasks,
|
||||
pendingLongTasks
|
||||
);
|
||||
state.capabilities.layoutShift = layoutShiftObserver !== null;
|
||||
state.capabilities.longTask = longTaskObserver !== null;
|
||||
|
||||
const finalize = (untilEpochMs: number): void => {
|
||||
if (layoutShiftObserver) {
|
||||
acceptLayoutShift(
|
||||
[...pendingLayoutShifts, ...layoutShiftObserver.takeRecords()],
|
||||
untilEpochMs
|
||||
);
|
||||
layoutShiftObserver.disconnect();
|
||||
}
|
||||
if (longTaskObserver) {
|
||||
acceptLongTasks(
|
||||
[...pendingLongTasks, ...longTaskObserver.takeRecords()],
|
||||
untilEpochMs
|
||||
);
|
||||
longTaskObserver.disconnect();
|
||||
}
|
||||
state.firstCardPaintEpochMs = untilEpochMs;
|
||||
state.final = true;
|
||||
};
|
||||
const sendSentinel = (): void => {
|
||||
const method = bridge?.[options.sentinelMethod];
|
||||
if (typeof method !== 'function') {
|
||||
state.sentinel = { epochMs: null, status: 'bridge-missing' };
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const result: unknown = method.call(bridge, options.sentinelId);
|
||||
state.sentinel = { epochMs: epoch(), status: 'sent' };
|
||||
void Promise.resolve(result).catch(() => undefined);
|
||||
} catch {
|
||||
state.sentinel = { epochMs: null, status: 'failed' };
|
||||
}
|
||||
};
|
||||
const isVisible = (element: Element | null): element is HTMLElement =>
|
||||
element instanceof HTMLElement && element.getClientRects().length > 0;
|
||||
const readNavigation = (): JourneyRendererProbeState['navigation'] => {
|
||||
if (typeof performance.getEntriesByType !== 'function') {
|
||||
return null;
|
||||
}
|
||||
const entry = performance.getEntriesByType('navigation')[0] as
|
||||
PerformanceNavigationTiming | undefined;
|
||||
if (!entry || entry.loadEventEnd <= 0) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
domContentLoadedEpochMs:
|
||||
performance.timeOrigin + entry.domContentLoadedEventEnd,
|
||||
loadEventEndEpochMs: performance.timeOrigin + entry.loadEventEnd,
|
||||
};
|
||||
};
|
||||
|
||||
const mutationObserver = new MutationObserver((records) => {
|
||||
if (state.terminal !== null) return;
|
||||
state.counters.domMutations += records.length;
|
||||
if (
|
||||
!location.pathname.includes(options.routeFragment) ||
|
||||
document.getElementById(options.splashId) !== null
|
||||
) {
|
||||
return;
|
||||
}
|
||||
const card = document.querySelector(options.cardSelector);
|
||||
if (!isVisible(card)) return;
|
||||
state.terminal = {
|
||||
cardTag: card.tagName.toLowerCase(),
|
||||
cardTestId: card.getAttribute('data-test-id'),
|
||||
epochMs: epoch(),
|
||||
pathname: location.pathname,
|
||||
};
|
||||
mutationObserver.disconnect();
|
||||
sendSentinel();
|
||||
state.navigation = readNavigation();
|
||||
if (state.navigation === null) {
|
||||
state.invalidReasons.push('load-event-not-finished-at-first-card');
|
||||
}
|
||||
const ng = target['ng'] as Record<string, unknown> | undefined;
|
||||
state.capabilities.changeDetectionTicks =
|
||||
typeof ng?.['ɵsetProfiler'] === 'function'
|
||||
? 'hook-present-not-counted'
|
||||
: 'unavailable-ng-global-not-published';
|
||||
// A rAF callback runs before that frame's style, layout and paint,
|
||||
// so the cutoff is sampled in a timer queued from it: by then the
|
||||
// frame that paints the card has been committed, and the render
|
||||
// task's own long task and layout shift fall inside the cutoff.
|
||||
requestAnimationFrame(() => {
|
||||
setTimeout(() => finalize(epoch()), 0);
|
||||
});
|
||||
});
|
||||
mutationObserver.observe(document.documentElement ?? document, {
|
||||
attributes: true,
|
||||
characterData: true,
|
||||
childList: true,
|
||||
subtree: true,
|
||||
});
|
||||
}
|
||||
|
||||
export function createLaunchJourneyProbeOptions(): JourneyRendererProbeOptions {
|
||||
return {
|
||||
cardSelector:
|
||||
'[data-test-id="dashboard-recent-sources-rail-card"], app-playlist-item',
|
||||
journey: 'launch',
|
||||
routeFragment: '/workspace',
|
||||
sentinelId: JOURNEY_IPC_SENTINEL_ID,
|
||||
sentinelMethod: JOURNEY_IPC_SENTINEL_METHOD,
|
||||
splashId: 'initial-splash',
|
||||
stateKey: JOURNEY_PROBE_STATE_KEY,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Registers the probe on a page that is still parked on `about:blank` by the
|
||||
* journey gate, so it is guaranteed to run at the start of the next document.
|
||||
*/
|
||||
export async function installJourneyRendererProbe(
|
||||
page: Page,
|
||||
options: JourneyRendererProbeOptions
|
||||
): Promise<void> {
|
||||
await page.addInitScript(journeyRendererProbeScript, options);
|
||||
}
|
||||
|
||||
export async function waitForJourneyRendererProbe(
|
||||
page: Page,
|
||||
stateKey: string,
|
||||
timeoutMs: number
|
||||
): Promise<JourneyRendererProbeState> {
|
||||
await page.waitForFunction(
|
||||
(key) =>
|
||||
(globalThis as unknown as Record<string, { final?: boolean }>)[key]
|
||||
?.final === true,
|
||||
stateKey,
|
||||
{ polling: 50, timeout: timeoutMs }
|
||||
);
|
||||
const state = await page.evaluate(
|
||||
(key) =>
|
||||
JSON.parse(
|
||||
JSON.stringify(
|
||||
(globalThis as unknown as Record<string, unknown>)[key]
|
||||
)
|
||||
) as unknown,
|
||||
stateKey
|
||||
);
|
||||
return assertJourneyRendererProbeState(state);
|
||||
}
|
||||
|
||||
export function assertJourneyRendererProbeState(
|
||||
value: unknown
|
||||
): JourneyRendererProbeState {
|
||||
const state = value as JourneyRendererProbeState | null;
|
||||
if (
|
||||
!state ||
|
||||
state.schemaVersion !== JOURNEY_PROBE_SCHEMA_VERSION ||
|
||||
state.final !== true ||
|
||||
state.terminal === null
|
||||
) {
|
||||
throw new Error('journey-renderer-probe-incomplete');
|
||||
}
|
||||
if (state.invalidReasons.length > 0) {
|
||||
throw new Error(
|
||||
`journey-renderer-probe-invalid: ${state.invalidReasons.join(', ')}`
|
||||
);
|
||||
}
|
||||
if (state.sentinel.status !== 'sent') {
|
||||
throw new Error(
|
||||
`journey-renderer-probe-sentinel-${state.sentinel.status}`
|
||||
);
|
||||
}
|
||||
// A zero from an observer that never ran is not a measurement; a build
|
||||
// without these entry types must fail the iteration, never lower a
|
||||
// baseline.
|
||||
const missing = (['layoutShift', 'longTask'] as const).filter(
|
||||
(capability) => !state.capabilities[capability]
|
||||
);
|
||||
if (missing.length > 0) {
|
||||
throw new Error(
|
||||
`journey-renderer-probe-observer-unavailable: ${missing.join(', ')}`
|
||||
);
|
||||
}
|
||||
return state;
|
||||
}
|
||||
@@ -0,0 +1,198 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdtemp, readFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import test from 'node:test';
|
||||
|
||||
import {
|
||||
formatJourneyOutputTimestamp,
|
||||
JOURNEY_SUMMARY_SCHEMA_VERSION,
|
||||
percentile,
|
||||
resolveJourneySummaryPath,
|
||||
summarizeJourneyIterations,
|
||||
writeJourneySummary,
|
||||
type JourneyIterationRecord,
|
||||
type JourneySummary,
|
||||
} from './journey-summary';
|
||||
|
||||
function iteration(
|
||||
index: number,
|
||||
counters: Record<string, number>,
|
||||
wallClock: Record<string, number>,
|
||||
warmup = false
|
||||
): JourneyIterationRecord {
|
||||
return {
|
||||
counters,
|
||||
evidence: {},
|
||||
index,
|
||||
pid: 100 + index,
|
||||
wallClock,
|
||||
warmup,
|
||||
};
|
||||
}
|
||||
|
||||
test('percentile interpolates linearly like the shared statistics helper', () => {
|
||||
assert.equal(percentile([10], 90), 10);
|
||||
assert.equal(percentile([30, 10, 20], 50), 20);
|
||||
assert.equal(percentile([10, 20, 30, 40, 50], 90), 46);
|
||||
assert.equal(percentile([10, 20, 30, 40, 50], 0), 10);
|
||||
assert.throws(() => percentile([], 50), /percentile-input/);
|
||||
assert.throws(() => percentile([1], 101), /percentile-input/);
|
||||
});
|
||||
|
||||
test('summarizes exact counters and P50/P90 wall-clock from measured iterations only', () => {
|
||||
const entry = summarizeJourneyIterations(
|
||||
[
|
||||
iteration(
|
||||
0,
|
||||
{ 'renderer.x': 99 },
|
||||
{ spawnToFirstCardMs: 9_000 },
|
||||
true
|
||||
),
|
||||
iteration(1, { 'renderer.x': 12 }, { spawnToFirstCardMs: 1_000 }),
|
||||
iteration(2, { 'renderer.x': 12 }, { spawnToFirstCardMs: 1_200 }),
|
||||
iteration(3, { 'renderer.x': 12 }, { spawnToFirstCardMs: 1_100 }),
|
||||
iteration(4, { 'renderer.x': 12 }, { spawnToFirstCardMs: 1_300 }),
|
||||
iteration(5, { 'renderer.x': 12 }, { spawnToFirstCardMs: 1_400 }),
|
||||
],
|
||||
{ 'renderer.y': 'not measurable' }
|
||||
);
|
||||
assert.deepEqual(entry.counters, { 'renderer.x': 12 });
|
||||
assert.deepEqual(entry.counterStability, {
|
||||
'renderer.x': { stable: true, values: [12, 12, 12, 12, 12] },
|
||||
});
|
||||
assert.deepEqual(entry.wallClock, {
|
||||
'spawnToFirstCardMs.p50': 1_200,
|
||||
'spawnToFirstCardMs.p90': 1_360,
|
||||
});
|
||||
assert.deepEqual(entry.unavailable, { 'renderer.y': 'not measurable' });
|
||||
assert.equal(entry.iterations.length, 6);
|
||||
});
|
||||
|
||||
test('reports the maximum and flags instability when measured counters disagree', () => {
|
||||
const entry = summarizeJourneyIterations(
|
||||
[
|
||||
iteration(0, { a: 3, b: 0.5 }, { w: 1 }),
|
||||
iteration(1, { a: 5, b: 0.5 }, { w: 2 }),
|
||||
iteration(2, { a: 4, b: 0.5 }, { w: 3 }),
|
||||
],
|
||||
{}
|
||||
);
|
||||
assert.deepEqual(entry.counters, { a: 5, b: 0.5 });
|
||||
assert.deepEqual(entry.counterStability['a'], {
|
||||
stable: false,
|
||||
values: [3, 5, 4],
|
||||
});
|
||||
assert.deepEqual(entry.counterStability['b'], {
|
||||
stable: true,
|
||||
values: [0.5, 0.5, 0.5],
|
||||
});
|
||||
assert.deepEqual(entry.wallClock, { 'w.p50': 2, 'w.p90': 2.8 });
|
||||
});
|
||||
|
||||
test('rejects runs that cannot produce an exact summary', () => {
|
||||
assert.throws(
|
||||
() =>
|
||||
summarizeJourneyIterations(
|
||||
[iteration(0, { a: 1 }, { w: 1 }, true)],
|
||||
{}
|
||||
),
|
||||
/no-measured-iterations/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
summarizeJourneyIterations(
|
||||
[
|
||||
iteration(0, { a: 1 }, { w: 1 }),
|
||||
iteration(1, { b: 1 }, { w: 1 }),
|
||||
],
|
||||
{}
|
||||
),
|
||||
/counter-set-mismatch-iteration-1/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
summarizeJourneyIterations(
|
||||
[
|
||||
iteration(0, { a: 1 }, { w: 1 }),
|
||||
iteration(1, { a: 1 }, { v: 1 }),
|
||||
],
|
||||
{}
|
||||
),
|
||||
/wall-clock-set-mismatch-iteration-1/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
summarizeJourneyIterations(
|
||||
[iteration(0, { a: Number.NaN }, { w: 1 })],
|
||||
{}
|
||||
),
|
||||
/counter-not-finite-a/
|
||||
);
|
||||
const duplicate = { ...iteration(1, { a: 1 }, { w: 1 }), pid: 100 };
|
||||
assert.throws(
|
||||
() =>
|
||||
summarizeJourneyIterations(
|
||||
[iteration(0, { a: 1 }, { w: 1 }), duplicate],
|
||||
{}
|
||||
),
|
||||
/duplicate-pid/
|
||||
);
|
||||
});
|
||||
|
||||
test('writes the summary below dist/performance/journeys/<timestamp> and never overwrites', async () => {
|
||||
const date = new Date('2026-09-26T10:49:12.345Z');
|
||||
assert.equal(formatJourneyOutputTimestamp(date), '20260926T104912Z');
|
||||
assert.throws(
|
||||
() => formatJourneyOutputTimestamp(new Date('nope')),
|
||||
/invalid-date/
|
||||
);
|
||||
const root = await mkdtemp(join(tmpdir(), 'iptvnator-journey-summary-'));
|
||||
try {
|
||||
const summaryPath = resolveJourneySummaryPath(root, date);
|
||||
assert.equal(
|
||||
summaryPath,
|
||||
join(
|
||||
root,
|
||||
'dist',
|
||||
'performance',
|
||||
'journeys',
|
||||
'20260926T104912Z',
|
||||
'summary.json'
|
||||
)
|
||||
);
|
||||
const summary: JourneySummary = {
|
||||
generatedAt: date.toISOString(),
|
||||
harness: {
|
||||
arch: 'arm64',
|
||||
ci: false,
|
||||
electron: '43.0.0',
|
||||
electronMain: 'dist/apps/electron-backend/main.js',
|
||||
measuredIterations: 1,
|
||||
node: 'v22',
|
||||
platform: 'darwin',
|
||||
rendererIndex: 'dist/apps/web/index.html',
|
||||
warmupIterations: 0,
|
||||
},
|
||||
journeys: {
|
||||
launch: summarizeJourneyIterations(
|
||||
[iteration(0, { a: 1 }, { w: 1 })],
|
||||
{}
|
||||
),
|
||||
},
|
||||
schemaVersion: JOURNEY_SUMMARY_SCHEMA_VERSION,
|
||||
};
|
||||
await writeJourneySummary(summaryPath, summary);
|
||||
const written = JSON.parse(
|
||||
await readFile(summaryPath, 'utf8')
|
||||
) as JourneySummary;
|
||||
assert.equal(written.journeys['launch']?.counters['a'], 1);
|
||||
assert.equal(written.journeys['launch']?.wallClock['w.p50'], 1);
|
||||
await assert.rejects(
|
||||
writeJourneySummary(summaryPath, summary),
|
||||
/EEXIST/
|
||||
);
|
||||
} finally {
|
||||
await rm(root, { force: true, recursive: true });
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,204 @@
|
||||
import { mkdir, writeFile } from 'node:fs/promises';
|
||||
import { dirname, join } from 'node:path';
|
||||
|
||||
/**
|
||||
* Journey summary schema written to
|
||||
* `dist/performance/journeys/<timestamp>/summary.json`.
|
||||
*
|
||||
* `journeys.<id>.counters.<name>` and `journeys.<id>.wallClock.<name>` are
|
||||
* plain numbers so the ratchet checker in `tools/performance/` can compare
|
||||
* them with the committed baselines. Everything else is evidence.
|
||||
*/
|
||||
export const JOURNEY_SUMMARY_SCHEMA_VERSION = 1;
|
||||
|
||||
export interface JourneyIterationRecord {
|
||||
readonly counters: Readonly<Record<string, number>>;
|
||||
readonly evidence: Readonly<Record<string, unknown>>;
|
||||
readonly index: number;
|
||||
readonly pid: number;
|
||||
readonly wallClock: Readonly<Record<string, number>>;
|
||||
readonly warmup: boolean;
|
||||
}
|
||||
|
||||
export interface JourneyCounterStability {
|
||||
readonly stable: boolean;
|
||||
readonly values: readonly number[];
|
||||
}
|
||||
|
||||
export interface JourneySummaryEntry {
|
||||
readonly counterStability: Readonly<
|
||||
Record<string, JourneyCounterStability>
|
||||
>;
|
||||
readonly counters: Readonly<Record<string, number>>;
|
||||
readonly iterations: readonly JourneyIterationRecord[];
|
||||
readonly unavailable: Readonly<Record<string, string>>;
|
||||
readonly wallClock: Readonly<Record<string, number>>;
|
||||
}
|
||||
|
||||
export interface JourneySummaryHarness {
|
||||
readonly arch: string;
|
||||
readonly ci: boolean;
|
||||
readonly electron: string;
|
||||
readonly electronMain: string;
|
||||
readonly measuredIterations: number;
|
||||
readonly node: string;
|
||||
readonly platform: string;
|
||||
readonly rendererIndex: string;
|
||||
readonly warmupIterations: number;
|
||||
}
|
||||
|
||||
export interface JourneySummary {
|
||||
readonly generatedAt: string;
|
||||
readonly harness: JourneySummaryHarness;
|
||||
readonly journeys: Readonly<Record<string, JourneySummaryEntry>>;
|
||||
readonly schemaVersion: number;
|
||||
}
|
||||
|
||||
/** Linear-interpolation percentile, the method `performance-statistics.ts` uses. */
|
||||
export function percentile(values: readonly number[], rank: number): number {
|
||||
if (
|
||||
values.length === 0 ||
|
||||
!Number.isFinite(rank) ||
|
||||
rank < 0 ||
|
||||
rank > 100
|
||||
) {
|
||||
throw new Error('journey-summary-percentile-input');
|
||||
}
|
||||
const sorted = [...values].sort((left, right) => left - right);
|
||||
if (sorted.length === 1) {
|
||||
return sorted[0] ?? 0;
|
||||
}
|
||||
const position = ((sorted.length - 1) * rank) / 100;
|
||||
const lowerIndex = Math.floor(position);
|
||||
const upperIndex = Math.ceil(position);
|
||||
const lower = sorted[lowerIndex] ?? 0;
|
||||
const upper = sorted[upperIndex] ?? lower;
|
||||
return lower + (upper - lower) * (position - lowerIndex);
|
||||
}
|
||||
|
||||
function roundTenth(value: number): number {
|
||||
return Math.round(value * 10) / 10;
|
||||
}
|
||||
|
||||
function assertSameKeys(
|
||||
expected: readonly string[],
|
||||
actual: Readonly<Record<string, number>>,
|
||||
kind: string,
|
||||
index: number
|
||||
): void {
|
||||
const keys = Object.keys(actual).sort();
|
||||
if (
|
||||
keys.length !== expected.length ||
|
||||
keys.some((key, position) => key !== expected[position])
|
||||
) {
|
||||
throw new Error(
|
||||
`journey-summary-${kind}-set-mismatch-iteration-${index}`
|
||||
);
|
||||
}
|
||||
for (const key of keys) {
|
||||
if (!Number.isFinite(actual[key])) {
|
||||
throw new Error(`journey-summary-${kind}-not-finite-${key}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Counters are exact: the summary carries the value shared by every measured
|
||||
* iteration. When iterations disagree the maximum is reported (a ratchet
|
||||
* must never read a value lower than what a run produced) and the
|
||||
* disagreement is recorded in `counterStability` so the counter is not
|
||||
* promoted to a guardrail until it is deterministic.
|
||||
*/
|
||||
export function summarizeJourneyIterations(
|
||||
iterations: readonly JourneyIterationRecord[],
|
||||
unavailable: Readonly<Record<string, string>>
|
||||
): JourneySummaryEntry {
|
||||
const measured = iterations.filter((iteration) => !iteration.warmup);
|
||||
if (measured.length === 0) {
|
||||
throw new Error('journey-summary-no-measured-iterations');
|
||||
}
|
||||
const pids = new Set(iterations.map((iteration) => iteration.pid));
|
||||
if (pids.size !== iterations.length) {
|
||||
throw new Error('journey-summary-duplicate-pid');
|
||||
}
|
||||
const first = measured[0] as JourneyIterationRecord;
|
||||
const counterNames = Object.keys(first.counters).sort();
|
||||
const wallClockNames = Object.keys(first.wallClock).sort();
|
||||
for (const iteration of measured) {
|
||||
assertSameKeys(
|
||||
counterNames,
|
||||
iteration.counters,
|
||||
'counter',
|
||||
iteration.index
|
||||
);
|
||||
assertSameKeys(
|
||||
wallClockNames,
|
||||
iteration.wallClock,
|
||||
'wall-clock',
|
||||
iteration.index
|
||||
);
|
||||
}
|
||||
const counters: Record<string, number> = {};
|
||||
const counterStability: Record<string, JourneyCounterStability> = {};
|
||||
for (const name of counterNames) {
|
||||
const values = measured.map(
|
||||
(iteration) => iteration.counters[name] as number
|
||||
);
|
||||
counters[name] = Math.max(...values);
|
||||
counterStability[name] = Object.freeze({
|
||||
stable: values.every((value) => value === values[0]),
|
||||
values: Object.freeze(values),
|
||||
});
|
||||
}
|
||||
const wallClock: Record<string, number> = {};
|
||||
for (const name of wallClockNames) {
|
||||
const values = measured.map(
|
||||
(iteration) => iteration.wallClock[name] as number
|
||||
);
|
||||
wallClock[`${name}.p50`] = roundTenth(percentile(values, 50));
|
||||
wallClock[`${name}.p90`] = roundTenth(percentile(values, 90));
|
||||
}
|
||||
return Object.freeze({
|
||||
counterStability: Object.freeze(counterStability),
|
||||
counters: Object.freeze(counters),
|
||||
iterations: Object.freeze([...iterations]),
|
||||
unavailable: Object.freeze({ ...unavailable }),
|
||||
wallClock: Object.freeze(wallClock),
|
||||
});
|
||||
}
|
||||
|
||||
/** `YYYYMMDDTHHMMSSZ`, the timestamp form the other benchmarks use. */
|
||||
export function formatJourneyOutputTimestamp(date: Date): string {
|
||||
if (Number.isNaN(date.getTime())) {
|
||||
throw new Error('journey-summary-invalid-date');
|
||||
}
|
||||
return date
|
||||
.toISOString()
|
||||
.replace(/[-:]/g, '')
|
||||
.replace(/\.\d{3}Z$/, 'Z');
|
||||
}
|
||||
|
||||
export function resolveJourneySummaryPath(
|
||||
repositoryRoot: string,
|
||||
date: Date = new Date()
|
||||
): string {
|
||||
return join(
|
||||
repositoryRoot,
|
||||
'dist',
|
||||
'performance',
|
||||
'journeys',
|
||||
formatJourneyOutputTimestamp(date),
|
||||
'summary.json'
|
||||
);
|
||||
}
|
||||
|
||||
export async function writeJourneySummary(
|
||||
summaryPath: string,
|
||||
summary: JourneySummary
|
||||
): Promise<void> {
|
||||
await mkdir(dirname(summaryPath), { recursive: true });
|
||||
await writeFile(summaryPath, `${JSON.stringify(summary, null, 2)}\n`, {
|
||||
encoding: 'utf8',
|
||||
flag: 'wx',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import type { JourneyMainIpcCaptureState } from './journey-main-ipc-capture';
|
||||
import type { JourneyRendererProbeState } from './journey-renderer-probe';
|
||||
import {
|
||||
LAUNCH_JOURNEY_UNAVAILABLE_COUNTERS,
|
||||
toLaunchIterationRecord,
|
||||
type LaunchJourneyMeasurement,
|
||||
} from './launch-journey-record';
|
||||
|
||||
function measurement(
|
||||
overrides: Partial<LaunchJourneyMeasurement> = {}
|
||||
): LaunchJourneyMeasurement {
|
||||
const renderer: JourneyRendererProbeState = {
|
||||
capabilities: {
|
||||
changeDetectionTicks: 'unavailable-ng-global-not-published',
|
||||
layoutShift: true,
|
||||
longTask: true,
|
||||
observedTarget: 'document',
|
||||
},
|
||||
counters: {
|
||||
domMutations: 480,
|
||||
layoutShiftScore: 0.123456789,
|
||||
longTasks: 2,
|
||||
},
|
||||
final: true,
|
||||
firstCardPaintEpochMs: 2_650,
|
||||
installed: {
|
||||
bridgePresent: true,
|
||||
documentElementPresent: false,
|
||||
epochMs: 1_200,
|
||||
readyState: 'loading',
|
||||
scriptCount: 0,
|
||||
},
|
||||
invalidReasons: [],
|
||||
journey: 'launch',
|
||||
longTaskDurationsMs: [71.26, 120.04],
|
||||
navigation: {
|
||||
domContentLoadedEpochMs: 1_300,
|
||||
loadEventEndEpochMs: 1_400.26,
|
||||
},
|
||||
schemaVersion: 1,
|
||||
sentinel: { epochMs: 2_601, status: 'sent' },
|
||||
terminal: {
|
||||
cardTag: 'div',
|
||||
cardTestId: 'dashboard-recent-sources-rail-card',
|
||||
epochMs: 2_600.04,
|
||||
pathname: '/dist/apps/web/workspace/dashboard',
|
||||
},
|
||||
};
|
||||
const ipc: JourneyMainIpcCaptureState = {
|
||||
callsAfterSentinel: 3,
|
||||
callsBeforeSentinel: 14,
|
||||
callsByMethod: { dbGetAppPlaylists: 1, getSettings: 13 },
|
||||
installedEpochMs: 1_100,
|
||||
malformedEvents: 0,
|
||||
processStartEpochMs: 900,
|
||||
senderIds: [1],
|
||||
sentinel: { occurrences: 1, receivedEpochMs: 2_602 },
|
||||
};
|
||||
return {
|
||||
electronVersion: '43.3.0',
|
||||
gate: {
|
||||
blankLoadedEpochMs: 1_050,
|
||||
errors: [],
|
||||
gatedEpochMs: 1_020,
|
||||
gatedMethod: 'loadFile',
|
||||
passThroughLoads: 0,
|
||||
releasedEpochMs: 1_150,
|
||||
timedOut: false,
|
||||
},
|
||||
ipc,
|
||||
pid: 4242,
|
||||
renderer,
|
||||
spawnEpochMs: 1_000,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
test('maps the probe and IPC capture to exact counters and spawn-relative wall-clock', () => {
|
||||
const record = toLaunchIterationRecord(2, false, measurement());
|
||||
assert.equal(record.index, 2);
|
||||
assert.equal(record.warmup, false);
|
||||
assert.equal(record.pid, 4242);
|
||||
assert.deepEqual(record.counters, {
|
||||
'renderer.domMutationsToFirstCard': 480,
|
||||
'renderer.ipcCallsToFirstCard': 14,
|
||||
'renderer.layoutShiftScore': 0.123,
|
||||
'renderer.longTasks': 2,
|
||||
});
|
||||
assert.deepEqual(record.wallClock, {
|
||||
spawnToDidFinishLoadMs: 400.3,
|
||||
spawnToFirstCardMs: 1_600,
|
||||
});
|
||||
assert.deepEqual(record.evidence['ipcCallsByMethod'], {
|
||||
dbGetAppPlaylists: 1,
|
||||
getSettings: 13,
|
||||
});
|
||||
assert.deepEqual(record.evidence['longTaskDurationsMs'], [71.3, 120]);
|
||||
assert.equal(record.evidence['ipcCallsAfterFirstCard'], 3);
|
||||
assert.deepEqual(record.evidence['epochs'], {
|
||||
firstCard: 2_600.04,
|
||||
firstCardPaint: 2_650,
|
||||
loadEventEnd: 1_400.26,
|
||||
mainIpcCaptureInstalled: 1_100,
|
||||
mainProcessStart: 900,
|
||||
rendererGateBlankLoaded: 1_050,
|
||||
rendererGateReleased: 1_150,
|
||||
rendererProbeInstalled: 1_200,
|
||||
spawn: 1_000,
|
||||
});
|
||||
});
|
||||
|
||||
test('rejects measurements whose clocks or probes are inconsistent', () => {
|
||||
const base = measurement();
|
||||
assert.throws(
|
||||
() =>
|
||||
toLaunchIterationRecord(0, false, {
|
||||
...base,
|
||||
renderer: { ...base.renderer, navigation: null },
|
||||
}),
|
||||
/incomplete-probe/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
toLaunchIterationRecord(0, false, { ...base, spawnEpochMs: 2_700 }),
|
||||
/clock-order/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
toLaunchIterationRecord(0, false, {
|
||||
...base,
|
||||
renderer: {
|
||||
...base.renderer,
|
||||
capabilities: {
|
||||
...base.renderer.capabilities,
|
||||
changeDetectionTicks: 'hook-present-not-counted',
|
||||
},
|
||||
},
|
||||
}),
|
||||
/cd-hook-hook-present-not-counted/
|
||||
);
|
||||
});
|
||||
|
||||
test('names the counters the harness cannot measure yet', () => {
|
||||
assert.deepEqual(Object.keys(LAUNCH_JOURNEY_UNAVAILABLE_COUNTERS).sort(), [
|
||||
'main.sqlStatementsBeforeReadyToShow',
|
||||
'renderer.cdTicksToFirstCard',
|
||||
]);
|
||||
});
|
||||
@@ -0,0 +1,120 @@
|
||||
import type { JourneyRendererGateState } from '../journeys/journey-renderer-gate-client';
|
||||
import type { JourneyMainIpcCaptureState } from './journey-main-ipc-capture';
|
||||
import type { JourneyRendererProbeState } from './journey-renderer-probe';
|
||||
import type { JourneyIterationRecord } from './journey-summary';
|
||||
|
||||
/**
|
||||
* Maps one measured launch (renderer probe + main IPC capture) to the
|
||||
* journey summary's iteration record for J1 "Launch to usable".
|
||||
*/
|
||||
export const LAUNCH_JOURNEY_ID = 'launch';
|
||||
|
||||
export const LAUNCH_JOURNEY_COUNTER = {
|
||||
DOM_MUTATIONS: 'renderer.domMutationsToFirstCard',
|
||||
IPC_CALLS: 'renderer.ipcCallsToFirstCard',
|
||||
LAYOUT_SHIFT_SCORE: 'renderer.layoutShiftScore',
|
||||
LONG_TASKS: 'renderer.longTasks',
|
||||
} as const;
|
||||
|
||||
export const LAUNCH_JOURNEY_WALL_CLOCK = {
|
||||
SPAWN_TO_DID_FINISH_LOAD: 'spawnToDidFinishLoadMs',
|
||||
SPAWN_TO_FIRST_CARD: 'spawnToFirstCardMs',
|
||||
} as const;
|
||||
|
||||
/**
|
||||
* Counters the plan lists for J1 that this harness cannot measure without
|
||||
* production changes. They are reported instead of faked.
|
||||
*/
|
||||
export const LAUNCH_JOURNEY_UNAVAILABLE_COUNTERS: Readonly<
|
||||
Record<string, string>
|
||||
> = Object.freeze({
|
||||
'main.sqlStatementsBeforeReadyToShow':
|
||||
'SQL statements are only visible as worker stdout trace lines, which are forwarded asynchronously; plan item A2 adds a countable channel.',
|
||||
'renderer.cdTicksToFirstCard':
|
||||
'The electron-performance build optimizes scripts (ngDevMode=false), so Angular does not publish window.ng and ɵsetProfiler is unavailable.',
|
||||
});
|
||||
|
||||
export interface LaunchJourneyMeasurement {
|
||||
readonly electronVersion: string;
|
||||
readonly gate: JourneyRendererGateState;
|
||||
readonly ipc: JourneyMainIpcCaptureState;
|
||||
readonly pid: number;
|
||||
readonly renderer: JourneyRendererProbeState;
|
||||
readonly spawnEpochMs: number;
|
||||
}
|
||||
|
||||
function roundTenth(value: number): number {
|
||||
return Math.round(value * 10) / 10;
|
||||
}
|
||||
|
||||
export function toLaunchIterationRecord(
|
||||
index: number,
|
||||
warmup: boolean,
|
||||
measurement: LaunchJourneyMeasurement
|
||||
): JourneyIterationRecord {
|
||||
const { ipc, renderer, spawnEpochMs } = measurement;
|
||||
if (renderer.terminal === null || renderer.navigation === null) {
|
||||
throw new Error('launch-journey-record-incomplete-probe');
|
||||
}
|
||||
const spawnToFirstCardMs = renderer.terminal.epochMs - spawnEpochMs;
|
||||
const spawnToDidFinishLoadMs =
|
||||
renderer.navigation.loadEventEndEpochMs - spawnEpochMs;
|
||||
if (
|
||||
spawnToDidFinishLoadMs <= 0 ||
|
||||
spawnToFirstCardMs <= spawnToDidFinishLoadMs
|
||||
) {
|
||||
throw new Error('launch-journey-record-clock-order');
|
||||
}
|
||||
if (
|
||||
renderer.capabilities.changeDetectionTicks !==
|
||||
'unavailable-ng-global-not-published'
|
||||
) {
|
||||
throw new Error(
|
||||
`launch-journey-record-cd-hook-${renderer.capabilities.changeDetectionTicks}`
|
||||
);
|
||||
}
|
||||
return Object.freeze({
|
||||
counters: Object.freeze({
|
||||
[LAUNCH_JOURNEY_COUNTER.DOM_MUTATIONS]:
|
||||
renderer.counters.domMutations,
|
||||
[LAUNCH_JOURNEY_COUNTER.IPC_CALLS]: ipc.callsBeforeSentinel,
|
||||
[LAUNCH_JOURNEY_COUNTER.LAYOUT_SHIFT_SCORE]:
|
||||
Math.round(renderer.counters.layoutShiftScore * 1_000) / 1_000,
|
||||
[LAUNCH_JOURNEY_COUNTER.LONG_TASKS]: renderer.counters.longTasks,
|
||||
}),
|
||||
evidence: Object.freeze({
|
||||
capabilities: renderer.capabilities,
|
||||
electronVersion: measurement.electronVersion,
|
||||
epochs: Object.freeze({
|
||||
firstCard: renderer.terminal.epochMs,
|
||||
firstCardPaint: renderer.firstCardPaintEpochMs,
|
||||
loadEventEnd: renderer.navigation.loadEventEndEpochMs,
|
||||
mainIpcCaptureInstalled: ipc.installedEpochMs,
|
||||
mainProcessStart: ipc.processStartEpochMs,
|
||||
rendererGateBlankLoaded: measurement.gate.blankLoadedEpochMs,
|
||||
rendererGateReleased: measurement.gate.releasedEpochMs,
|
||||
rendererProbeInstalled: renderer.installed.epochMs,
|
||||
spawn: spawnEpochMs,
|
||||
}),
|
||||
firstCard: Object.freeze({
|
||||
cardTag: renderer.terminal.cardTag,
|
||||
cardTestId: renderer.terminal.cardTestId,
|
||||
pathname: renderer.terminal.pathname,
|
||||
}),
|
||||
ipcCallsAfterFirstCard: ipc.callsAfterSentinel,
|
||||
ipcCallsByMethod: ipc.callsByMethod,
|
||||
longTaskDurationsMs: renderer.longTaskDurationsMs.map(roundTenth),
|
||||
observedTarget: renderer.capabilities.observedTarget,
|
||||
}),
|
||||
index,
|
||||
pid: measurement.pid,
|
||||
wallClock: Object.freeze({
|
||||
[LAUNCH_JOURNEY_WALL_CLOCK.SPAWN_TO_DID_FINISH_LOAD]: roundTenth(
|
||||
spawnToDidFinishLoadMs
|
||||
),
|
||||
[LAUNCH_JOURNEY_WALL_CLOCK.SPAWN_TO_FIRST_CARD]:
|
||||
roundTenth(spawnToFirstCardMs),
|
||||
}),
|
||||
warmup,
|
||||
});
|
||||
}
|
||||
@@ -4,8 +4,10 @@ IPTVnator measures performance through a small set of everyday user journeys.
|
||||
Each journey has deterministic counters that are asserted exactly, and
|
||||
wall-clock timings that are recorded as evidence. Counters are ratcheted in CI:
|
||||
a committed baseline may only be lowered, and only with the measured output as
|
||||
evidence. This document is the contract for that loop; `tools/performance/`
|
||||
holds the scripts.
|
||||
evidence. This document is the contract for that loop. The journey harness lives in
|
||||
`apps/electron-backend-e2e/src/journeys` and
|
||||
`apps/electron-backend-e2e/src/performance/journey-*.ts`; the ratchet scripts
|
||||
live in `tools/performance/`.
|
||||
|
||||
## Journeys
|
||||
|
||||
@@ -16,9 +18,159 @@ holds the scripts.
|
||||
| J3 `playback` | click on a channel | HTML5 `playing` event |
|
||||
| J4 `search` | six-character query typed into global search | results list settled |
|
||||
|
||||
Only the J1 counter `renderer.initialBytes` is instrumented today. The other
|
||||
journeys and counters follow the plan in `.plans/` and are added one thread at
|
||||
a time; each thread names its journey and counter in the PR description.
|
||||
J1 is instrumented today: `renderer.initialBytes` from the built output, and
|
||||
the runtime counters of the launch benchmark below. J2 to J4 follow the plan
|
||||
in `.plans/` and are added one thread at a time; each thread names its journey
|
||||
and counter in the PR description.
|
||||
|
||||
## Running the journeys
|
||||
|
||||
```bash
|
||||
pnpm run perf:journeys
|
||||
```
|
||||
|
||||
The script runs the Nx target `electron-backend-e2e:journeys`, which builds the
|
||||
`electron-performance` configuration of the Electron app and the renderer
|
||||
first, starts the Xtream mock server on the dedicated loopback port
|
||||
`127.0.0.1:3231` (override with `IPTVNATOR_JOURNEY_XTREAM_MOCK_PORT`), and runs
|
||||
`playwright.journeys.config.ts` with one worker. Each run writes one file:
|
||||
|
||||
```
|
||||
dist/performance/journeys/<YYYYMMDDTHHMMSSZ>/summary.json
|
||||
```
|
||||
|
||||
The file is never overwritten; a second run in the same second fails instead.
|
||||
`IPTVNATOR_JOURNEY_MEASURED_ITERATIONS` lowers the five measured iterations
|
||||
for a quick local check; the warm-up iteration always runs. Numbers from a
|
||||
laptop are previews: the Linux CI runner is the canonical measurer for
|
||||
baselines, as it is for `renderer.initialBytes`.
|
||||
|
||||
## J1 `launch`: launch to usable
|
||||
|
||||
The profile holds one M3U source and one Xtream portal, both served by the
|
||||
Xtream mock (`/playlist.m3u` and `player_api.php` on the same origin). The
|
||||
profile is seeded once per run through the app's own "Add playlist" dialogs,
|
||||
then every iteration copies that seeded data directory into a fresh temporary
|
||||
directory and spawns a fresh Electron process on it. One warm-up iteration is
|
||||
recorded but excluded from the summary; five measured iterations follow. The
|
||||
app lands on `/workspace/dashboard`, so the first card is a card of the
|
||||
"Recent sources" rail; an `app-playlist-item` row on `/workspace/sources`
|
||||
also ends the journey for profiles that disable the dashboard.
|
||||
|
||||
The journey ends at the first `MutationObserver` batch in which all of the
|
||||
following hold: the location is below `/workspace`, `#initial-splash` is no
|
||||
longer in the DOM, and a source card has a non-empty client rect. Counters are
|
||||
frozen at that microtask checkpoint, so bridge calls and mutations issued
|
||||
later in the same task are included and everything after it is not.
|
||||
|
||||
Three test-side pieces are injected; production code is not changed:
|
||||
|
||||
- `journey-renderer-gate.cjs` is loaded into the main process with `-r`, the
|
||||
mechanism Playwright uses for its own loader. Playwright resolves
|
||||
`electron.launch()` while the app is already creating its window, and
|
||||
Electron reports no page until a navigation commits, so an init script
|
||||
registered afterwards would race the first document. The gate makes the
|
||||
first `loadFile` navigate to `about:blank` and holds the real load until
|
||||
the test releases it. A 15 s safety timeout releases it on its own and the
|
||||
iteration is then invalid.
|
||||
- `journey-renderer-probe.ts` is registered with `addInitScript` on that
|
||||
`about:blank` page, so it runs at the start of the real document. It
|
||||
records that it ran while the document was still `loading` with zero
|
||||
scripts and emits one JSON blob under `window.__iptvnatorJourneyProbe`.
|
||||
- `journey-main-ipc-capture.ts` subscribes to the preload's renderer-API trace
|
||||
channel (`IPTVNATOR_DEBUG_TRACE_EVENT`, enabled with
|
||||
`IPTVNATOR_TRACE_IPC=1`) through `electronApp.evaluate`, also before the
|
||||
release. The record refuses an iteration whose gate timed out, saw a second
|
||||
load, or released before the probe was in place.
|
||||
|
||||
### Counters
|
||||
|
||||
| Counter | Source |
|
||||
| ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `renderer.ipcCallsToFirstCard` | `start` trace events the preload emits for every bridge invocation (listener registrations `on*`/`remove*` excluded, as in `wrapElectronApi`). The renderer probe fires one sentinel `dbGetAppPlaylist('__iptvnator-journey-sentinel__')` at the terminal moment; renderer-to-main IPC is ordered, so events before the sentinel are the exact count. |
|
||||
| `renderer.domMutationsToFirstCard` | `MutationRecord`s (not callback batches) from a `MutationObserver` on the document element with `childList`, `attributes`, `characterData` and `subtree`. When the init script runs before `<html>` exists the observer watches `document`, which the blob reports in `capabilities.observedTarget`. |
|
||||
| `renderer.layoutShiftScore` | Sum of `layout-shift` entries with `hadRecentInput === false`, rounded to three decimals (a shift of 0.0001 flips in and out of the cutoff between runs; the CLS "good" threshold is 0.1, so three decimals keep the counter exact without hiding anything a user could see). The cutoff is sampled in a timer queued from the first `requestAnimationFrame` after the terminal batch, that is after the frame that paints the card has been committed; entries delivered live after the terminal batch are buffered and filtered by the same cutoff. |
|
||||
| `renderer.longTasks` | `longtask` entries over 50 ms up to that same cutoff, which includes the task that rendered the card. The count depends on machine speed, so it is evidence until a run shows it is stable on the CI runner. |
|
||||
|
||||
Counters are exact: the summary carries the value shared by every measured
|
||||
iteration. When iterations disagree, the summary reports the maximum and marks
|
||||
the counter `stable: false` under `counterStability`; such a counter is not
|
||||
promoted to a guardrail until it is deterministic.
|
||||
|
||||
Two counters from the plan are listed under `unavailable` with the reason
|
||||
instead of being faked:
|
||||
|
||||
- `renderer.cdTicksToFirstCard`: the `electron-performance` build optimizes
|
||||
scripts, which sets `ngDevMode` to false, so Angular does not publish
|
||||
`window.ng` and `ɵsetProfiler` is unavailable. The probe checks this at the
|
||||
terminal moment and the record refuses a build where the hook exists but was
|
||||
not counted.
|
||||
- `main.sqlStatementsBeforeReadyToShow`: SQL statements are only visible as
|
||||
worker-thread trace lines on stdout, which Node forwards asynchronously, so
|
||||
they cannot be ordered against `ready-to-show`. Plan item A2 adds a channel
|
||||
that can be counted.
|
||||
|
||||
### Wall-clock
|
||||
|
||||
| Entry | Derivation |
|
||||
| --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `spawnToDidFinishLoadMs.p50/.p90` | `performance.timeOrigin + loadEventEnd` of the navigation entry (the main frame's `load`, which is what `did-finish-load` reports) minus the test-side timestamp taken just before `electron.launch`. |
|
||||
| `spawnToFirstCardMs.p50/.p90` | Terminal epoch of the renderer probe minus the same spawn timestamp. |
|
||||
|
||||
Percentiles use linear interpolation over the five measured iterations. The
|
||||
spawn timestamp includes Playwright's own launch overhead and the gate's
|
||||
`about:blank` detour: Playwright holds `app.whenReady()` until its CDP session
|
||||
is attached, and the real document loads only after the probes are in place,
|
||||
so absolute values are larger than a bare launch. They are comparable between runs of the same
|
||||
harness, which is what the ratchet needs. The main process start
|
||||
(`Date.now() - process.uptime()`) is recorded per iteration under
|
||||
`evidence.epochs` for cross-checks.
|
||||
|
||||
### Summary schema
|
||||
|
||||
```json
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"generatedAt": "2026-09-26T11:02:14.318Z",
|
||||
"harness": {
|
||||
"platform": "darwin",
|
||||
"electron": "43.3.0",
|
||||
"measuredIterations": 5,
|
||||
"warmupIterations": 1
|
||||
},
|
||||
"journeys": {
|
||||
"launch": {
|
||||
"counters": { "renderer.ipcCallsToFirstCard": 12 },
|
||||
"counterStability": {
|
||||
"renderer.ipcCallsToFirstCard": {
|
||||
"stable": true,
|
||||
"values": [12, 12, 12, 12, 12]
|
||||
}
|
||||
},
|
||||
"wallClock": {
|
||||
"spawnToFirstCardMs.p50": 1234.5,
|
||||
"spawnToFirstCardMs.p90": 1300.1
|
||||
},
|
||||
"unavailable": { "renderer.cdTicksToFirstCard": "reason" },
|
||||
"iterations": [
|
||||
{
|
||||
"index": 0,
|
||||
"warmup": true,
|
||||
"pid": 1,
|
||||
"counters": {},
|
||||
"wallClock": {},
|
||||
"evidence": {}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`journeys.<id>.counters.<name>` and `journeys.<id>.wallClock.<name>` are plain
|
||||
numbers so `tools/performance/check-journey-ratchet.mjs` can compare them with
|
||||
`tools/performance/journey-baselines.json`. A J1 baseline is added once the
|
||||
numbers are stable on the CI runner; until then the summary is evidence only.
|
||||
|
||||
## `renderer.initialBytes`
|
||||
|
||||
@@ -65,17 +217,17 @@ counter:
|
||||
|
||||
```json
|
||||
{
|
||||
"journeys": {
|
||||
"launch": {
|
||||
"renderer.initialBytes": {
|
||||
"value": 2739510,
|
||||
"unit": "bytes",
|
||||
"updatedAt": "2026-09-26",
|
||||
"evidencePr": 1693,
|
||||
"measuredWith": "pnpm nx build web && pnpm run perf:initial-bytes"
|
||||
}
|
||||
}
|
||||
"journeys": {
|
||||
"launch": {
|
||||
"renderer.initialBytes": {
|
||||
"value": 2739510,
|
||||
"unit": "bytes",
|
||||
"updatedAt": "2026-09-26",
|
||||
"evidencePr": 1693,
|
||||
"measuredWith": "pnpm nx build web && pnpm run perf:initial-bytes"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
@@ -144,3 +296,24 @@ trade-off, say so in the PR and let the maintainer decide.
|
||||
the test file in `tools/performance/project.json`.
|
||||
4. Validate the counter before it becomes a guardrail: one PR must show that
|
||||
lowering it moved wall-clock in the same journey.
|
||||
|
||||
## Adding a journey
|
||||
|
||||
1. Add `apps/electron-backend-e2e/src/journeys/<journey>.journey.ts`. Seed the
|
||||
profile through the app's dialogs, spawn a fresh process per iteration
|
||||
with `measureLaunchJourney` as the model, and drive the journey's start
|
||||
action with Playwright.
|
||||
2. Give the journey its own probe options (`cardSelector`, `routeFragment`,
|
||||
terminal condition) or extend `journey-renderer-probe.ts` when the end
|
||||
condition is not "an element became visible". Keep the probe
|
||||
self-contained: Playwright serializes it with `toString()`.
|
||||
3. Map the measurement to a `JourneyIterationRecord` in a
|
||||
`<journey>-journey-record.ts` under `src/performance/`; name counters
|
||||
`renderer.*` or `main.*`, and list counters you cannot measure under
|
||||
`unavailable` with the reason.
|
||||
4. Add the journey under `journeys.<id>` in the summary through
|
||||
`summarizeJourneyIterations`; the schema needs no change.
|
||||
5. Cover the probe with jsdom fixtures and the record and summary code with
|
||||
`node:test` (`pnpm nx run electron-backend-e2e:test-performance-harness`).
|
||||
6. Validate a counter before it becomes a guardrail: one PR must show that
|
||||
lowering it moved wall-clock in the same journey.
|
||||
@@ -160,6 +160,7 @@ pnpm nx build web
|
||||
pnpm run perf:initial-bytes # breakdown only
|
||||
pnpm run perf:initial-bytes:check # measure, then compare with the committed baseline
|
||||
pnpm nx test performance-tools
|
||||
pnpm run perf:journeys # J1 launch benchmark, writes dist/performance/journeys/<timestamp>/summary.json
|
||||
```
|
||||
|
||||
`perf:initial-bytes` reads the built `dist/apps/web/index.html` and sums the
|
||||
@@ -168,7 +169,10 @@ bytes on the initial path (the J1 counter `renderer.initialBytes`).
|
||||
`tools/performance/journey-baselines.json`; baselines only move down. CI runs
|
||||
the same check in the `Initial bytes ratchet` job of `ci.yml` for PRs that
|
||||
target `master` and for `master` pushes (dispatch it with
|
||||
`gh workflow run ci.yml --ref <branch>` for a stacked branch). The contract, what counts and how to add a counter are in the
|
||||
`gh workflow run ci.yml --ref <branch>` for a stacked branch). `perf:journeys` builds the `electron-performance` configuration and runs the
|
||||
J1 launch benchmark against the Xtream mock; its probe specs run with
|
||||
`pnpm nx run electron-backend-e2e:test-performance-harness`. The contract, what
|
||||
counts and how to add a counter or a journey are in the
|
||||
[performance journeys](performance-journeys.md) document.
|
||||
|
||||
## Logging
|
||||
|
||||
@@ -14,7 +14,7 @@ are not prerequisites for reading repository contracts.
|
||||
| Bootstrap, project placement, dependencies, aliases and lint configuration; root Nx config and project-local project.json files | [Nx boundaries](../architecture/nx-workspace-boundaries.md), [security overrides](../architecture/dependency-security-overrides.md) | [Nx architecture](../../.codex/skills/iptvnator-nx-architecture/SKILL.md) |
|
||||
| Angular conventions; docs and skills maintenance | [Agent workflow](../development/agent-workflow.md) | Use the area's skill below |
|
||||
| Unit, E2E, lint and coverage; `tools/coverage` | [Validation map](../architecture/validation-map.md) | Use the area's validation section |
|
||||
| Performance journeys, counters and the CI ratchet; `tools/performance` | [Performance journeys](../architecture/performance-journeys.md) | Read the contract directly |
|
||||
| Performance journeys, counters, benchmark probes and the CI ratchet; `apps/electron-backend-e2e/src/journeys`, `apps/electron-backend-e2e/src/performance`, `tools/performance` | [Performance journeys](../architecture/performance-journeys.md) | Read the contract directly |
|
||||
| Electron entry/events/preload and CDP; `apps/electron-backend` | [Debugging and trace flags](../development/electron-debugging.md), [Electron security](../architecture/electron-security.md) | Use the available global electron skill for automation |
|
||||
| Releases, notes, screenshots, native assets, Linux manager metadata; `tools/release` | [Release pipeline](../architecture/release-pipeline.md), [note format](../../.changes/README.md) | [Release notes](../../.codex/skills/release-notes/SKILL.md), [release cut](../../.codex/skills/release-cut/SKILL.md) |
|
||||
|
||||
|
||||
@@ -73,6 +73,7 @@
|
||||
"i18n:check": "node tools/i18n/check-drift.mjs",
|
||||
"perf:initial-bytes": "node tools/performance/measure-initial-bytes.mjs",
|
||||
"perf:initial-bytes:check": "node tools/performance/measure-initial-bytes.mjs --summary dist/performance/initial-bytes.summary.json && node tools/performance/check-journey-ratchet.mjs --summary dist/performance/initial-bytes.summary.json --only launch/renderer.initialBytes",
|
||||
"perf:journeys": "nx run electron-backend-e2e:journeys",
|
||||
"perf:ratchet:check": "node tools/performance/check-journey-ratchet.mjs --summary dist/performance/journey-summary.json",
|
||||
"perf:tools:test": "node --test tools/performance/measure-initial-bytes.test.mjs tools/performance/check-journey-ratchet.test.mjs tools/performance/check-baseline-direction.test.mjs",
|
||||
"agents:validate": "node tools/skills/validate-agent-guidance.mjs",
|
||||
@@ -197,6 +198,7 @@
|
||||
"@types/cors": "2.8.19",
|
||||
"@types/express": "5.0.6",
|
||||
"@types/jest": "^30.0.0",
|
||||
"@types/jsdom": "21.1.7",
|
||||
"@types/mocha": "9.0.0",
|
||||
"@types/node": "20.19.9",
|
||||
"@types/proxy-from-env": "1.0.4",
|
||||
@@ -230,6 +232,7 @@
|
||||
"jest-environment-node": "^30.5.1",
|
||||
"jest-preset-angular": "17.0.0",
|
||||
"jest-util": "^30.5.1",
|
||||
"jsdom": "26.1.0",
|
||||
"jsonc-eslint-parser": "^2.1.0",
|
||||
"material-design-icons-iconfont": "6.7.0",
|
||||
"mrmime": "2.0.1",
|
||||
|
||||
Generated
+6
@@ -338,6 +338,9 @@ importers:
|
||||
'@types/jest':
|
||||
specifier: ^30.0.0
|
||||
version: 30.0.0
|
||||
'@types/jsdom':
|
||||
specifier: 21.1.7
|
||||
version: 21.1.7
|
||||
'@types/mocha':
|
||||
specifier: 9.0.0
|
||||
version: 9.0.0
|
||||
@@ -437,6 +440,9 @@ importers:
|
||||
jest-util:
|
||||
specifier: ^30.5.1
|
||||
version: 30.5.1
|
||||
jsdom:
|
||||
specifier: 26.1.0
|
||||
version: 26.1.0
|
||||
jsonc-eslint-parser:
|
||||
specifier: ^2.1.0
|
||||
version: 2.4.2
|
||||
|
||||
Reference in new issue
Block a user