mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-08 09:01:03 -08:00
test(perf): add the J2 open-source journey (#1730)
* test(perf): add the J2 open-source journey Measure the click on the Xtream portal card until the category list and the first page of the opened section are painted, in the same fresh process as J1 after its counters are final and the app has settled. - journey-renderer-probe: optional click start (capture-phase listener on window, start sentinel before the app sees the click, entries before the click dropped), companion selectors, recent-input layout shifts tallied - journey-main-ipc-capture: optional start sentinel; counts calls between the two sentinels - journey-mock-request-ledger: loopback proxy that counts every request the app sends to the mock without storing credentials - open-source-journey-record: J2 counters and evidence - journey-run / journey-summary: every journey spec of one perf:journeys run adds its entry to the same summary.json - docs: J2 contract in performance-journeys.md Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): stop echoing the request URL from the ledger spec's upstream CodeQL flagged the fake upstream as reflected XSS. It now records what it received server-side and answers with a fixed text/plain body. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): address J2 review findings - Sentinels use cancelSourceProbe: the preload traces the call before forwarding it and SOURCE_HEALTH_CANCEL is an in-memory map lookup, so a marker no longer runs a SQLite query on the worker ahead of the measured work (Codex P1). A spec pins that handler contract. - A run is started only in the Playwright runner, replacing inherited values, and carries a random harness.runId; summaries from another invocation are never merged (Greptile P1, Codex P2). - The mock ledger tracks in-flight requests; settling and the HTTP window require none in flight (Codex P2). - clickToFirstPagePaintMs reports click to the committed paint next to the terminal-batch clickToFirstPageMs (Codex P1). - The Playwright attachment carries the whole summary (Greptile P2). - jsdom probe specs wait for the post-paint cutoff instead of a fixed 40 ms, which flaked when the harness runs all files in parallel. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(validation): describe perf:journeys as running J1 and J2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): settle J2 on pending bridge calls and start HTTP at the click - The journey IPC capture pairs every traced start with its success or error and exposes the calls still in flight. J2 settles only when J1's capture, installed before the document loaded, has none pending, so a slow startup call cannot resolve after the click and count as J2. - The mock HTTP window starts at the renderer's click stamp instead of the test-side mark taken before Playwright's actionability checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): restart the J2 quiet period when pending work completes Both waits in the open-source journey (settling before the click, closing the mock window after the terminal) now use one waitForJourneyQuiet helper that compares whole samples, in-flight counts included. The poll that first sees a request or bridge call complete restarts the quiet period, so the window is never measured from a poll at which work was still pending. A fake-clock spec covers the in-flight to zero case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): align J2 with the main-process counters from #1715 After rebasing on #1715, J1 measures main.sqlStatementsBeforeReadyToShow, so J2's reason for listing main.sqlStatementsToFirstPage as unavailable (no countable channel) was stale. State the actual limit: the running total is read from the test process and cannot be bounded at the click or the first-page batch. The performance-journeys CI job comment now names both journeys. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): launch J2 without SQL counting and stamp mock requests in sub-ms - runLaunchJourney takes the launch instrumentation; only J1 turns on the main-process counters and IPTVNATOR_PERF_COUNT_SQL, so J2's click is not measured under the hook that wraps every SQLite statement. The flags are built in journey-launch-environment.ts, which the SQL opt-in guard now expects, and a launch record without main counters is rejected. - The mock ledger stamps arrivals with performance.timeOrigin + performance.now(), the same sub-millisecond epoch as the renderer's click, so a request later in the click's millisecond is not counted before it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): reject J2 iterations with activity after settling - The open-source record compares the settle snapshot with what the probe and the IPC capture counted up to the click event, and with the mock requests between the snapshot and the click stamp. Any change means background work began during Playwright's actionability checks and could land in J2, so the iteration is rejected. - The SQL opt-in guard also checks who passes mainCounters: true: only measureLaunchJourney may, and J2 must pass false. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): count the long task that dispatches the J2 click A long task's startTime precedes the click event's timestamp when the listener runs inside it, so the start-time filter dropped the task that performs the interaction. Long tasks now count when their range overlaps the window: on one main thread only the dispatching task can overlap the click. Layout shifts keep the start-time filter. J1 is unchanged (its window starts at -Infinity). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): bound J2's late-request check by the quiet sample's mark The late-activity check compared requests against a fresh ledger mark taken after waitForQuiet returned. A request that arrived while the final quiet sample was still reading the IPC capture advanced that mark and escaped the check. The boundary is now the ledger position read by the accepted sample itself, like its DOM and IPC counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): end J2's HTTP window at the accepted quiet sample The post-terminal window read the ledger after waitForMockQuiet returned, so a request arriving in between was counted although its completion was never waited for. waitForMockQuiet now returns the ledger position its accepted sample read; later requests are kept as evidence (httpRequestsAfterSettledByRoute) instead of the counter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): observe late mock requests before reading J2's ledger httpRequestsAfterSettledByRoute read the ledger right after the accepted quiet sample, so late requests had no chance to appear in it. The ledger is now read after another quiet interval; the counter stays bounded by the quiet sample's mark and late traffic shows up in the evidence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): fail the J2 quiet wait when a sample stalls past its deadline waitForJourneyQuiet accepted a sample that returned unchanged after a stall longer than the timeout as the end of a quiet period, before the deadline check ran. The deadline is now checked first, so a stalled sample fails the wait instead of letting the click go ahead unobserved. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): detach J1's IPC capture before the J2 click J2 used J1's capture to see pending launch bridge calls while settling, but its ipcMain listener stayed attached and ran for every bridge call of the measured click. The capture can now be detached; J2 detaches J1's right after settling, before the click. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(perf): sample both J2 settle captures in one main-process snapshot The settle sample read J1's capture (pending calls) and J2's capture (call count) in two evaluate calls, so a call starting in between was counted with a stale zero in flight and its completion went unseen. Both states are now read in one synchronous pass, where no ipcMain event can be handled in between. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: 4gray <fourgray@proton.me> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
26 files changed
+2987
-188
No files matched your search
@@ -295,8 +295,9 @@ jobs:
|
||||
needs: performance-journeys-scope
|
||||
if: needs.performance-journeys-scope.outputs.run == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
# The electron-performance build is the bulk of the time; the launch
|
||||
# journey itself is six fresh Electron processes plus one seeding run.
|
||||
# The electron-performance build is the bulk of the time; each journey
|
||||
# (launch, open-source) is six fresh Electron processes plus one
|
||||
# seeding run.
|
||||
timeout-minutes: 30
|
||||
# Warn-only for the first two weeks of plan item B3: a failure is
|
||||
# visible on the run but does not fail the workflow.
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
import { randomUUID } from 'node:crypto';
|
||||
|
||||
import { workspaceRoot } from '@nx/devkit';
|
||||
import { defineConfig } from '@playwright/test';
|
||||
|
||||
/**
|
||||
* Performance journeys (docs/architecture/performance-journeys.md). One
|
||||
* worker, no retries: every journey spawns its own Electron processes and
|
||||
* writes one summary per run. The Xtream mock serves both the M3U playlist
|
||||
* adds its entry to the run's single summary file. The Xtream mock serves both the M3U playlist
|
||||
* and the portal on a dedicated loopback port so a normal E2E server on
|
||||
* 3211 cannot be reused by accident. The mock runs as one node process
|
||||
* rather than through `nx run …:serve`, whose detached process group outlived
|
||||
@@ -15,6 +17,16 @@ import { defineConfig } from '@playwright/test';
|
||||
const xtreamMockPort =
|
||||
process.env['IPTVNATOR_JOURNEY_XTREAM_MOCK_PORT'] ?? '3231';
|
||||
|
||||
// One summary file per invocation. The runner loads this config before it
|
||||
// forks the worker, and the worker (which has TEST_WORKER_INDEX) loads it
|
||||
// again: only the runner starts a run, so every journey spec of one
|
||||
// invocation shares its start time and id, a restarted worker keeps them,
|
||||
// and values left in the environment by an earlier invocation are replaced.
|
||||
if (process.env['TEST_WORKER_INDEX'] === undefined) {
|
||||
process.env['IPTVNATOR_JOURNEY_RUN_STARTED_AT'] = new Date().toISOString();
|
||||
process.env['IPTVNATOR_JOURNEY_RUN_ID'] = randomUUID();
|
||||
}
|
||||
|
||||
export default defineConfig({
|
||||
fullyParallel: false,
|
||||
reporter: [['list']],
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { relative } from 'node:path';
|
||||
|
||||
import { test } from '@playwright/test';
|
||||
|
||||
import {
|
||||
electronMainPath,
|
||||
packagedRendererIndexPath,
|
||||
workspaceRoot,
|
||||
} from '../electron-test-fixtures';
|
||||
import {
|
||||
JOURNEY_RUN_ID_ENV,
|
||||
recordJourneySummaryEntry,
|
||||
resolveJourneyRunSummaryPath,
|
||||
summarizeJourneyIterations,
|
||||
type JourneyIterationRecord,
|
||||
type JourneySummaryHarness,
|
||||
} from '../performance/journey-summary';
|
||||
|
||||
/**
|
||||
* Run-wide settings and summary output shared by the journey specs.
|
||||
* Contract: docs/architecture/performance-journeys.md.
|
||||
*/
|
||||
export const JOURNEY_WARMUP_ITERATIONS = 1;
|
||||
export const JOURNEY_MEASURED_ITERATIONS = readPositiveInteger(
|
||||
'IPTVNATOR_JOURNEY_MEASURED_ITERATIONS',
|
||||
5
|
||||
);
|
||||
export const JOURNEY_ITERATION_TIMEOUT_MS = 120_000;
|
||||
/** Outside `playwright.journeys.config.ts` every worker is its own run. */
|
||||
const JOURNEY_RUN_ID = process.env[JOURNEY_RUN_ID_ENV] || randomUUID();
|
||||
|
||||
function readPositiveInteger(name: string, fallback: number): number {
|
||||
const raw = process.env[name];
|
||||
if (raw === undefined || raw === '') {
|
||||
return fallback;
|
||||
}
|
||||
const value = Number(raw);
|
||||
if (!Number.isSafeInteger(value) || value < 1) {
|
||||
throw new Error(`${name} must be a positive integer`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
export function logJourneyIteration(
|
||||
journeyId: string,
|
||||
record: JourneyIterationRecord
|
||||
): void {
|
||||
console.log(
|
||||
`[journey:${journeyId}] iteration ${record.index}${record.warmup ? ' (warm-up)' : ''} pid=${record.pid} ${JSON.stringify(
|
||||
{ ...record.counters, ...record.wallClock }
|
||||
)}`
|
||||
);
|
||||
}
|
||||
|
||||
/** Summarizes one journey and adds it to this run's summary file. */
|
||||
export async function writeJourneyRunEntry(
|
||||
journeyId: string,
|
||||
iterations: readonly JourneyIterationRecord[],
|
||||
unavailable: Readonly<Record<string, string>>,
|
||||
electronVersion: string
|
||||
): Promise<void> {
|
||||
const entry = summarizeJourneyIterations(iterations, unavailable);
|
||||
const harness: JourneySummaryHarness = {
|
||||
arch: process.arch,
|
||||
ci: Boolean(process.env['CI']),
|
||||
electron: electronVersion,
|
||||
electronMain: relative(workspaceRoot, electronMainPath),
|
||||
measuredIterations: JOURNEY_MEASURED_ITERATIONS,
|
||||
node: process.version,
|
||||
platform: process.platform,
|
||||
rendererIndex: relative(workspaceRoot, packagedRendererIndexPath),
|
||||
runId: JOURNEY_RUN_ID,
|
||||
warmupIterations: JOURNEY_WARMUP_ITERATIONS,
|
||||
};
|
||||
const summaryPath = resolveJourneyRunSummaryPath(workspaceRoot);
|
||||
const summary = await recordJourneySummaryEntry(
|
||||
summaryPath,
|
||||
harness,
|
||||
journeyId,
|
||||
entry
|
||||
);
|
||||
// The whole summary as written so far, so the attachment is a valid
|
||||
// journey summary on its own.
|
||||
await test.info().attach(`journey-summary-${journeyId}`, {
|
||||
body: JSON.stringify(summary, null, 2),
|
||||
contentType: 'application/json',
|
||||
});
|
||||
console.log(
|
||||
`[journey:${journeyId}] summary ${relative(workspaceRoot, summaryPath)}\n${JSON.stringify(
|
||||
{
|
||||
counters: entry.counters,
|
||||
counterStability: entry.counterStability,
|
||||
unavailable: entry.unavailable,
|
||||
wallClock: entry.wallClock,
|
||||
},
|
||||
null,
|
||||
2
|
||||
)}`
|
||||
);
|
||||
}
|
||||
@@ -2,7 +2,11 @@ import { cp, mkdtemp, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join, resolve } from 'node:path';
|
||||
|
||||
import { _electron as electron, type Page } from '@playwright/test';
|
||||
import {
|
||||
_electron as electron,
|
||||
type ElectronApplication,
|
||||
type Page,
|
||||
} from '@playwright/test';
|
||||
|
||||
import { captureElectronProcess } from '../electron-process-lifecycle';
|
||||
import {
|
||||
@@ -28,6 +32,10 @@ import {
|
||||
assertJourneyMainCounters,
|
||||
readJourneyMainCounters,
|
||||
} from '../performance/journey-main-counters';
|
||||
import {
|
||||
journeyLaunchEnvironment,
|
||||
type JourneyLaunchInstrumentation,
|
||||
} from '../performance/journey-launch-environment';
|
||||
import {
|
||||
createLaunchJourneyProbeOptions,
|
||||
installJourneyRendererProbe,
|
||||
@@ -111,34 +119,54 @@ export function removeLaunchJourneyProfile(directory: string): Promise<void> {
|
||||
return removeDirectory(directory);
|
||||
}
|
||||
|
||||
/** The running app after J1 ended, for journeys that continue from there. */
|
||||
export interface LaunchJourneySession {
|
||||
readonly electronApp: ElectronApplication;
|
||||
readonly launch: LaunchJourneyMeasurement;
|
||||
readonly mainWindow: Page;
|
||||
}
|
||||
|
||||
export async function measureLaunchJourney(
|
||||
templateDirectory: string,
|
||||
timeoutMs: number
|
||||
): Promise<LaunchJourneyMeasurement> {
|
||||
const { launch } = await runLaunchJourney(
|
||||
templateDirectory,
|
||||
timeoutMs,
|
||||
{ mainCounters: true },
|
||||
async () => undefined
|
||||
);
|
||||
return launch;
|
||||
}
|
||||
|
||||
/**
|
||||
* Spawns a fresh Electron process on a copy of the seeded profile. The gate
|
||||
* hook parks the first renderer load on `about:blank`, which gives Playwright
|
||||
* a page to attach the renderer probe to; the main-process IPC capture is
|
||||
* installed next, and only then is the real load released. Both captures are
|
||||
* therefore in place before the renderer runs any script, and the probe,
|
||||
* capture and gate records still prove it.
|
||||
* capture and gate records still prove it. `continueJourney` runs in the
|
||||
* same process after J1's counters are final, before the app is closed.
|
||||
* Without `instrumentation.mainCounters` the main-process counters and SQL
|
||||
* counting stay off and `launch.mainCounters` is null.
|
||||
*/
|
||||
export async function measureLaunchJourney(
|
||||
export async function runLaunchJourney<T>(
|
||||
templateDirectory: string,
|
||||
timeoutMs: number
|
||||
): Promise<LaunchJourneyMeasurement> {
|
||||
timeoutMs: number,
|
||||
instrumentation: JourneyLaunchInstrumentation,
|
||||
continueJourney: (session: LaunchJourneySession) => Promise<T>
|
||||
): Promise<{
|
||||
readonly continuation: T;
|
||||
readonly launch: LaunchJourneyMeasurement;
|
||||
}> {
|
||||
const dataDirectory = await mkdtemp(
|
||||
join(tmpdir(), 'iptvnator-journey-launch-')
|
||||
);
|
||||
try {
|
||||
await cp(templateDirectory, dataDirectory, { recursive: true });
|
||||
// IPTVNATOR_PERF_CAPTURE turns on the main-process counters and
|
||||
// their read handler, IPTVNATOR_PERF_COUNT_SQL the SQL statement
|
||||
// count behind main.sqlStatementsBeforeReadyToShow; only this journey
|
||||
// sets it. See journey-main-counters.ts.
|
||||
const env = buildElectronLaunchEnvironment(
|
||||
dataDirectory,
|
||||
launchOptions({
|
||||
IPTVNATOR_PERF_CAPTURE: '1',
|
||||
IPTVNATOR_PERF_COUNT_SQL: '1',
|
||||
IPTVNATOR_TRACE_IPC: '1',
|
||||
})
|
||||
launchOptions(journeyLaunchEnvironment(instrumentation))
|
||||
);
|
||||
const args = buildElectronLaunchArgs([
|
||||
'-r',
|
||||
@@ -201,20 +229,22 @@ export async function measureLaunchJourney(
|
||||
10_000
|
||||
);
|
||||
// Read after the probe finished, so both frozen counters exist.
|
||||
const mainCounters = assertJourneyMainCounters(
|
||||
await readJourneyMainCounters(
|
||||
electronApp,
|
||||
JOURNEY_RENDERER_GATE_KEY
|
||||
),
|
||||
gate
|
||||
);
|
||||
const mainCounters = instrumentation.mainCounters
|
||||
? assertJourneyMainCounters(
|
||||
await readJourneyMainCounters(
|
||||
electronApp,
|
||||
JOURNEY_RENDERER_GATE_KEY
|
||||
),
|
||||
gate
|
||||
)
|
||||
: null;
|
||||
if (ipc.installedEpochMs > renderer.installed.epochMs) {
|
||||
throw new Error('journey-main-ipc-capture-installed-late');
|
||||
}
|
||||
const electronVersion = await electronApp.evaluate(
|
||||
() => process.versions.electron
|
||||
);
|
||||
return {
|
||||
const launch: LaunchJourneyMeasurement = {
|
||||
electronVersion,
|
||||
gate,
|
||||
ipc,
|
||||
@@ -223,6 +253,12 @@ export async function measureLaunchJourney(
|
||||
renderer,
|
||||
spawnEpochMs,
|
||||
};
|
||||
const continuation = await continueJourney({
|
||||
electronApp,
|
||||
launch,
|
||||
mainWindow,
|
||||
});
|
||||
return { continuation, launch };
|
||||
} finally {
|
||||
await closeElectronApplicationAndConfirmExit(
|
||||
electronApp,
|
||||
|
||||
@@ -1,25 +1,18 @@
|
||||
import { relative } from 'node:path';
|
||||
|
||||
import { test } from '@playwright/test';
|
||||
|
||||
import {
|
||||
electronMainPath,
|
||||
packagedRendererIndexPath,
|
||||
workspaceRoot,
|
||||
} from '../electron-test-fixtures';
|
||||
import {
|
||||
JOURNEY_SUMMARY_SCHEMA_VERSION,
|
||||
resolveJourneySummaryPath,
|
||||
summarizeJourneyIterations,
|
||||
writeJourneySummary,
|
||||
type JourneyIterationRecord,
|
||||
type JourneySummary,
|
||||
} from '../performance/journey-summary';
|
||||
import type { JourneyIterationRecord } from '../performance/journey-summary';
|
||||
import {
|
||||
LAUNCH_JOURNEY_ID,
|
||||
LAUNCH_JOURNEY_UNAVAILABLE_COUNTERS,
|
||||
toLaunchIterationRecord,
|
||||
} from '../performance/launch-journey-record';
|
||||
import {
|
||||
JOURNEY_ITERATION_TIMEOUT_MS,
|
||||
JOURNEY_MEASURED_ITERATIONS,
|
||||
JOURNEY_WARMUP_ITERATIONS,
|
||||
logJourneyIteration,
|
||||
writeJourneyRunEntry,
|
||||
} from './journey-run';
|
||||
import {
|
||||
LAUNCH_JOURNEY_MOCK_ORIGIN,
|
||||
measureLaunchJourney,
|
||||
@@ -32,25 +25,6 @@ import {
|
||||
* portal card is visible on /workspace with the inline splash removed.
|
||||
* Contract: docs/architecture/performance-journeys.md.
|
||||
*/
|
||||
const WARMUP_ITERATIONS = 1;
|
||||
const MEASURED_ITERATIONS = readPositiveInteger(
|
||||
'IPTVNATOR_JOURNEY_MEASURED_ITERATIONS',
|
||||
5
|
||||
);
|
||||
const ITERATION_TIMEOUT_MS = 120_000;
|
||||
|
||||
function readPositiveInteger(name: string, fallback: number): number {
|
||||
const raw = process.env[name];
|
||||
if (raw === undefined || raw === '') {
|
||||
return fallback;
|
||||
}
|
||||
const value = Number(raw);
|
||||
if (!Number.isSafeInteger(value) || value < 1) {
|
||||
throw new Error(`${name} must be a positive integer`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
test.describe.configure({ mode: 'serial' });
|
||||
|
||||
test('J1 launch to usable', async () => {
|
||||
@@ -60,62 +34,26 @@ test('J1 launch to usable', async () => {
|
||||
const iterations: JourneyIterationRecord[] = [];
|
||||
let electronVersion = 'unknown';
|
||||
try {
|
||||
const total = WARMUP_ITERATIONS + MEASURED_ITERATIONS;
|
||||
const total = JOURNEY_WARMUP_ITERATIONS + JOURNEY_MEASURED_ITERATIONS;
|
||||
for (let index = 0; index < total; index += 1) {
|
||||
const warmup = index < WARMUP_ITERATIONS;
|
||||
const warmup = index < JOURNEY_WARMUP_ITERATIONS;
|
||||
const measurement = await measureLaunchJourney(
|
||||
templateDirectory,
|
||||
ITERATION_TIMEOUT_MS
|
||||
JOURNEY_ITERATION_TIMEOUT_MS
|
||||
);
|
||||
electronVersion = measurement.electronVersion;
|
||||
const record = toLaunchIterationRecord(index, warmup, measurement);
|
||||
iterations.push(record);
|
||||
console.log(
|
||||
`[journey:launch] iteration ${index}${warmup ? ' (warm-up)' : ''} pid=${record.pid} ${JSON.stringify(
|
||||
{ ...record.counters, ...record.wallClock }
|
||||
)}`
|
||||
);
|
||||
logJourneyIteration(LAUNCH_JOURNEY_ID, record);
|
||||
}
|
||||
} finally {
|
||||
await removeLaunchJourneyProfile(templateDirectory);
|
||||
}
|
||||
|
||||
const entry = summarizeJourneyIterations(
|
||||
await writeJourneyRunEntry(
|
||||
LAUNCH_JOURNEY_ID,
|
||||
iterations,
|
||||
LAUNCH_JOURNEY_UNAVAILABLE_COUNTERS
|
||||
);
|
||||
const summary: JourneySummary = {
|
||||
generatedAt: new Date().toISOString(),
|
||||
harness: {
|
||||
arch: process.arch,
|
||||
ci: Boolean(process.env['CI']),
|
||||
electron: electronVersion,
|
||||
electronMain: relative(workspaceRoot, electronMainPath),
|
||||
measuredIterations: MEASURED_ITERATIONS,
|
||||
node: process.version,
|
||||
platform: process.platform,
|
||||
rendererIndex: relative(workspaceRoot, packagedRendererIndexPath),
|
||||
warmupIterations: WARMUP_ITERATIONS,
|
||||
},
|
||||
journeys: { [LAUNCH_JOURNEY_ID]: entry },
|
||||
schemaVersion: JOURNEY_SUMMARY_SCHEMA_VERSION,
|
||||
};
|
||||
const summaryPath = resolveJourneySummaryPath(workspaceRoot);
|
||||
await writeJourneySummary(summaryPath, summary);
|
||||
await test.info().attach('journey-summary', {
|
||||
contentType: 'application/json',
|
||||
path: summaryPath,
|
||||
});
|
||||
console.log(
|
||||
`[journey:launch] summary ${relative(workspaceRoot, summaryPath)}\n${JSON.stringify(
|
||||
{
|
||||
counters: entry.counters,
|
||||
counterStability: entry.counterStability,
|
||||
unavailable: entry.unavailable,
|
||||
wallClock: entry.wallClock,
|
||||
},
|
||||
null,
|
||||
2
|
||||
)}`
|
||||
LAUNCH_JOURNEY_UNAVAILABLE_COUNTERS,
|
||||
electronVersion
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,256 @@
|
||||
import type { ElectronApplication, Page } from '@playwright/test';
|
||||
|
||||
import { defaultXtreamPortalName } from '../electron-test-fixtures';
|
||||
import {
|
||||
countJourneyMainIpcInFlight,
|
||||
detachJourneyMainIpcCapture,
|
||||
installJourneyMainIpcCapture,
|
||||
JOURNEY_MAIN_IPC_STATE_KEY,
|
||||
JOURNEY_RENDERER_API_TRACE_CHANNEL,
|
||||
peekJourneyMainIpcCaptures,
|
||||
readJourneyMainIpcCapture,
|
||||
} from '../performance/journey-main-ipc-capture';
|
||||
import type { JourneyMockRequestLedger } from '../performance/journey-mock-request-ledger';
|
||||
import { waitForJourneyQuiet } from '../performance/journey-quiet-wait';
|
||||
import {
|
||||
armJourneyRendererProbe,
|
||||
createOpenSourceJourneyProbeOptions,
|
||||
waitForJourneyRendererProbe,
|
||||
type JourneyRendererProbeState,
|
||||
} from '../performance/journey-renderer-probe';
|
||||
import type {
|
||||
OpenSourceJourneyMeasurement,
|
||||
OpenSourceJourneySettle,
|
||||
} from '../performance/open-source-journey-record';
|
||||
import type { LaunchJourneySession } from './launch-journey-app';
|
||||
|
||||
/**
|
||||
* J2 "Open a source": runs inside a process that J1 has just launched, after
|
||||
* J1's counters are final. The app is first allowed to settle (no DOM
|
||||
* mutation, bridge call or mock request for `QUIET_MS`), so leftovers of the
|
||||
* startup are not attributed to the click. Then the Xtream portal card on
|
||||
* the dashboard is clicked and the probe, the IPC capture and the mock
|
||||
* request ledger measure until the category list and the first page of
|
||||
* items are painted.
|
||||
*/
|
||||
export const OPEN_SOURCE_JOURNEY_MAIN_IPC_STATE_KEY =
|
||||
'__iptvnatorJourneyOpenSourceMainIpcCapture';
|
||||
const QUIET_MS = 1_000;
|
||||
const POLL_MS = 100;
|
||||
const SETTLE_TIMEOUT_MS = 30_000;
|
||||
/**
|
||||
* After the mock settled, the ledger is watched this much longer before it
|
||||
* is read, so requests that arrive after the accepted quiet sample show up
|
||||
* in `httpRequestsAfterSettledByRoute` instead of vanishing unseen.
|
||||
*/
|
||||
const LATE_REQUEST_OBSERVATION_MS = QUIET_MS;
|
||||
|
||||
interface ActivitySample {
|
||||
readonly domMutations: number;
|
||||
readonly httpInFlight: number;
|
||||
readonly httpRequests: number;
|
||||
readonly ipcCalls: number;
|
||||
readonly ipcInFlight: number;
|
||||
}
|
||||
|
||||
async function readPreStartMutations(
|
||||
page: Page,
|
||||
stateKey: string
|
||||
): Promise<number> {
|
||||
return page.evaluate((key) => {
|
||||
const state = (globalThis as unknown as Record<string, unknown>)[
|
||||
key
|
||||
] as { preStart?: { domMutations?: number } } | undefined;
|
||||
const count = state?.preStart?.domMutations;
|
||||
if (typeof count !== 'number') {
|
||||
throw new Error('journey-renderer-probe-not-armed');
|
||||
}
|
||||
return count;
|
||||
}, stateKey);
|
||||
}
|
||||
|
||||
/**
|
||||
* Waits until DOM, bridge and mock traffic have all been unchanged for
|
||||
* `QUIET_MS` with no mock request and no bridge call in flight: a slow
|
||||
* response or a pending bridge call can still change the DOM or trigger
|
||||
* follow-up work after the click. Pending bridge calls come from J1's
|
||||
* capture, which was installed before the document loaded and so has seen
|
||||
* every call start. An app that never settles fails the iteration instead
|
||||
* of producing a count that includes its background work.
|
||||
*/
|
||||
async function waitForQuiet(
|
||||
electronApp: ElectronApplication,
|
||||
page: Page,
|
||||
ledger: JourneyMockRequestLedger,
|
||||
probeStateKey: string
|
||||
): Promise<{
|
||||
/** Ledger position read by the accepted quiet sample itself. */
|
||||
readonly ledgerMark: number;
|
||||
readonly settle: OpenSourceJourneySettle;
|
||||
}> {
|
||||
const armMark = ledger.mark();
|
||||
const sample = async (): Promise<ActivitySample> => {
|
||||
// Both captures in one snapshot: a call counted by J2's capture is
|
||||
// then also pending in J1's, never counted with a stale in-flight 0.
|
||||
const [launchCapture, openSourceCapture] =
|
||||
await peekJourneyMainIpcCaptures(electronApp, [
|
||||
JOURNEY_MAIN_IPC_STATE_KEY,
|
||||
OPEN_SOURCE_JOURNEY_MAIN_IPC_STATE_KEY,
|
||||
]);
|
||||
if (launchCapture.unmatchedCompletions > 0) {
|
||||
throw new Error('open-source-journey-bridge-completions-unmatched');
|
||||
}
|
||||
return {
|
||||
domMutations: await readPreStartMutations(page, probeStateKey),
|
||||
httpInFlight: ledger.inFlight(),
|
||||
httpRequests: ledger.mark(),
|
||||
ipcCalls: openSourceCapture.callsBeforeStart,
|
||||
ipcInFlight: countJourneyMainIpcInFlight(launchCapture),
|
||||
};
|
||||
};
|
||||
const { sample: quiet, waitedMs } = await waitForJourneyQuiet({
|
||||
inFlight: (activity) => activity.httpInFlight + activity.ipcInFlight,
|
||||
pollMs: POLL_MS,
|
||||
quietMs: QUIET_MS,
|
||||
sample,
|
||||
timeoutError: (activity) =>
|
||||
new Error(
|
||||
`open-source-journey-not-quiet: ${JSON.stringify(activity)}`
|
||||
),
|
||||
timeoutMs: SETTLE_TIMEOUT_MS,
|
||||
});
|
||||
return {
|
||||
ledgerMark: quiet.httpRequests,
|
||||
settle: {
|
||||
preStartDomMutations: quiet.domMutations,
|
||||
preStartHttpRequests: quiet.httpRequests - armMark,
|
||||
preStartIpcCalls: quiet.ipcCalls,
|
||||
quietMs: QUIET_MS,
|
||||
waitedMs,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Waits until the mock has seen no new request for `QUIET_MS` and none is
|
||||
* in flight, so responses slower than the quiet interval and the requests
|
||||
* they trigger stay inside the measured window.
|
||||
*/
|
||||
async function waitForMockQuiet(
|
||||
ledger: JourneyMockRequestLedger
|
||||
): Promise<number> {
|
||||
const { sample: quiet } = await waitForJourneyQuiet({
|
||||
inFlight: (activity) => activity.inFlight,
|
||||
pollMs: POLL_MS,
|
||||
quietMs: QUIET_MS,
|
||||
sample: async () => ({
|
||||
inFlight: ledger.inFlight(),
|
||||
requests: ledger.mark(),
|
||||
}),
|
||||
timeoutError: () => new Error('open-source-journey-mock-not-quiet'),
|
||||
timeoutMs: SETTLE_TIMEOUT_MS,
|
||||
});
|
||||
// The window ends at the ledger position this accepted sample read. A
|
||||
// request that arrives after it was never seen in flight, so its
|
||||
// response and follow-ups are not waited for; counting it would make
|
||||
// the counter depend on when the ledger is read.
|
||||
return quiet.requests;
|
||||
}
|
||||
|
||||
/**
|
||||
* `spawnLedgerMark` is the ledger position taken before the process was
|
||||
* spawned, so the launch's own mock traffic is kept as evidence.
|
||||
*/
|
||||
export async function measureOpenSourceJourney(
|
||||
session: LaunchJourneySession,
|
||||
ledger: JourneyMockRequestLedger,
|
||||
spawnLedgerMark: number,
|
||||
timeoutMs: number
|
||||
): Promise<OpenSourceJourneyMeasurement> {
|
||||
const { electronApp, mainWindow } = session;
|
||||
const probeOptions = createOpenSourceJourneyProbeOptions();
|
||||
const startClick = probeOptions.startClick;
|
||||
if (!startClick) {
|
||||
throw new Error('open-source-journey-probe-without-start');
|
||||
}
|
||||
await installJourneyMainIpcCapture(electronApp, {
|
||||
channel: JOURNEY_RENDERER_API_TRACE_CHANNEL,
|
||||
sentinelId: probeOptions.sentinelId,
|
||||
sentinelMethod: probeOptions.sentinelMethod,
|
||||
startSentinelId: startClick.sentinelId,
|
||||
stateKey: OPEN_SOURCE_JOURNEY_MAIN_IPC_STATE_KEY,
|
||||
});
|
||||
await armJourneyRendererProbe(mainWindow, probeOptions);
|
||||
const card = mainWindow
|
||||
.locator(startClick.selector)
|
||||
.filter({ hasText: defaultXtreamPortalName })
|
||||
.first();
|
||||
// Hover first so hover effects (and anything they trigger) happen
|
||||
// before the app settles, not inside the measured window.
|
||||
await card.hover({ timeout: timeoutMs });
|
||||
// The boundary for late requests is the ledger position the accepted
|
||||
// quiet sample read, like its DOM and IPC counts; a fresh mark taken
|
||||
// here would skip a request that arrived while that sample was still
|
||||
// reading the IPC capture. Requests from that position on but before
|
||||
// the renderer's click stamp arrived after the app settled, and the
|
||||
// record rejects such an iteration.
|
||||
const { ledgerMark: settledLedgerMark, settle } = await waitForQuiet(
|
||||
electronApp,
|
||||
mainWindow,
|
||||
ledger,
|
||||
probeOptions.stateKey
|
||||
);
|
||||
// J1's capture was only needed to see pending launch calls while
|
||||
// settling; detached, it no longer runs for every J2 bridge call.
|
||||
await detachJourneyMainIpcCapture(electronApp, JOURNEY_MAIN_IPC_STATE_KEY);
|
||||
await card.click({ timeout: timeoutMs });
|
||||
const renderer: JourneyRendererProbeState =
|
||||
await waitForJourneyRendererProbe(
|
||||
mainWindow,
|
||||
probeOptions.stateKey,
|
||||
timeoutMs
|
||||
);
|
||||
const ipc = await readJourneyMainIpcCapture(
|
||||
electronApp,
|
||||
OPEN_SOURCE_JOURNEY_MAIN_IPC_STATE_KEY,
|
||||
10_000
|
||||
);
|
||||
const settledAfterLedgerMark = await waitForMockQuiet(ledger);
|
||||
await new Promise((resolve) =>
|
||||
setTimeout(resolve, LATE_REQUEST_OBSERVATION_MS)
|
||||
);
|
||||
// The journey starts at the renderer's click stamp, not when Playwright
|
||||
// began its actionability checks, so a request that arrives in between
|
||||
// stays before the click like it does for every other J2 counter. The
|
||||
// ledger's clock is this process's wall clock and the stamp is the
|
||||
// renderer's; both read the same host clock.
|
||||
const clickEpochMs = renderer.start?.epochMs;
|
||||
if (clickEpochMs === undefined) {
|
||||
throw new Error('open-source-journey-click-not-started');
|
||||
}
|
||||
const sinceSpawn = ledger.since(spawnLedgerMark);
|
||||
const beforeClick = sinceSpawn.filter(
|
||||
(entry) => entry.epochMs < clickEpochMs
|
||||
);
|
||||
const afterClick = sinceSpawn.filter(
|
||||
(entry) => entry.epochMs >= clickEpochMs
|
||||
);
|
||||
return {
|
||||
http: {
|
||||
afterSettleBeforeClick: beforeClick.filter(
|
||||
(entry) => entry.sequence >= settledLedgerMark
|
||||
).length,
|
||||
afterSettled: afterClick.filter(
|
||||
(entry) => entry.sequence >= settledAfterLedgerMark
|
||||
),
|
||||
beforeClick,
|
||||
requests: afterClick.filter(
|
||||
(entry) => entry.sequence < settledAfterLedgerMark
|
||||
),
|
||||
},
|
||||
ipc,
|
||||
pid: session.launch.pid,
|
||||
renderer,
|
||||
settle,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
import { test } from '@playwright/test';
|
||||
|
||||
import { startJourneyMockRequestLedger } from '../performance/journey-mock-request-ledger';
|
||||
import type { JourneyIterationRecord } from '../performance/journey-summary';
|
||||
import {
|
||||
OPEN_SOURCE_JOURNEY_ID,
|
||||
OPEN_SOURCE_JOURNEY_UNAVAILABLE_COUNTERS,
|
||||
toOpenSourceIterationRecord,
|
||||
} from '../performance/open-source-journey-record';
|
||||
import {
|
||||
JOURNEY_ITERATION_TIMEOUT_MS,
|
||||
JOURNEY_MEASURED_ITERATIONS,
|
||||
JOURNEY_WARMUP_ITERATIONS,
|
||||
logJourneyIteration,
|
||||
writeJourneyRunEntry,
|
||||
} from './journey-run';
|
||||
import {
|
||||
LAUNCH_JOURNEY_MOCK_ORIGIN,
|
||||
removeLaunchJourneyProfile,
|
||||
runLaunchJourney,
|
||||
seedLaunchJourneyProfile,
|
||||
} from './launch-journey-app';
|
||||
import { measureOpenSourceJourney } from './open-source-journey-app';
|
||||
|
||||
/**
|
||||
* J2 "Open a source": click on the Xtream portal card on the dashboard until
|
||||
* the section's category list and first page of items are painted. Every
|
||||
* iteration is a fresh J1 launch on a copy of the seeded profile; the click
|
||||
* happens after J1's terminal condition and after the app has settled.
|
||||
* The profile is seeded through the mock request ledger, so every request
|
||||
* the app sends to the mock is counted.
|
||||
* Contract: docs/architecture/performance-journeys.md.
|
||||
*/
|
||||
test.describe.configure({ mode: 'serial' });
|
||||
|
||||
test('J2 open a source', async () => {
|
||||
const ledger = await startJourneyMockRequestLedger(
|
||||
LAUNCH_JOURNEY_MOCK_ORIGIN
|
||||
);
|
||||
const iterations: JourneyIterationRecord[] = [];
|
||||
let electronVersion = 'unknown';
|
||||
try {
|
||||
const templateDirectory = await seedLaunchJourneyProfile(ledger.origin);
|
||||
try {
|
||||
const total =
|
||||
JOURNEY_WARMUP_ITERATIONS + JOURNEY_MEASURED_ITERATIONS;
|
||||
for (let index = 0; index < total; index += 1) {
|
||||
const warmup = index < JOURNEY_WARMUP_ITERATIONS;
|
||||
const spawnLedgerMark = ledger.mark();
|
||||
const { continuation, launch } = await runLaunchJourney(
|
||||
templateDirectory,
|
||||
JOURNEY_ITERATION_TIMEOUT_MS,
|
||||
// J2 does not read J1's main-process counters, so their
|
||||
// SQL instrumentation stays off during the click.
|
||||
{ mainCounters: false },
|
||||
(session) =>
|
||||
measureOpenSourceJourney(
|
||||
session,
|
||||
ledger,
|
||||
spawnLedgerMark,
|
||||
JOURNEY_ITERATION_TIMEOUT_MS
|
||||
)
|
||||
);
|
||||
electronVersion = launch.electronVersion;
|
||||
const record = toOpenSourceIterationRecord(
|
||||
index,
|
||||
warmup,
|
||||
continuation
|
||||
);
|
||||
iterations.push(record);
|
||||
logJourneyIteration(OPEN_SOURCE_JOURNEY_ID, record);
|
||||
}
|
||||
} finally {
|
||||
await removeLaunchJourneyProfile(templateDirectory);
|
||||
}
|
||||
} finally {
|
||||
await ledger.close();
|
||||
}
|
||||
|
||||
await writeJourneyRunEntry(
|
||||
OPEN_SOURCE_JOURNEY_ID,
|
||||
iterations,
|
||||
OPEN_SOURCE_JOURNEY_UNAVAILABLE_COUNTERS,
|
||||
electronVersion
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,18 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import { journeyLaunchEnvironment } from './journey-launch-environment';
|
||||
|
||||
test('J1 launches with the main-process counters and SQL counting', () => {
|
||||
assert.deepEqual(journeyLaunchEnvironment({ mainCounters: true }), {
|
||||
IPTVNATOR_PERF_CAPTURE: '1',
|
||||
IPTVNATOR_PERF_COUNT_SQL: '1',
|
||||
IPTVNATOR_TRACE_IPC: '1',
|
||||
});
|
||||
});
|
||||
|
||||
test('a journey that does not read the main counters launches with the IPC trace only', () => {
|
||||
assert.deepEqual(journeyLaunchEnvironment({ mainCounters: false }), {
|
||||
IPTVNATOR_TRACE_IPC: '1',
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,26 @@
|
||||
/**
|
||||
* Instrumentation flags a journey launch sets on the Electron process.
|
||||
*
|
||||
* Every journey needs the renderer-API trace (`IPTVNATOR_TRACE_IPC`) for its
|
||||
* IPC counters. Only J1 records the main-process counters:
|
||||
* `IPTVNATOR_PERF_CAPTURE` turns on the counters and their read handler, and
|
||||
* `IPTVNATOR_PERF_COUNT_SQL` wraps every main-thread and worker SQLite
|
||||
* statement to count it (see journey-main-counters.ts). A journey that
|
||||
* continues from the launch without reading them (J2) leaves both off, so
|
||||
* its latency and workload are not measured under that extra
|
||||
* instrumentation.
|
||||
*/
|
||||
export interface JourneyLaunchInstrumentation {
|
||||
readonly mainCounters: boolean;
|
||||
}
|
||||
|
||||
export function journeyLaunchEnvironment(
|
||||
instrumentation: JourneyLaunchInstrumentation
|
||||
): Record<string, string> {
|
||||
return {
|
||||
...(instrumentation.mainCounters
|
||||
? { IPTVNATOR_PERF_CAPTURE: '1', IPTVNATOR_PERF_COUNT_SQL: '1' }
|
||||
: {}),
|
||||
IPTVNATOR_TRACE_IPC: '1',
|
||||
};
|
||||
}
|
||||
@@ -136,12 +136,36 @@ test('only the launch journey opts into SQL statement counting', () => {
|
||||
.filter(
|
||||
(file) => /\.(ts|cjs)$/.test(file) && !/\.spec\.ts$/.test(file)
|
||||
);
|
||||
const optedIn = files
|
||||
.filter((file) =>
|
||||
readFileSync(join(sourceRoot, file), 'utf8').includes(
|
||||
'IPTVNATOR_PERF_COUNT_SQL'
|
||||
const containing = (needle: RegExp) =>
|
||||
files
|
||||
.filter((file) =>
|
||||
needle.test(readFileSync(join(sourceRoot, file), 'utf8'))
|
||||
)
|
||||
)
|
||||
.map((file) => relative(sourceRoot, join(sourceRoot, file)));
|
||||
assert.deepEqual(optedIn, [join('journeys', 'launch-journey-app.ts')]);
|
||||
.map((file) => relative(sourceRoot, join(sourceRoot, file)));
|
||||
// The journey launch builds its flags in one place...
|
||||
assert.deepEqual(containing(/IPTVNATOR_PERF_COUNT_SQL/), [
|
||||
join('performance', 'journey-launch-environment.ts'),
|
||||
]);
|
||||
// ...and only J1's launch asks for them there. Another journey that
|
||||
// passed `mainCounters: true` to runLaunchJourney would be measured
|
||||
// under the statement hook without recording its count.
|
||||
assert.deepEqual(containing(/mainCounters:\s*true/), [
|
||||
join('journeys', 'launch-journey-app.ts'),
|
||||
]);
|
||||
const launchApp = readFileSync(
|
||||
join(sourceRoot, 'journeys', 'launch-journey-app.ts'),
|
||||
'utf8'
|
||||
);
|
||||
assert.equal(launchApp.match(/mainCounters:\s*true/g)?.length, 1);
|
||||
assert.match(
|
||||
launchApp,
|
||||
/export async function measureLaunchJourney\([\s\S]*?\{ mainCounters: true \}[\s\S]*?\n\}/
|
||||
);
|
||||
assert.match(
|
||||
readFileSync(
|
||||
join(sourceRoot, 'journeys', 'open-source.journey.ts'),
|
||||
'utf8'
|
||||
),
|
||||
/\{ mainCounters: false \}/
|
||||
);
|
||||
});
|
||||
@@ -1,14 +1,27 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { EventEmitter } from 'node:events';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { resolve } from 'node:path';
|
||||
import test from 'node:test';
|
||||
|
||||
import type { ElectronApplication } from '@playwright/test';
|
||||
|
||||
import {
|
||||
assertJourneyMainIpcCapture,
|
||||
countJourneyMainIpcInFlight,
|
||||
detachJourneyMainIpcCapture,
|
||||
installJourneyMainIpcCapture,
|
||||
JOURNEY_RENDERER_API_TRACE_CHANNEL,
|
||||
peekJourneyMainIpcCapture,
|
||||
peekJourneyMainIpcCaptures,
|
||||
type JourneyMainIpcCaptureOptions,
|
||||
type JourneyMainIpcCaptureState,
|
||||
} from './journey-main-ipc-capture';
|
||||
import { JOURNEY_IPC_SENTINEL_METHOD } from './journey-renderer-probe';
|
||||
import {
|
||||
JOURNEY_IPC_SENTINEL_METHOD,
|
||||
JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
|
||||
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
|
||||
} from './journey-renderer-probe';
|
||||
|
||||
const electronBackendSource = resolve(
|
||||
__dirname,
|
||||
@@ -20,13 +33,17 @@ function validCapture(
|
||||
): JourneyMainIpcCaptureState {
|
||||
return {
|
||||
callsAfterSentinel: 2,
|
||||
callsBeforeStart: 0,
|
||||
callsBeforeSentinel: 7,
|
||||
callsByMethod: { dbGetAppPlaylists: 1, getSettings: 6 },
|
||||
inFlightByMethod: {},
|
||||
installedEpochMs: 1,
|
||||
malformedEvents: 0,
|
||||
processStartEpochMs: 0,
|
||||
senderIds: [1],
|
||||
sentinel: { occurrences: 1, receivedEpochMs: 2 },
|
||||
unmatchedCompletions: 0,
|
||||
start: null,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
@@ -57,7 +74,9 @@ test('the preload traces every bridge invocation on that channel when IPC tracin
|
||||
assert.match(preload, /name\.startsWith\('remove'\)/);
|
||||
assert.match(
|
||||
preload,
|
||||
new RegExp(`${JOURNEY_IPC_SENTINEL_METHOD}: \\(playlistId: string`)
|
||||
new RegExp(
|
||||
`${JOURNEY_IPC_SENTINEL_METHOD}: \\(id: string\\) =>\\s+ipcRenderer\\.invoke\\('SOURCE_HEALTH_CANCEL', id\\)`
|
||||
)
|
||||
);
|
||||
const debugTrace = readFileSync(
|
||||
resolve(electronBackendSource, 'services/debug-trace.ts'),
|
||||
@@ -66,6 +85,19 @@ test('the preload traces every bridge invocation on that channel when IPC tracin
|
||||
assert.match(debugTrace, /readFlag\('IPTVNATOR_TRACE_IPC'\)/);
|
||||
});
|
||||
|
||||
test('the sentinel method is a no-op for an unknown id in the main process', () => {
|
||||
// The markers must not queue work ahead of the measured journey: the
|
||||
// handler only aborts a probe registered under that id, if any.
|
||||
const control = readFileSync(
|
||||
resolve(electronBackendSource, 'events/source-probe-control.ts'),
|
||||
'utf8'
|
||||
);
|
||||
assert.match(
|
||||
control,
|
||||
/ipcMain\.handle\(SOURCE_HEALTH_CANCEL, \(event, requestId: string\) => \{\s+requests\.get\(`\$\{event\.sender\.id\}:\$\{requestId\}`\)\?\.abort\(\);\s+\}\);/
|
||||
);
|
||||
});
|
||||
|
||||
test('accepts a capture with exactly one sentinel from one renderer', () => {
|
||||
assert.equal(
|
||||
assertJourneyMainIpcCapture(validCapture()).callsBeforeSentinel,
|
||||
@@ -102,3 +134,278 @@ test('rejects captures that cannot bound the counter exactly', () => {
|
||||
/malformed/
|
||||
);
|
||||
});
|
||||
|
||||
/**
|
||||
* Runs the capture's main-process function in this process against a fake
|
||||
* `ipcMain`, the way `electronApp.evaluate` runs it in Electron.
|
||||
*/
|
||||
function createFakeElectronApp(): {
|
||||
readonly app: ElectronApplication;
|
||||
listeners(): number;
|
||||
send(
|
||||
senderId: number,
|
||||
method: string,
|
||||
args: unknown[],
|
||||
phase?: string
|
||||
): void;
|
||||
} {
|
||||
const ipcMain = new EventEmitter();
|
||||
let channel = '';
|
||||
const app = {
|
||||
evaluate: async (
|
||||
fn: (electron: unknown, arg: unknown) => unknown,
|
||||
arg: unknown
|
||||
) => {
|
||||
channel =
|
||||
(arg as Partial<JourneyMainIpcCaptureOptions>).channel ??
|
||||
channel;
|
||||
return fn({ ipcMain }, arg);
|
||||
},
|
||||
} as unknown as ElectronApplication;
|
||||
return {
|
||||
app,
|
||||
listeners: () => ipcMain.listenerCount(channel),
|
||||
send: (senderId, method, args, phase = 'start') => {
|
||||
ipcMain.emit(
|
||||
channel,
|
||||
{ sender: { id: senderId } },
|
||||
{ args, method, phase }
|
||||
);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function withCapture(
|
||||
options: Partial<JourneyMainIpcCaptureOptions>,
|
||||
run: (
|
||||
fake: ReturnType<typeof createFakeElectronApp>,
|
||||
read: () => Promise<JourneyMainIpcCaptureState>,
|
||||
stateKey: string
|
||||
) => Promise<void>
|
||||
): Promise<void> {
|
||||
const stateKey = `__journeyIpcCaptureTest${Math.random()}`;
|
||||
const fake = createFakeElectronApp();
|
||||
try {
|
||||
await installJourneyMainIpcCapture(fake.app, {
|
||||
channel: JOURNEY_RENDERER_API_TRACE_CHANNEL,
|
||||
sentinelId: JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
|
||||
sentinelMethod: JOURNEY_IPC_SENTINEL_METHOD,
|
||||
stateKey,
|
||||
...options,
|
||||
});
|
||||
await run(
|
||||
fake,
|
||||
() => peekJourneyMainIpcCapture(fake.app, stateKey),
|
||||
stateKey
|
||||
);
|
||||
} finally {
|
||||
const target = globalThis as unknown as Record<string, unknown>;
|
||||
delete target[stateKey];
|
||||
delete target[`${stateKey}:detach`];
|
||||
}
|
||||
}
|
||||
|
||||
test('without a start marker, counts every call from install to the sentinel', async () => {
|
||||
await withCapture({}, async (fake, read) => {
|
||||
fake.send(1, 'getSettings', []);
|
||||
fake.send(1, 'getSettings', [], 'end');
|
||||
fake.send(1, 'onSomething', []);
|
||||
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [
|
||||
JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
|
||||
]);
|
||||
fake.send(1, 'dbGetAppPlaylists', []);
|
||||
const state = assertJourneyMainIpcCapture(await read());
|
||||
assert.equal(state.start, null);
|
||||
assert.equal(state.callsBeforeStart, 0);
|
||||
assert.equal(state.callsBeforeSentinel, 2);
|
||||
assert.equal(state.callsAfterSentinel, 1);
|
||||
assert.deepEqual(state.callsByMethod, {
|
||||
getSettings: 1,
|
||||
onSomething: 1,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
test('with a start marker, counts only the calls between the two sentinels', async () => {
|
||||
await withCapture(
|
||||
{ startSentinelId: JOURNEY_OPEN_SOURCE_START_SENTINEL_ID },
|
||||
async (fake, read) => {
|
||||
fake.send(1, 'getSettings', []);
|
||||
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, ['other-playlist']);
|
||||
assert.equal((await read()).callsBeforeStart, 2);
|
||||
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [
|
||||
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
|
||||
]);
|
||||
fake.send(1, 'dbGetAppPlaylist', ['playlist-1']);
|
||||
fake.send(1, 'xtreamRequest', [{ action: 'get_account_info' }]);
|
||||
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [
|
||||
JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
|
||||
]);
|
||||
fake.send(1, 'getSettings', []);
|
||||
const state = assertJourneyMainIpcCapture(await read());
|
||||
assert.equal(state.callsBeforeStart, 2);
|
||||
assert.equal(state.callsBeforeSentinel, 2);
|
||||
assert.equal(state.callsAfterSentinel, 1);
|
||||
assert.deepEqual(state.callsByMethod, {
|
||||
dbGetAppPlaylist: 1,
|
||||
xtreamRequest: 1,
|
||||
});
|
||||
assert.equal(state.start?.occurrences, 1);
|
||||
}
|
||||
);
|
||||
});
|
||||
|
||||
test('rejects a start marker that is missing, repeated or after the sentinel', async () => {
|
||||
await withCapture(
|
||||
{ startSentinelId: JOURNEY_OPEN_SOURCE_START_SENTINEL_ID },
|
||||
async (fake, read) => {
|
||||
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [
|
||||
JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
|
||||
]);
|
||||
const early = await read();
|
||||
const state = () => early;
|
||||
assert.throws(
|
||||
() => assertJourneyMainIpcCapture(state()),
|
||||
/start-count-0/
|
||||
);
|
||||
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [
|
||||
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
|
||||
]);
|
||||
const late = await read();
|
||||
assert.equal(late.start?.occurrences, 1);
|
||||
assert.throws(
|
||||
() => assertJourneyMainIpcCapture(late),
|
||||
/sentinel-before-start/
|
||||
);
|
||||
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [
|
||||
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
|
||||
]);
|
||||
const repeated = await read();
|
||||
assert.throws(
|
||||
() => assertJourneyMainIpcCapture(repeated),
|
||||
/start-count-2/
|
||||
);
|
||||
}
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
assertJourneyMainIpcCapture(
|
||||
validCapture({
|
||||
start: { occurrences: 1, receivedEpochMs: 3 },
|
||||
})
|
||||
),
|
||||
/sentinel-before-start/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
assertJourneyMainIpcCapture(
|
||||
validCapture({
|
||||
start: { occurrences: 1, receivedEpochMs: null },
|
||||
})
|
||||
),
|
||||
/sentinel-before-start/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
assertJourneyMainIpcCapture(
|
||||
validCapture({
|
||||
start: { occurrences: 2, receivedEpochMs: 1 },
|
||||
})
|
||||
),
|
||||
/start-count-2/
|
||||
);
|
||||
assert.equal(
|
||||
assertJourneyMainIpcCapture(
|
||||
validCapture({ start: { occurrences: 1, receivedEpochMs: 1 } })
|
||||
).callsBeforeSentinel,
|
||||
7
|
||||
);
|
||||
});
|
||||
|
||||
test('tracks bridge calls in flight from start to success or error', async () => {
|
||||
await withCapture({}, async (fake, read) => {
|
||||
fake.send(1, 'getSettings', []);
|
||||
fake.send(1, 'getSettings', []);
|
||||
fake.send(1, 'xtreamRequest', [{ action: 'get_account_info' }]);
|
||||
let state = await read();
|
||||
assert.deepEqual(state.inFlightByMethod, {
|
||||
getSettings: 2,
|
||||
xtreamRequest: 1,
|
||||
});
|
||||
assert.equal(countJourneyMainIpcInFlight(state), 3);
|
||||
fake.send(1, 'getSettings', [], 'success');
|
||||
fake.send(1, 'xtreamRequest', [], 'error');
|
||||
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [
|
||||
JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
|
||||
]);
|
||||
state = await read();
|
||||
assert.deepEqual(state.inFlightByMethod, {
|
||||
[JOURNEY_IPC_SENTINEL_METHOD]: 1,
|
||||
getSettings: 1,
|
||||
});
|
||||
fake.send(1, JOURNEY_IPC_SENTINEL_METHOD, [], 'success');
|
||||
fake.send(1, 'getSettings', [], 'success');
|
||||
state = await read();
|
||||
assert.equal(countJourneyMainIpcInFlight(state), 0);
|
||||
assert.equal(state.unmatchedCompletions, 0);
|
||||
// A completion whose start this capture never saw.
|
||||
fake.send(1, 'dbGetAppState', [], 'success');
|
||||
state = await read();
|
||||
assert.equal(state.unmatchedCompletions, 1);
|
||||
assert.equal(countJourneyMainIpcInFlight(state), 0);
|
||||
});
|
||||
});
|
||||
|
||||
test('a detached capture stops listening and keeps its last state', async () => {
|
||||
await withCapture({}, async (fake, read, stateKey) => {
|
||||
fake.send(1, 'getSettings', []);
|
||||
assert.equal(fake.listeners(), 1);
|
||||
await detachJourneyMainIpcCapture(fake.app, stateKey);
|
||||
assert.equal(fake.listeners(), 0);
|
||||
fake.send(1, 'getSettings', []);
|
||||
fake.send(1, 'dbGetAppState', []);
|
||||
const state = await read();
|
||||
assert.equal(state.callsBeforeSentinel, 1);
|
||||
assert.deepEqual(state.inFlightByMethod, { getSettings: 1 });
|
||||
await assert.rejects(
|
||||
detachJourneyMainIpcCapture(fake.app, stateKey),
|
||||
/not-attached/
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('reads several captures in one main-process pass', async () => {
|
||||
const fake = createFakeElectronApp();
|
||||
let evaluations = 0;
|
||||
const counting = {
|
||||
evaluate: (...args: Parameters<ElectronApplication['evaluate']>) => {
|
||||
evaluations += 1;
|
||||
return fake.app.evaluate(...args);
|
||||
},
|
||||
} as unknown as ElectronApplication;
|
||||
const target = globalThis as unknown as Record<string, unknown>;
|
||||
const [first, second] = ['__journeyPeekA', '__journeyPeekB'];
|
||||
target[first] = { callsBeforeStart: 1 };
|
||||
target[second] = { callsBeforeStart: 2 };
|
||||
try {
|
||||
const states = await peekJourneyMainIpcCaptures(counting, [
|
||||
first,
|
||||
second,
|
||||
]);
|
||||
assert.equal(evaluations, 1);
|
||||
assert.deepEqual(
|
||||
states.map((state) => state.callsBeforeStart),
|
||||
[1, 2]
|
||||
);
|
||||
await assert.rejects(
|
||||
peekJourneyMainIpcCaptures(counting, [
|
||||
first,
|
||||
'__journeyPeekMissing',
|
||||
]),
|
||||
/capture-missing/
|
||||
);
|
||||
} finally {
|
||||
delete target[first];
|
||||
delete target[second];
|
||||
}
|
||||
});
|
||||
@@ -11,6 +11,11 @@ import type { ElectronApplication } from '@playwright/test';
|
||||
* until the renderer probe's sentinel call arrives. Renderer-to-main IPC is
|
||||
* delivered in order, so every call started before the sentinel is counted
|
||||
* and nothing after it is.
|
||||
*
|
||||
* A journey that starts inside a running app (J2 "Open a source") passes
|
||||
* `startSentinelId`: the probe sends that id before the app handles the
|
||||
* start click, calls before it are only tallied in `callsBeforeStart`, and
|
||||
* `callsBeforeSentinel` then counts the calls between the two sentinels.
|
||||
*/
|
||||
|
||||
/** Literal of `DEBUG_TRACE_EVENT_CHANNEL` in `services/debug-trace.ts`. */
|
||||
@@ -21,21 +26,39 @@ export interface JourneyMainIpcCaptureOptions {
|
||||
readonly channel: string;
|
||||
readonly sentinelId: string;
|
||||
readonly sentinelMethod: string;
|
||||
/** Start marker; absent: counting starts when the capture is installed. */
|
||||
readonly startSentinelId?: string;
|
||||
readonly stateKey: string;
|
||||
}
|
||||
|
||||
export interface JourneyMainIpcSentinelState {
|
||||
readonly occurrences: number;
|
||||
readonly receivedEpochMs: number | null;
|
||||
}
|
||||
|
||||
export interface JourneyMainIpcCaptureState {
|
||||
readonly callsAfterSentinel: number;
|
||||
/** Calls before the start marker; always 0 without one. */
|
||||
readonly callsBeforeStart: number;
|
||||
/** Calls from the start marker (or install) up to the sentinel. */
|
||||
readonly callsBeforeSentinel: number;
|
||||
readonly callsByMethod: Record<string, number>;
|
||||
/**
|
||||
* Bridge calls started but not yet completed, per method. The preload
|
||||
* follows every `start` with exactly one `success` or `error` (sync and
|
||||
* async results alike), so a capture installed before the document
|
||||
* loads sees every pair.
|
||||
*/
|
||||
readonly inFlightByMethod: Record<string, number>;
|
||||
readonly installedEpochMs: number;
|
||||
readonly malformedEvents: number;
|
||||
readonly processStartEpochMs: number;
|
||||
readonly senderIds: number[];
|
||||
readonly sentinel: {
|
||||
readonly occurrences: number;
|
||||
readonly receivedEpochMs: number | null;
|
||||
};
|
||||
readonly sentinel: JourneyMainIpcSentinelState;
|
||||
/** Completions without a start seen by this capture (installed late). */
|
||||
readonly unmatchedCompletions: number;
|
||||
/** Null when the capture has no start marker. */
|
||||
readonly start: JourneyMainIpcSentinelState | null;
|
||||
}
|
||||
|
||||
export async function installJourneyMainIpcCapture(
|
||||
@@ -47,21 +70,42 @@ export async function installJourneyMainIpcCapture(
|
||||
if (target[input.stateKey] !== undefined) {
|
||||
throw new Error('journey-main-ipc-capture-already-installed');
|
||||
}
|
||||
const startSentinelId = input.startSentinelId ?? null;
|
||||
const state = {
|
||||
callsAfterSentinel: 0,
|
||||
callsBeforeStart: 0,
|
||||
callsBeforeSentinel: 0,
|
||||
callsByMethod: {} as Record<string, number>,
|
||||
installedEpochMs: Date.now(),
|
||||
malformedEvents: 0,
|
||||
processStartEpochMs: Date.now() - process.uptime() * 1000,
|
||||
inFlightByMethod: {} as Record<string, number>,
|
||||
senderIds: [] as number[],
|
||||
sentinel: {
|
||||
occurrences: 0,
|
||||
receivedEpochMs: null as number | null,
|
||||
},
|
||||
unmatchedCompletions: 0,
|
||||
start:
|
||||
startSentinelId === null
|
||||
? null
|
||||
: {
|
||||
occurrences: 0,
|
||||
receivedEpochMs: null as number | null,
|
||||
},
|
||||
};
|
||||
const carries = (args: unknown, id: string): boolean => {
|
||||
try {
|
||||
return JSON.stringify(args ?? null).includes(id);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
target[input.stateKey] = state;
|
||||
ipcMain.on(input.channel, (event, payload: unknown) => {
|
||||
const listener = (
|
||||
event: { sender: { id: number } },
|
||||
payload: unknown
|
||||
): void => {
|
||||
const record =
|
||||
typeof payload === 'object' && payload !== null
|
||||
? (payload as Record<string, unknown>)
|
||||
@@ -70,25 +114,45 @@ export async function installJourneyMainIpcCapture(
|
||||
state.malformedEvents += 1;
|
||||
return;
|
||||
}
|
||||
if (record['phase'] !== 'start') {
|
||||
const phase = record['phase'];
|
||||
if (phase === 'success' || phase === 'error') {
|
||||
const pending = state.inFlightByMethod[record['method']] ?? 0;
|
||||
if (pending === 0) {
|
||||
state.unmatchedCompletions += 1;
|
||||
} else if (pending === 1) {
|
||||
delete state.inFlightByMethod[record['method']];
|
||||
} else {
|
||||
state.inFlightByMethod[record['method']] = pending - 1;
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (phase !== 'start') {
|
||||
return;
|
||||
}
|
||||
// Sentinels included: their completions arrive like any other.
|
||||
state.inFlightByMethod[record['method']] =
|
||||
(state.inFlightByMethod[record['method']] ?? 0) + 1;
|
||||
const senderId = event.sender.id;
|
||||
if (!state.senderIds.includes(senderId)) {
|
||||
state.senderIds.push(senderId);
|
||||
}
|
||||
const method = record['method'];
|
||||
let isSentinel = false;
|
||||
if (method === input.sentinelMethod) {
|
||||
try {
|
||||
isSentinel = JSON.stringify(
|
||||
record['args'] ?? null
|
||||
).includes(input.sentinelId);
|
||||
} catch {
|
||||
isSentinel = false;
|
||||
const isMarker = method === input.sentinelMethod;
|
||||
if (
|
||||
isMarker &&
|
||||
state.start !== null &&
|
||||
startSentinelId !== null &&
|
||||
carries(record['args'], startSentinelId)
|
||||
) {
|
||||
state.start.occurrences += 1;
|
||||
// A start marker after the sentinel stays unstamped, which
|
||||
// the assertion rejects.
|
||||
if (state.sentinel.receivedEpochMs === null) {
|
||||
state.start.receivedEpochMs ??= Date.now();
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (isSentinel) {
|
||||
if (isMarker && carries(record['args'], input.sentinelId)) {
|
||||
state.sentinel.occurrences += 1;
|
||||
state.sentinel.receivedEpochMs ??= Date.now();
|
||||
return;
|
||||
@@ -97,13 +161,91 @@ export async function installJourneyMainIpcCapture(
|
||||
state.callsAfterSentinel += 1;
|
||||
return;
|
||||
}
|
||||
if (state.start !== null && state.start.receivedEpochMs === null) {
|
||||
state.callsBeforeStart += 1;
|
||||
return;
|
||||
}
|
||||
state.callsBeforeSentinel += 1;
|
||||
state.callsByMethod[method] =
|
||||
(state.callsByMethod[method] ?? 0) + 1;
|
||||
});
|
||||
};
|
||||
ipcMain.on(input.channel, listener);
|
||||
// Kept next to the state (which is read as JSON) so the capture can
|
||||
// be detached from the same main process later.
|
||||
target[`${input.stateKey}:detach`] = () => {
|
||||
ipcMain.removeListener(input.channel, listener);
|
||||
};
|
||||
}, options);
|
||||
}
|
||||
|
||||
/**
|
||||
* Removes a capture's listener. J2 detaches J1's capture once it has used it
|
||||
* to settle, so the launch listener does not run for every bridge call of
|
||||
* the measured click.
|
||||
*/
|
||||
export async function detachJourneyMainIpcCapture(
|
||||
electronApp: ElectronApplication,
|
||||
stateKey: string
|
||||
): Promise<void> {
|
||||
await electronApp.evaluate((_electron, key) => {
|
||||
const target = globalThis as unknown as Record<string, unknown>;
|
||||
const detach = target[`${key}:detach`];
|
||||
if (typeof detach !== 'function') {
|
||||
throw new Error('journey-main-ipc-capture-not-attached');
|
||||
}
|
||||
(detach as () => void)();
|
||||
delete target[`${key}:detach`];
|
||||
}, stateKey);
|
||||
}
|
||||
|
||||
/** Total of `inFlightByMethod`. */
|
||||
export function countJourneyMainIpcInFlight(
|
||||
state: JourneyMainIpcCaptureState
|
||||
): number {
|
||||
return Object.values(state.inFlightByMethod).reduce(
|
||||
(total, count) => total + count,
|
||||
0
|
||||
);
|
||||
}
|
||||
|
||||
/** Raw state without waiting for the sentinel, for settling checks. */
|
||||
export async function peekJourneyMainIpcCapture(
|
||||
electronApp: ElectronApplication,
|
||||
stateKey: string
|
||||
): Promise<JourneyMainIpcCaptureState> {
|
||||
const [state] = await peekJourneyMainIpcCaptures(electronApp, [stateKey]);
|
||||
return state as JourneyMainIpcCaptureState;
|
||||
}
|
||||
|
||||
/**
|
||||
* Several captures read in one synchronous pass in the main process. No
|
||||
* `ipcMain` event can be handled in between, so the states are one coherent
|
||||
* snapshot: a call counted by one capture is also pending in the other.
|
||||
*/
|
||||
export async function peekJourneyMainIpcCaptures(
|
||||
electronApp: ElectronApplication,
|
||||
stateKeys: readonly string[]
|
||||
): Promise<JourneyMainIpcCaptureState[]> {
|
||||
const states = (await electronApp.evaluate(
|
||||
(_electron, keys) =>
|
||||
JSON.parse(
|
||||
JSON.stringify(
|
||||
keys.map(
|
||||
(key) =>
|
||||
(globalThis as unknown as Record<string, unknown>)[
|
||||
key
|
||||
] ?? null
|
||||
)
|
||||
)
|
||||
) as unknown,
|
||||
[...stateKeys]
|
||||
)) as (JourneyMainIpcCaptureState | null)[];
|
||||
if (states.some((state) => !state)) {
|
||||
throw new Error('journey-main-ipc-capture-missing');
|
||||
}
|
||||
return states as JourneyMainIpcCaptureState[];
|
||||
}
|
||||
|
||||
export async function readJourneyMainIpcCapture(
|
||||
electronApp: ElectronApplication,
|
||||
stateKey: string,
|
||||
@@ -151,5 +293,19 @@ export function assertJourneyMainIpcCapture(
|
||||
if (state.malformedEvents > 0) {
|
||||
throw new Error('journey-main-ipc-capture-malformed-events');
|
||||
}
|
||||
if (state.start !== null) {
|
||||
if (state.start.occurrences !== 1) {
|
||||
throw new Error(
|
||||
`journey-main-ipc-capture-start-count-${state.start.occurrences}`
|
||||
);
|
||||
}
|
||||
if (
|
||||
state.start.receivedEpochMs === null ||
|
||||
state.sentinel.receivedEpochMs === null ||
|
||||
state.start.receivedEpochMs > state.sentinel.receivedEpochMs
|
||||
) {
|
||||
throw new Error('journey-main-ipc-capture-sentinel-before-start');
|
||||
}
|
||||
}
|
||||
return state;
|
||||
}
|
||||
@@ -0,0 +1,172 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { createServer, type Server } from 'node:http';
|
||||
import type { AddressInfo } from 'node:net';
|
||||
import test from 'node:test';
|
||||
|
||||
import {
|
||||
countJourneyMockRoutes,
|
||||
describeJourneyMockRoute,
|
||||
startJourneyMockRequestLedger,
|
||||
type JourneyMockRequestLedger,
|
||||
} from './journey-mock-request-ledger';
|
||||
|
||||
test('routes keep the Xtream action but never the credentials', () => {
|
||||
assert.equal(
|
||||
describeJourneyMockRoute(
|
||||
'/player_api.php?username=user1&password=pass1&action=get_live_streams&category_id=3'
|
||||
),
|
||||
'/player_api.php?action=get_live_streams'
|
||||
);
|
||||
assert.equal(
|
||||
describeJourneyMockRoute(
|
||||
'/player_api.php?username=user1&password=pass1'
|
||||
),
|
||||
'/player_api.php'
|
||||
);
|
||||
assert.equal(
|
||||
describeJourneyMockRoute('/live/user1/pass1/1001.m3u8'),
|
||||
'/live/:username/:password/1001.m3u8'
|
||||
);
|
||||
assert.equal(
|
||||
describeJourneyMockRoute('/movie/user1/pass1/2001.mp4?token=x'),
|
||||
'/movie/:username/:password/2001.mp4'
|
||||
);
|
||||
assert.equal(describeJourneyMockRoute('/playlist.m3u'), '/playlist.m3u');
|
||||
assert.equal(describeJourneyMockRoute(undefined), '/');
|
||||
});
|
||||
|
||||
test('counts requests per route in a stable order', () => {
|
||||
assert.deepEqual(
|
||||
countJourneyMockRoutes([
|
||||
{ epochMs: 1, method: 'GET', route: '/b', sequence: 0 },
|
||||
{ epochMs: 2, method: 'GET', route: '/a', sequence: 1 },
|
||||
{ epochMs: 3, method: 'GET', route: '/b', sequence: 2 },
|
||||
]),
|
||||
{ '/a': 1, '/b': 2 }
|
||||
);
|
||||
});
|
||||
|
||||
interface Upstream {
|
||||
readonly origin: string;
|
||||
/** What the upstream received; kept server-side, never echoed back. */
|
||||
readonly received: { host: string; method: string; url: string }[];
|
||||
readonly server: Server;
|
||||
}
|
||||
|
||||
async function startUpstream(): Promise<Upstream> {
|
||||
const received: Upstream['received'] = [];
|
||||
const server = createServer((request, response) => {
|
||||
received.push({
|
||||
host: request.headers.host ?? '',
|
||||
method: request.method ?? '',
|
||||
url: request.url ?? '',
|
||||
});
|
||||
response.setHeader('content-type', 'text/plain');
|
||||
response.end('upstream');
|
||||
});
|
||||
await new Promise<void>((resolve) =>
|
||||
server.listen(0, '127.0.0.1', () => resolve())
|
||||
);
|
||||
const { port } = server.address() as AddressInfo;
|
||||
return { origin: `http://127.0.0.1:${port}`, received, server };
|
||||
}
|
||||
|
||||
test('forwards every request to the mock and records it from a mark', async () => {
|
||||
const upstream = await startUpstream();
|
||||
const ledger = await startJourneyMockRequestLedger(upstream.origin);
|
||||
try {
|
||||
assert.notEqual(ledger.origin, upstream.origin);
|
||||
const first = await fetch(`${ledger.origin}/playlist.m3u`);
|
||||
assert.equal(await first.text(), 'upstream');
|
||||
assert.deepEqual(upstream.received, [
|
||||
{
|
||||
host: new URL(upstream.origin).host,
|
||||
method: 'GET',
|
||||
url: '/playlist.m3u',
|
||||
},
|
||||
]);
|
||||
const mark = ledger.mark();
|
||||
assert.equal(mark, 1);
|
||||
const beforeSecond = performance.timeOrigin + performance.now();
|
||||
const second = await fetch(
|
||||
`${ledger.origin}/player_api.php?username=u&password=p&action=get_account_info`
|
||||
);
|
||||
assert.equal(await second.text(), 'upstream');
|
||||
assert.equal(
|
||||
upstream.received[1]?.url,
|
||||
'/player_api.php?username=u&password=p&action=get_account_info'
|
||||
);
|
||||
const since = ledger.since(mark);
|
||||
assert.equal(since.length, 1);
|
||||
assert.equal(
|
||||
since[0]?.route,
|
||||
'/player_api.php?action=get_account_info'
|
||||
);
|
||||
assert.equal(since[0]?.method, 'GET');
|
||||
assert.equal(since[0]?.sequence, 1);
|
||||
// Stamped on the same high-resolution epoch as the renderer's click.
|
||||
const stamped = since[0]?.epochMs ?? 0;
|
||||
assert.ok(stamped >= beforeSecond);
|
||||
assert.ok(stamped <= performance.timeOrigin + performance.now());
|
||||
assert.ok(!JSON.stringify(ledger.since(0)).includes('password'));
|
||||
} finally {
|
||||
await ledger.close();
|
||||
await new Promise((resolve) => upstream.server.close(resolve));
|
||||
}
|
||||
});
|
||||
|
||||
/** `close` on the proxied response can fire just after the client has read it. */
|
||||
async function waitForIdle(ledger: JourneyMockRequestLedger): Promise<void> {
|
||||
for (let attempt = 0; attempt < 50 && ledger.inFlight() > 0; attempt++) {
|
||||
await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
}
|
||||
}
|
||||
|
||||
test('reports a request as in flight until its response has finished', async () => {
|
||||
let release: () => void = () => undefined;
|
||||
const server = createServer((_request, response) => {
|
||||
response.setHeader('content-type', 'text/plain');
|
||||
response.write('partial');
|
||||
release = () => response.end('done');
|
||||
});
|
||||
await new Promise<void>((resolve) =>
|
||||
server.listen(0, '127.0.0.1', () => resolve())
|
||||
);
|
||||
const { port } = server.address() as AddressInfo;
|
||||
const ledger = await startJourneyMockRequestLedger(
|
||||
`http://127.0.0.1:${port}`
|
||||
);
|
||||
try {
|
||||
assert.equal(ledger.inFlight(), 0);
|
||||
const response = await fetch(`${ledger.origin}/player_api.php`);
|
||||
// Headers arrived; the body is still open upstream.
|
||||
assert.equal(ledger.mark(), 1);
|
||||
assert.equal(ledger.inFlight(), 1);
|
||||
release();
|
||||
assert.equal(await response.text(), 'partialdone');
|
||||
await waitForIdle(ledger);
|
||||
assert.equal(ledger.inFlight(), 0);
|
||||
} finally {
|
||||
await ledger.close();
|
||||
await new Promise((resolve) => server.close(resolve));
|
||||
}
|
||||
});
|
||||
|
||||
test('answers 502 when the mock is gone and refuses non-HTTP targets', async () => {
|
||||
const upstream = await startUpstream();
|
||||
await new Promise((resolve) => upstream.server.close(resolve));
|
||||
const ledger = await startJourneyMockRequestLedger(upstream.origin);
|
||||
try {
|
||||
const response = await fetch(`${ledger.origin}/health`);
|
||||
assert.equal(response.status, 502);
|
||||
assert.equal(ledger.mark(), 1);
|
||||
await waitForIdle(ledger);
|
||||
assert.equal(ledger.inFlight(), 0);
|
||||
} finally {
|
||||
await ledger.close();
|
||||
}
|
||||
await assert.rejects(
|
||||
startJourneyMockRequestLedger('https://127.0.0.1:1'),
|
||||
/http-only/
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,149 @@
|
||||
import {
|
||||
createServer,
|
||||
request as httpRequest,
|
||||
type IncomingMessage,
|
||||
type Server,
|
||||
type ServerResponse,
|
||||
} from 'node:http';
|
||||
import type { AddressInfo } from 'node:net';
|
||||
|
||||
/**
|
||||
* Counts the HTTP requests the app sends to the Xtream mock.
|
||||
*
|
||||
* The journey profile is seeded with the origin of this loopback proxy
|
||||
* instead of the mock's own origin, so every request for the portal and the
|
||||
* M3U source passes through it, whether the main process sends it (Xtream
|
||||
* API, M3U) or the renderer does (artwork served by the mock). The mock's
|
||||
* `/__control/state` ledger only exists in performance-control mode, which
|
||||
* disables `/playlist.m3u` and tracks only the 100k scenario, and a
|
||||
* Playwright request listener sees renderer traffic only.
|
||||
*
|
||||
* The ledger keeps the method, the path and, for `player_api.php`, the
|
||||
* `action` parameter. Query strings carry the portal credentials and are
|
||||
* never stored.
|
||||
*/
|
||||
export interface JourneyMockRequest {
|
||||
/**
|
||||
* Arrival as a sub-millisecond epoch (`performance.timeOrigin +
|
||||
* performance.now()`), the same form as the renderer's click stamp. With
|
||||
* `Date.now()` a request later in the click's millisecond would compare
|
||||
* as earlier than the fractional click and be counted before it.
|
||||
*/
|
||||
readonly epochMs: number;
|
||||
readonly method: string;
|
||||
readonly route: string;
|
||||
readonly sequence: number;
|
||||
}
|
||||
|
||||
export interface JourneyMockRequestLedger {
|
||||
close(): Promise<void>;
|
||||
/** Requests whose response has not finished yet. */
|
||||
inFlight(): number;
|
||||
/** Sequence number of the next request; pass it to `since`. */
|
||||
mark(): number;
|
||||
readonly origin: string;
|
||||
since(mark: number): readonly JourneyMockRequest[];
|
||||
}
|
||||
|
||||
/** `GET /player_api.php?action=get_live_streams&...` → `/player_api.php?action=get_live_streams`. */
|
||||
export function describeJourneyMockRoute(rawUrl: string | undefined): string {
|
||||
const url = new URL(rawUrl ?? '/', 'http://journey.invalid');
|
||||
const action = url.searchParams.get('action');
|
||||
if (url.pathname.endsWith('/player_api.php')) {
|
||||
return action === null
|
||||
? url.pathname
|
||||
: `${url.pathname}?action=${encodeURIComponent(action)}`;
|
||||
}
|
||||
// Stream paths embed the credentials: /live/<user>/<password>/<id>.ts.
|
||||
const segments = url.pathname.split('/');
|
||||
if (
|
||||
['live', 'movie', 'series', 'timeshift'].includes(segments[1] ?? '') &&
|
||||
segments.length > 3
|
||||
) {
|
||||
segments[2] = ':username';
|
||||
segments[3] = ':password';
|
||||
}
|
||||
return segments.join('/');
|
||||
}
|
||||
|
||||
/** Route → count, sorted by route, for evidence. */
|
||||
export function countJourneyMockRoutes(
|
||||
requests: readonly JourneyMockRequest[]
|
||||
): Record<string, number> {
|
||||
const counts = new Map<string, number>();
|
||||
for (const entry of requests) {
|
||||
counts.set(entry.route, (counts.get(entry.route) ?? 0) + 1);
|
||||
}
|
||||
return Object.fromEntries(
|
||||
[...counts.entries()].sort(([left], [right]) =>
|
||||
left.localeCompare(right)
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
export async function startJourneyMockRequestLedger(
|
||||
targetOrigin: string
|
||||
): Promise<JourneyMockRequestLedger> {
|
||||
const target = new URL(targetOrigin);
|
||||
if (target.protocol !== 'http:') {
|
||||
throw new Error('journey-mock-ledger-http-only');
|
||||
}
|
||||
const requests: JourneyMockRequest[] = [];
|
||||
let active = 0;
|
||||
const forward = (
|
||||
incoming: IncomingMessage,
|
||||
outgoing: ServerResponse
|
||||
): void => {
|
||||
active += 1;
|
||||
// `close` fires once per response, finished or aborted.
|
||||
outgoing.once('close', () => {
|
||||
active -= 1;
|
||||
});
|
||||
requests.push({
|
||||
epochMs: performance.timeOrigin + performance.now(),
|
||||
method: incoming.method ?? 'GET',
|
||||
route: describeJourneyMockRoute(incoming.url),
|
||||
sequence: requests.length,
|
||||
});
|
||||
const upstream = httpRequest(
|
||||
{
|
||||
headers: { ...incoming.headers, host: target.host },
|
||||
hostname: target.hostname,
|
||||
method: incoming.method,
|
||||
path: incoming.url,
|
||||
port: target.port,
|
||||
},
|
||||
(response) => {
|
||||
outgoing.writeHead(
|
||||
response.statusCode ?? 502,
|
||||
response.headers
|
||||
);
|
||||
response.pipe(outgoing);
|
||||
}
|
||||
);
|
||||
upstream.on('error', () => {
|
||||
if (!outgoing.headersSent) {
|
||||
outgoing.writeHead(502);
|
||||
}
|
||||
outgoing.end();
|
||||
});
|
||||
incoming.pipe(upstream);
|
||||
};
|
||||
const server: Server = createServer(forward);
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
server.once('error', reject);
|
||||
server.listen(0, '127.0.0.1', () => resolve());
|
||||
});
|
||||
const { port } = server.address() as AddressInfo;
|
||||
return {
|
||||
close: () =>
|
||||
new Promise<void>((resolve) => {
|
||||
server.closeAllConnections();
|
||||
server.close(() => resolve());
|
||||
}),
|
||||
inFlight: () => active,
|
||||
mark: () => requests.length,
|
||||
origin: `http://127.0.0.1:${port}`,
|
||||
since: (mark) => requests.slice(mark),
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import { waitForJourneyQuiet } from './journey-quiet-wait';
|
||||
|
||||
interface Activity {
|
||||
readonly inFlight: number;
|
||||
readonly requests: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs the waiter on a fake clock: each poll advances time by `pollMs`, and
|
||||
* `script(ms)` says what the app looks like at that time.
|
||||
*/
|
||||
async function run(
|
||||
script: (atMs: number) => Activity,
|
||||
quietMs = 1_000,
|
||||
timeoutMs = 30_000
|
||||
) {
|
||||
let clock = 0;
|
||||
const polls: number[] = [];
|
||||
const result = await waitForJourneyQuiet<Activity>({
|
||||
inFlight: (activity) => activity.inFlight,
|
||||
now: () => clock,
|
||||
pollMs: 100,
|
||||
quietMs,
|
||||
sample: async () => {
|
||||
polls.push(clock);
|
||||
return script(clock);
|
||||
},
|
||||
sleep: async (ms) => {
|
||||
clock += ms;
|
||||
},
|
||||
timeoutError: (activity) =>
|
||||
new Error(`not-quiet ${JSON.stringify(activity)}`),
|
||||
timeoutMs,
|
||||
});
|
||||
return { ...result, polls };
|
||||
}
|
||||
|
||||
test('returns once nothing changed for the whole quiet period', async () => {
|
||||
const result = await run(() => ({ inFlight: 0, requests: 3 }));
|
||||
assert.equal(result.waitedMs, 1_000);
|
||||
assert.deepEqual(result.sample, { inFlight: 0, requests: 3 });
|
||||
});
|
||||
|
||||
test('restarts the quiet period at every new request', async () => {
|
||||
const result = await run((atMs) => ({
|
||||
inFlight: 0,
|
||||
requests: atMs < 500 ? 1 : 2,
|
||||
}));
|
||||
// The second request is first seen at 500 ms.
|
||||
assert.equal(result.waitedMs, 1_500);
|
||||
});
|
||||
|
||||
test('counts the quiet period from the poll that saw pending work complete', async () => {
|
||||
// One request is in flight until 750 ms; the count never changes. The
|
||||
// poll at 800 ms is the first to see it done, so the app is quiet only
|
||||
// from 800 ms on, not from the 700 ms poll at which it was pending.
|
||||
const result = await run((atMs) => ({
|
||||
inFlight: atMs < 750 ? 1 : 0,
|
||||
requests: 1,
|
||||
}));
|
||||
assert.equal(result.waitedMs, 1_800);
|
||||
});
|
||||
|
||||
test('never returns while work stays in flight and fails at the timeout', async () => {
|
||||
await assert.rejects(
|
||||
run(() => ({ inFlight: 1, requests: 1 }), 1_000, 3_000),
|
||||
/not-quiet \{"inFlight":1,"requests":1\}/
|
||||
);
|
||||
});
|
||||
|
||||
test('fails when a sample stalls past the deadline instead of accepting it as quiet', async () => {
|
||||
let clock = 0;
|
||||
let samples = 0;
|
||||
await assert.rejects(
|
||||
waitForJourneyQuiet<Activity>({
|
||||
inFlight: (activity) => activity.inFlight,
|
||||
now: () => clock,
|
||||
pollMs: 100,
|
||||
quietMs: 1_000,
|
||||
sample: async () => {
|
||||
samples += 1;
|
||||
// The second sample hangs for 5 s (e.g. behind a busy main
|
||||
// process) and then reports nothing changed.
|
||||
if (samples === 2) clock += 5_000;
|
||||
return { inFlight: 0, requests: 1 };
|
||||
},
|
||||
sleep: async (ms) => {
|
||||
clock += ms;
|
||||
},
|
||||
timeoutError: () => new Error('not-quiet-stalled'),
|
||||
timeoutMs: 3_000,
|
||||
}),
|
||||
/not-quiet-stalled/
|
||||
);
|
||||
assert.equal(samples, 2);
|
||||
});
|
||||
@@ -0,0 +1,57 @@
|
||||
/**
|
||||
* Polls activity until nothing has changed for `quietMs` and nothing is in
|
||||
* flight. Used by the click-started journeys to settle the app before the
|
||||
* click and to close the mock request window after the terminal.
|
||||
*
|
||||
* A sample is compared as a whole, in-flight counts included, so the poll
|
||||
* that first sees pending work complete restarts the quiet period: the
|
||||
* window is always at least `quietMs` after the last observed activity,
|
||||
* never measured from a poll at which work was still pending.
|
||||
*/
|
||||
export interface JourneyQuietWaitOptions<T> {
|
||||
/** Requests or calls still pending in a sample; any makes it busy. */
|
||||
readonly inFlight: (sample: T) => number;
|
||||
readonly now?: () => number;
|
||||
readonly pollMs: number;
|
||||
readonly quietMs: number;
|
||||
readonly sample: () => Promise<T>;
|
||||
readonly sleep?: (ms: number) => Promise<void>;
|
||||
readonly timeoutError: (last: T) => Error;
|
||||
readonly timeoutMs: number;
|
||||
}
|
||||
|
||||
export interface JourneyQuietWaitResult<T> {
|
||||
readonly sample: T;
|
||||
readonly waitedMs: number;
|
||||
}
|
||||
|
||||
export async function waitForJourneyQuiet<T>(
|
||||
options: JourneyQuietWaitOptions<T>
|
||||
): Promise<JourneyQuietWaitResult<T>> {
|
||||
const now = options.now ?? Date.now;
|
||||
const sleep =
|
||||
options.sleep ??
|
||||
((ms: number) =>
|
||||
new Promise<void>((resolve) => setTimeout(resolve, ms)));
|
||||
const startedMs = now();
|
||||
let lastKey = JSON.stringify(await options.sample());
|
||||
let quietSinceMs = now();
|
||||
for (;;) {
|
||||
await sleep(options.pollMs);
|
||||
const next = await options.sample();
|
||||
const nextKey = JSON.stringify(next);
|
||||
const sampledMs = now();
|
||||
// The deadline is checked first: a sample that stalled past it (for
|
||||
// example behind a busy main process) must fail the wait, not be
|
||||
// accepted as the end of a quiet period nobody observed.
|
||||
if (sampledMs - startedMs > options.timeoutMs) {
|
||||
throw options.timeoutError(next);
|
||||
}
|
||||
if (nextKey !== lastKey || options.inFlight(next) > 0) {
|
||||
lastKey = nextKey;
|
||||
quietSinceMs = sampledMs;
|
||||
} else if (sampledMs - quietSinceMs >= options.quietMs) {
|
||||
return { sample: next, waitedMs: sampledMs - startedMs };
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -6,8 +6,12 @@ import { JSDOM } from 'jsdom';
|
||||
import {
|
||||
assertJourneyRendererProbeState,
|
||||
createLaunchJourneyProbeOptions,
|
||||
createOpenSourceJourneyProbeOptions,
|
||||
JOURNEY_IPC_SENTINEL_ID,
|
||||
JOURNEY_IPC_SENTINEL_METHOD,
|
||||
JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
|
||||
JOURNEY_OPEN_SOURCE_PROBE_STATE_KEY,
|
||||
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
|
||||
JOURNEY_PROBE_STATE_KEY,
|
||||
journeyRendererProbeScript,
|
||||
type JourneyRendererProbeOptions,
|
||||
@@ -108,6 +112,18 @@ function createFixture(
|
||||
runScripts: 'outside-only',
|
||||
url,
|
||||
});
|
||||
return createFixtureFromDom(
|
||||
dom,
|
||||
{ ...createLaunchJourneyProbeOptions(), ...optionOverrides },
|
||||
bridge
|
||||
);
|
||||
}
|
||||
|
||||
function createFixtureFromDom(
|
||||
dom: JSDOM,
|
||||
options: JourneyRendererProbeOptions,
|
||||
bridge: boolean
|
||||
): Fixture {
|
||||
const { window } = dom;
|
||||
const observers: FakeObserver[] = [];
|
||||
const bridgeCalls: unknown[] = [];
|
||||
@@ -132,10 +148,6 @@ function createFixture(
|
||||
}),
|
||||
});
|
||||
}
|
||||
const options = {
|
||||
...createLaunchJourneyProbeOptions(),
|
||||
...optionOverrides,
|
||||
};
|
||||
window.eval(
|
||||
`(${journeyRendererProbeScript.toString()})(${JSON.stringify(options)})`
|
||||
);
|
||||
@@ -143,6 +155,7 @@ function createFixture(
|
||||
(window as unknown as Record<string, JourneyRendererProbeState>)[
|
||||
options.stateKey
|
||||
] as JourneyRendererProbeState;
|
||||
liveStates.push(rawState);
|
||||
return {
|
||||
bridgeCalls,
|
||||
observers,
|
||||
@@ -153,8 +166,26 @@ function createFixture(
|
||||
};
|
||||
}
|
||||
|
||||
function settle(ms = 40): Promise<void> {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
/** Probes created by this file, so `settle` can wait for their cutoff. */
|
||||
const liveStates: (() => JourneyRendererProbeState)[] = [];
|
||||
|
||||
/**
|
||||
* Waits `ms`, then until every probe that reached its terminal batch has also
|
||||
* passed the post-paint cutoff (a rAF plus a timer). A fixed delay alone
|
||||
* flakes when the harness runs all spec files in parallel.
|
||||
*/
|
||||
async function settle(ms = 40): Promise<void> {
|
||||
await new Promise((resolve) => setTimeout(resolve, ms));
|
||||
const deadline = Date.now() + 2_000;
|
||||
while (
|
||||
liveStates.some((read) => {
|
||||
const state = read();
|
||||
return state.terminal !== null && !state.final;
|
||||
}) &&
|
||||
Date.now() < deadline
|
||||
) {
|
||||
await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
}
|
||||
}
|
||||
|
||||
function renderFirstCard(fixture: Fixture): void {
|
||||
@@ -401,9 +432,286 @@ test('rejects a probe whose performance observers were unavailable instead of re
|
||||
test('launch options target the workspace source cards and the shared sentinel', () => {
|
||||
const options = createLaunchJourneyProbeOptions();
|
||||
assert.equal(options.stateKey, JOURNEY_PROBE_STATE_KEY);
|
||||
assert.equal(options.sentinelMethod, 'dbGetAppPlaylist');
|
||||
assert.equal(options.sentinelMethod, 'cancelSourceProbe');
|
||||
assert.equal(options.splashId, 'initial-splash');
|
||||
assert.equal(options.routeFragment, '/workspace');
|
||||
assert.match(options.cardSelector, /dashboard-recent-sources-rail-card/);
|
||||
assert.match(options.cardSelector, /app-playlist-item/);
|
||||
});
|
||||
|
||||
// J2 "Open a source": the probe is armed in a loaded document and starts at
|
||||
// the click on the portal card.
|
||||
|
||||
const DASHBOARD_URL = 'http://localhost/workspace/dashboard';
|
||||
|
||||
function createOpenSourceFixture(
|
||||
overrides: { bridge?: boolean } = {}
|
||||
): Fixture & { readonly card: HTMLElement } {
|
||||
const dom = new JSDOM(
|
||||
`<!doctype html><html><body><app-root>
|
||||
<app-workspace-context-panel></app-workspace-context-panel>
|
||||
<main></main></app-root></body></html>`,
|
||||
{
|
||||
pretendToBeVisual: true,
|
||||
runScripts: 'outside-only',
|
||||
url: DASHBOARD_URL,
|
||||
}
|
||||
);
|
||||
const card = dom.window.document.createElement('div');
|
||||
card.setAttribute('data-test-id', 'dashboard-recent-sources-rail-card');
|
||||
card.innerHTML = '<a><span class="title">Mock Xtream Portal</span></a>';
|
||||
dom.window.document.querySelector('main')?.append(card);
|
||||
const fixture = createFixtureFromDom(
|
||||
dom,
|
||||
createOpenSourceJourneyProbeOptions(),
|
||||
overrides.bridge ?? true
|
||||
);
|
||||
return { ...fixture, card };
|
||||
}
|
||||
|
||||
function openSource(
|
||||
fixture: Fixture,
|
||||
parts: { categories?: boolean; items?: boolean } = {}
|
||||
): void {
|
||||
const { categories = true, items = true } = parts;
|
||||
const { document, history } = fixture.window;
|
||||
history.pushState({}, '', '/workspace/xtreams/playlist-1/vod');
|
||||
document.querySelector('main')?.replaceChildren();
|
||||
if (categories) {
|
||||
const category = document.createElement('div');
|
||||
category.className = 'category-item';
|
||||
document.querySelector('app-workspace-context-panel')?.append(category);
|
||||
}
|
||||
if (items) {
|
||||
const grid = document.createElement('app-grid-list');
|
||||
grid.append(
|
||||
document.createElement('mat-card'),
|
||||
document.createElement('mat-card')
|
||||
);
|
||||
document.querySelector('main')?.append(grid);
|
||||
}
|
||||
}
|
||||
|
||||
test('a click-started probe only tracks activity before the click inside the start selector', async () => {
|
||||
const fixture = createOpenSourceFixture();
|
||||
const { document } = fixture.window;
|
||||
assert.deepEqual(fixture.state().invalidReasons, []);
|
||||
assert.equal(fixture.state().start, null);
|
||||
|
||||
document.body.append(document.createElement('div'));
|
||||
await settle();
|
||||
document.body.click();
|
||||
await settle();
|
||||
const before = fixture.state();
|
||||
assert.equal(before.start, null);
|
||||
assert.equal(before.counters.domMutations, 0);
|
||||
assert.equal(before.preStart.domMutations, 1);
|
||||
assert.equal(typeof before.preStart.lastMutationEpochMs, 'number');
|
||||
assert.deepEqual(fixture.bridgeCalls, []);
|
||||
// Already on a page with cards and categories: nothing ends before the
|
||||
// start.
|
||||
assert.equal(before.terminal, null);
|
||||
});
|
||||
|
||||
test('the click sends the start sentinel before the app sees it and the end sentinel when the first page is visible', async () => {
|
||||
const fixture = createOpenSourceFixture();
|
||||
const { document } = fixture.window;
|
||||
const order: string[] = [];
|
||||
fixture.card.addEventListener('click', () => {
|
||||
order.push(`app:${fixture.bridgeCalls.length}`);
|
||||
openSource(fixture);
|
||||
});
|
||||
(fixture.card.querySelector('.title') as HTMLElement).click();
|
||||
await settle();
|
||||
const state = fixture.state();
|
||||
assert.deepEqual(order, ['app:1']);
|
||||
assert.deepEqual(fixture.bridgeCalls, [
|
||||
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
|
||||
JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
|
||||
]);
|
||||
assert.ok(state.start, 'start must be recorded');
|
||||
assert.equal(state.start.sentinelStatus, 'sent');
|
||||
assert.equal(
|
||||
state.start.targetTestId,
|
||||
'dashboard-recent-sources-rail-card'
|
||||
);
|
||||
assert.equal(state.start.pathname, '/workspace/dashboard');
|
||||
assert.ok(state.start.epochMs <= state.start.listenerEpochMs);
|
||||
assert.ok(state.terminal, 'terminal must be recorded');
|
||||
assert.equal(state.terminal.pathname, '/workspace/xtreams/playlist-1/vod');
|
||||
assert.equal(state.terminal.cardTag, 'mat-card');
|
||||
assert.equal(state.terminal.cardCount, 2);
|
||||
assert.deepEqual(state.terminal.companionCounts, [1]);
|
||||
// main emptied (1) + category (1) + grid (1)
|
||||
assert.equal(state.counters.domMutations, 3);
|
||||
assert.equal(state.navigation, null);
|
||||
assert.equal(state.final, true);
|
||||
assert.ok(state.terminal.epochMs >= state.start.epochMs);
|
||||
assert.doesNotThrow(() => assertJourneyRendererProbeState(state));
|
||||
|
||||
// One start per armed probe.
|
||||
document.body.append(document.createElement('div'));
|
||||
fixture.card.click();
|
||||
await settle();
|
||||
assert.equal(fixture.bridgeCalls.length, 2);
|
||||
});
|
||||
|
||||
test('the first page needs the category list as well as the items', async () => {
|
||||
const fixture = createOpenSourceFixture();
|
||||
fixture.card.addEventListener('click', () =>
|
||||
openSource(fixture, { categories: false })
|
||||
);
|
||||
fixture.card.click();
|
||||
await settle();
|
||||
assert.equal(fixture.state().terminal, null);
|
||||
const category = fixture.window.document.createElement('div');
|
||||
category.className = 'category-item';
|
||||
fixture.window.document
|
||||
.querySelector('app-workspace-context-panel')
|
||||
?.append(category);
|
||||
await settle();
|
||||
const state = fixture.state();
|
||||
assert.ok(state.terminal);
|
||||
assert.equal(state.counters.domMutations, 3);
|
||||
|
||||
const skeletons = createOpenSourceFixture();
|
||||
skeletons.card.addEventListener('click', () => {
|
||||
openSource(skeletons, { items: false });
|
||||
const skeleton = skeletons.window.document.createElement('div');
|
||||
skeleton.className = 'grid-skeleton-card';
|
||||
skeletons.window.document.querySelector('main')?.append(skeleton);
|
||||
});
|
||||
skeletons.card.click();
|
||||
await settle();
|
||||
assert.equal(skeletons.state().terminal, null);
|
||||
});
|
||||
|
||||
test('drops performance entries from before the click and keeps recent-input shifts apart', async () => {
|
||||
const fixture = createOpenSourceFixture();
|
||||
const [layoutShift, longTask] = fixture.observers as [
|
||||
FakeObserver,
|
||||
FakeObserver,
|
||||
];
|
||||
const now = () => fixture.window.performance.now();
|
||||
const beforeClick = now() - 1;
|
||||
layoutShift.emit([
|
||||
{
|
||||
entryType: 'layout-shift',
|
||||
hadRecentInput: false,
|
||||
startTime: beforeClick,
|
||||
value: 3,
|
||||
},
|
||||
]);
|
||||
longTask.emit([
|
||||
{ duration: 400, entryType: 'longtask', startTime: beforeClick },
|
||||
]);
|
||||
fixture.card.click();
|
||||
await settle(5);
|
||||
// Delivered after the click but started before it (buffered J1 entries).
|
||||
layoutShift.emit([
|
||||
{
|
||||
entryType: 'layout-shift',
|
||||
hadRecentInput: false,
|
||||
startTime: beforeClick,
|
||||
value: 2,
|
||||
},
|
||||
{
|
||||
entryType: 'layout-shift',
|
||||
hadRecentInput: true,
|
||||
startTime: now(),
|
||||
value: 0.25,
|
||||
},
|
||||
{
|
||||
entryType: 'layout-shift',
|
||||
hadRecentInput: false,
|
||||
startTime: now(),
|
||||
value: 0.125,
|
||||
},
|
||||
]);
|
||||
longTask.emit([
|
||||
// A buffered J1 task that ended before the click.
|
||||
{
|
||||
duration: 250,
|
||||
entryType: 'longtask',
|
||||
startTime: beforeClick - 300,
|
||||
},
|
||||
{ duration: 90, entryType: 'longtask', startTime: now() },
|
||||
]);
|
||||
openSource(fixture);
|
||||
await settle();
|
||||
const state = fixture.state();
|
||||
assert.equal(state.final, true);
|
||||
assert.equal(state.counters.layoutShiftScore, 0.125);
|
||||
assert.equal(state.counters.recentInputLayoutShiftScore, 0.25);
|
||||
assert.equal(state.counters.longTasks, 1);
|
||||
assert.deepEqual(state.longTaskDurationsMs, [90]);
|
||||
});
|
||||
|
||||
test('counts the long task that dispatches the click although it began before the event', async () => {
|
||||
const fixture = createOpenSourceFixture();
|
||||
const [, longTask] = fixture.observers as [FakeObserver, FakeObserver];
|
||||
const now = () => fixture.window.performance.now();
|
||||
fixture.card.addEventListener('click', () => openSource(fixture));
|
||||
fixture.card.click();
|
||||
const clickMs =
|
||||
(fixture.rawState().start?.epochMs ?? 0) -
|
||||
fixture.window.performance.timeOrigin;
|
||||
longTask.emit([
|
||||
// Began 20 ms before the click stamp and ran through it: the task
|
||||
// that dispatched the click and rendered the page.
|
||||
{ duration: 120, entryType: 'longtask', startTime: clickMs - 20 },
|
||||
// Ended before the click: earlier work, not part of the journey.
|
||||
{ duration: 60, entryType: 'longtask', startTime: clickMs - 100 },
|
||||
{ duration: 70, entryType: 'longtask', startTime: now() },
|
||||
]);
|
||||
await settle();
|
||||
const state = fixture.state();
|
||||
assert.equal(state.final, true);
|
||||
assert.equal(state.counters.longTasks, 2);
|
||||
assert.deepEqual(state.longTaskDurationsMs, [120, 70]);
|
||||
});
|
||||
|
||||
test('rejects a click start whose sentinel could not be sent', async () => {
|
||||
const fixture = createOpenSourceFixture({ bridge: false });
|
||||
fixture.card.addEventListener('click', () => openSource(fixture));
|
||||
fixture.card.click();
|
||||
await settle();
|
||||
const state = fixture.state();
|
||||
assert.equal(state.start?.sentinelStatus, 'bridge-missing');
|
||||
assert.throws(
|
||||
() => assertJourneyRendererProbeState(state),
|
||||
/sentinel-bridge-missing/
|
||||
);
|
||||
const started = {
|
||||
...state,
|
||||
sentinel: { epochMs: 1, status: 'sent' as const },
|
||||
};
|
||||
assert.throws(
|
||||
() => assertJourneyRendererProbeState(started),
|
||||
/start-sentinel-bridge-missing/
|
||||
);
|
||||
});
|
||||
|
||||
test('open-source options start at the portal card and end on the source route', () => {
|
||||
const options = createOpenSourceJourneyProbeOptions();
|
||||
assert.equal(options.journey, 'open-source');
|
||||
assert.equal(options.stateKey, JOURNEY_OPEN_SOURCE_PROBE_STATE_KEY);
|
||||
assert.notEqual(options.stateKey, JOURNEY_PROBE_STATE_KEY);
|
||||
assert.equal(options.sentinelId, JOURNEY_OPEN_SOURCE_END_SENTINEL_ID);
|
||||
assert.equal(
|
||||
options.startClick?.sentinelId,
|
||||
JOURNEY_OPEN_SOURCE_START_SENTINEL_ID
|
||||
);
|
||||
assert.notEqual(options.sentinelId, JOURNEY_IPC_SENTINEL_ID);
|
||||
assert.match(
|
||||
options.startClick?.selector ?? '',
|
||||
/dashboard-recent-sources-rail-card/
|
||||
);
|
||||
assert.match(options.startClick?.selector ?? '', /app-playlist-item/);
|
||||
assert.equal(options.routeFragment, '/workspace/xtreams/');
|
||||
assert.match(options.cardSelector, /app-grid-list mat-card/);
|
||||
assert.match(options.cardSelector, /channel-item/);
|
||||
assert.deepEqual(options.companionSelectors, [
|
||||
'app-workspace-context-panel .category-item',
|
||||
]);
|
||||
});
|
||||
@@ -1,32 +1,60 @@
|
||||
import type { Page } from '@playwright/test';
|
||||
|
||||
/**
|
||||
* Renderer-side probe for the performance journeys (J1 "Launch to usable").
|
||||
* Renderer-side probe for the performance journeys.
|
||||
*
|
||||
* The probe is injected from the test side through `addInitScript` while the
|
||||
* journey gate (`journey-renderer-gate.cjs`) parks the window on
|
||||
* `about:blank`, so it runs before any renderer script and never touches
|
||||
* production code. It
|
||||
* counts DOM mutations, layout shifts and long tasks until the journey's
|
||||
* terminal condition and then emits one JSON blob under
|
||||
* `window.__iptvnatorJourneyProbe`.
|
||||
* J1 "Launch to usable" starts at document start: the probe is injected from
|
||||
* the test side through `addInitScript` while the journey gate
|
||||
* (`journey-renderer-gate.cjs`) parks the window on `about:blank`, so it runs
|
||||
* before any renderer script and never touches production code. Journeys that
|
||||
* start with a click (J2 "Open a source") pass `startClick`: the probe is
|
||||
* evaluated in the loaded document, arms a capture-phase `click` listener on
|
||||
* `window` (which runs before any listener of the app) and starts counting at
|
||||
* the first click inside `startClick.selector`. It counts DOM mutations,
|
||||
* layout shifts and long tasks until the journey's terminal condition and
|
||||
* then emits one JSON blob under `options.stateKey`.
|
||||
*
|
||||
* IPC invocations are not counted here: the bridge object exposed by
|
||||
* `contextBridge` is frozen, so the probe cannot wrap it. Instead the probe
|
||||
* fires one sentinel bridge call at the terminal moment; the main-process
|
||||
* fires one sentinel bridge call at the terminal moment (and, for a click
|
||||
* start, one start sentinel before the app sees the click); the main-process
|
||||
* capture (`journey-main-ipc-capture.ts`) counts the preload's renderer-API
|
||||
* trace events received before that sentinel. Renderer-to-main IPC is
|
||||
* ordered, so the count is exact regardless of clock skew.
|
||||
* trace events received between the two. Renderer-to-main IPC is ordered, so
|
||||
* the count is exact regardless of clock skew.
|
||||
*/
|
||||
export const JOURNEY_PROBE_STATE_KEY = '__iptvnatorJourneyProbe';
|
||||
export const JOURNEY_PROBE_SCHEMA_VERSION = 1;
|
||||
export const JOURNEY_IPC_SENTINEL_ID = '__iptvnator-journey-sentinel__';
|
||||
/** Bridge method used for the sentinel: a read-only lookup by id. */
|
||||
export const JOURNEY_IPC_SENTINEL_METHOD = 'dbGetAppPlaylist';
|
||||
export const JOURNEY_OPEN_SOURCE_PROBE_STATE_KEY =
|
||||
'__iptvnatorJourneyOpenSourceProbe';
|
||||
export const JOURNEY_OPEN_SOURCE_START_SENTINEL_ID =
|
||||
'__iptvnator-journey-open-source-start__';
|
||||
export const JOURNEY_OPEN_SOURCE_END_SENTINEL_ID =
|
||||
'__iptvnator-journey-open-source-end__';
|
||||
/** The Xtream portal card on the dashboard or its row on /workspace/sources. */
|
||||
export const JOURNEY_OPEN_SOURCE_START_SELECTOR =
|
||||
'[data-test-id="dashboard-recent-sources-rail-card"], app-playlist-item';
|
||||
/**
|
||||
* Bridge method used for the sentinels. The preload emits the trace event
|
||||
* before it forwards the call, and `SOURCE_HEALTH_CANCEL` only looks the id
|
||||
* up in an in-memory map in the main process, so a marker call never
|
||||
* reaches the database worker, the disk or the network and cannot queue
|
||||
* ahead of the work being measured.
|
||||
*/
|
||||
export const JOURNEY_IPC_SENTINEL_METHOD = 'cancelSourceProbe';
|
||||
|
||||
export interface JourneyRendererProbeStartClick {
|
||||
/** The journey starts at the first click inside this selector. */
|
||||
readonly selector: string;
|
||||
/** Id of the start sentinel sent before the app handles the click. */
|
||||
readonly sentinelId: string;
|
||||
}
|
||||
|
||||
export interface JourneyRendererProbeOptions {
|
||||
/** Selector for the element whose visibility ends the journey. */
|
||||
readonly cardSelector: string;
|
||||
/** Further selectors that must each match a visible element as well. */
|
||||
readonly companionSelectors?: readonly string[];
|
||||
readonly journey: string;
|
||||
/** Pathname fragment the terminal route must contain. */
|
||||
readonly routeFragment: string;
|
||||
@@ -34,13 +62,21 @@ export interface JourneyRendererProbeOptions {
|
||||
readonly sentinelMethod: string;
|
||||
/** Element id of the inline splash that must be gone at the end. */
|
||||
readonly splashId: string;
|
||||
/** Absent: the journey starts at document start (J1). */
|
||||
readonly startClick?: JourneyRendererProbeStartClick;
|
||||
readonly stateKey: string;
|
||||
}
|
||||
|
||||
export interface JourneyRendererProbeCounters {
|
||||
domMutations: number;
|
||||
/** Shifts with `hadRecentInput === false` (the CLS definition). */
|
||||
layoutShiftScore: number;
|
||||
longTasks: number;
|
||||
/**
|
||||
* Shifts with `hadRecentInput === true`. Zero for J1, which has no
|
||||
* input; a click-started journey runs inside the 500 ms input window.
|
||||
*/
|
||||
recentInputLayoutShiftScore: number;
|
||||
}
|
||||
|
||||
export interface JourneyRendererProbeState {
|
||||
@@ -67,14 +103,30 @@ export interface JourneyRendererProbeState {
|
||||
readonly domContentLoadedEpochMs: number;
|
||||
readonly loadEventEndEpochMs: number;
|
||||
} | null;
|
||||
/** Click-started journeys: activity before the click, for settling. */
|
||||
readonly preStart: {
|
||||
domMutations: number;
|
||||
lastMutationEpochMs: number | null;
|
||||
};
|
||||
readonly schemaVersion: number;
|
||||
sentinel: {
|
||||
readonly epochMs: number | null;
|
||||
readonly status: 'bridge-missing' | 'failed' | 'not-sent' | 'sent';
|
||||
};
|
||||
start: {
|
||||
/** `min(event.timeStamp, listener time)` as epoch milliseconds. */
|
||||
readonly epochMs: number;
|
||||
readonly listenerEpochMs: number;
|
||||
readonly pathname: string;
|
||||
readonly sentinelStatus: 'bridge-missing' | 'failed' | 'sent';
|
||||
readonly targetTag: string;
|
||||
readonly targetTestId: string | null;
|
||||
} | null;
|
||||
terminal: {
|
||||
readonly cardCount: number;
|
||||
readonly cardTag: string;
|
||||
readonly cardTestId: string | null;
|
||||
readonly companionCounts: readonly number[];
|
||||
readonly epochMs: number;
|
||||
readonly pathname: string;
|
||||
} | null;
|
||||
@@ -92,6 +144,8 @@ export function journeyRendererProbeScript(
|
||||
return;
|
||||
}
|
||||
const epoch = (): number => performance.timeOrigin + performance.now();
|
||||
const startClick = options.startClick ?? null;
|
||||
const companionSelectors = options.companionSelectors ?? [];
|
||||
const bridge = target['electron'] as Record<string, unknown> | undefined;
|
||||
const state: JourneyRendererProbeState = {
|
||||
capabilities: {
|
||||
@@ -102,7 +156,12 @@ export function journeyRendererProbeScript(
|
||||
? 'documentElement'
|
||||
: 'document',
|
||||
},
|
||||
counters: { domMutations: 0, layoutShiftScore: 0, longTasks: 0 },
|
||||
counters: {
|
||||
domMutations: 0,
|
||||
layoutShiftScore: 0,
|
||||
longTasks: 0,
|
||||
recentInputLayoutShiftScore: 0,
|
||||
},
|
||||
final: false,
|
||||
firstCardPaintEpochMs: null,
|
||||
installed: {
|
||||
@@ -116,17 +175,41 @@ export function journeyRendererProbeScript(
|
||||
journey: options.journey,
|
||||
longTaskDurationsMs: [],
|
||||
navigation: null,
|
||||
preStart: { domMutations: 0, lastMutationEpochMs: null },
|
||||
schemaVersion: 1,
|
||||
sentinel: { epochMs: null, status: 'not-sent' },
|
||||
start: null,
|
||||
terminal: null,
|
||||
};
|
||||
target[options.stateKey] = state;
|
||||
if (
|
||||
state.installed.scriptCount > 0 ||
|
||||
state.installed.readyState !== 'loading'
|
||||
startClick === null &&
|
||||
(state.installed.scriptCount > 0 ||
|
||||
state.installed.readyState !== 'loading')
|
||||
) {
|
||||
state.invalidReasons.push('probe-installed-after-document-start');
|
||||
}
|
||||
// Performance entries before the journey's start belong to an earlier
|
||||
// journey (buffered entries included) and are dropped.
|
||||
let fromEpochMs =
|
||||
startClick === null
|
||||
? Number.NEGATIVE_INFINITY
|
||||
: Number.POSITIVE_INFINITY;
|
||||
const inWindow = (entry: PerformanceEntry, untilEpochMs: number) => {
|
||||
const entryEpochMs = performance.timeOrigin + entry.startTime;
|
||||
return entryEpochMs >= fromEpochMs && entryEpochMs <= untilEpochMs;
|
||||
};
|
||||
// A task overlaps the window when it ends after the start. The main
|
||||
// thread runs one task at a time, so the only task that overlaps the
|
||||
// click is the one that dispatches it, which began before the event's
|
||||
// timestamp and must still count.
|
||||
const overlapsWindow = (entry: PerformanceEntry, untilEpochMs: number) => {
|
||||
const entryEpochMs = performance.timeOrigin + entry.startTime;
|
||||
return (
|
||||
entryEpochMs + entry.duration >= fromEpochMs &&
|
||||
entryEpochMs <= untilEpochMs
|
||||
);
|
||||
};
|
||||
|
||||
const acceptLayoutShift = (
|
||||
entries: readonly PerformanceEntry[],
|
||||
@@ -138,12 +221,15 @@ export function journeyRendererProbeScript(
|
||||
value?: number;
|
||||
};
|
||||
if (
|
||||
shift.hadRecentInput === true ||
|
||||
typeof shift.value !== 'number' ||
|
||||
performance.timeOrigin + shift.startTime > untilEpochMs
|
||||
!inWindow(entry, untilEpochMs)
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
if (shift.hadRecentInput === true) {
|
||||
state.counters.recentInputLayoutShiftScore += shift.value;
|
||||
continue;
|
||||
}
|
||||
state.counters.layoutShiftScore += shift.value;
|
||||
}
|
||||
};
|
||||
@@ -152,10 +238,7 @@ export function journeyRendererProbeScript(
|
||||
untilEpochMs: number
|
||||
): void => {
|
||||
for (const entry of entries) {
|
||||
if (
|
||||
entry.duration <= 50 ||
|
||||
performance.timeOrigin + entry.startTime > untilEpochMs
|
||||
) {
|
||||
if (entry.duration <= 50 || !overlapsWindow(entry, untilEpochMs)) {
|
||||
continue;
|
||||
}
|
||||
state.counters.longTasks += 1;
|
||||
@@ -217,20 +300,26 @@ export function journeyRendererProbeScript(
|
||||
state.firstCardPaintEpochMs = untilEpochMs;
|
||||
state.final = true;
|
||||
};
|
||||
const sendSentinel = (): void => {
|
||||
const callSentinel = (id: string): 'bridge-missing' | 'failed' | 'sent' => {
|
||||
const method = bridge?.[options.sentinelMethod];
|
||||
if (typeof method !== 'function') {
|
||||
state.sentinel = { epochMs: null, status: 'bridge-missing' };
|
||||
return;
|
||||
return 'bridge-missing';
|
||||
}
|
||||
try {
|
||||
const result: unknown = method.call(bridge, options.sentinelId);
|
||||
state.sentinel = { epochMs: epoch(), status: 'sent' };
|
||||
const result: unknown = method.call(bridge, id);
|
||||
void Promise.resolve(result).catch(() => undefined);
|
||||
return 'sent';
|
||||
} catch {
|
||||
state.sentinel = { epochMs: null, status: 'failed' };
|
||||
return 'failed';
|
||||
}
|
||||
};
|
||||
const sendSentinel = (): void => {
|
||||
const status = callSentinel(options.sentinelId);
|
||||
state.sentinel = {
|
||||
epochMs: status === 'sent' ? epoch() : null,
|
||||
status,
|
||||
};
|
||||
};
|
||||
const isVisible = (element: Element | null): element is HTMLElement =>
|
||||
element instanceof HTMLElement && element.getClientRects().length > 0;
|
||||
const readNavigation = (): JourneyRendererProbeState['navigation'] => {
|
||||
@@ -249,8 +338,17 @@ export function journeyRendererProbeScript(
|
||||
};
|
||||
};
|
||||
|
||||
const countPreStart = (count: number): void => {
|
||||
if (count === 0) return;
|
||||
state.preStart.domMutations += count;
|
||||
state.preStart.lastMutationEpochMs = epoch();
|
||||
};
|
||||
const mutationObserver = new MutationObserver((records) => {
|
||||
if (state.terminal !== null) return;
|
||||
if (startClick !== null && state.start === null) {
|
||||
countPreStart(records.length);
|
||||
return;
|
||||
}
|
||||
state.counters.domMutations += records.length;
|
||||
if (
|
||||
!location.pathname.includes(options.routeFragment) ||
|
||||
@@ -260,17 +358,28 @@ export function journeyRendererProbeScript(
|
||||
}
|
||||
const card = document.querySelector(options.cardSelector);
|
||||
if (!isVisible(card)) return;
|
||||
const companionCounts: number[] = [];
|
||||
for (const selector of companionSelectors) {
|
||||
if (!isVisible(document.querySelector(selector))) return;
|
||||
companionCounts.push(document.querySelectorAll(selector).length);
|
||||
}
|
||||
state.terminal = {
|
||||
cardCount: document.querySelectorAll(options.cardSelector).length,
|
||||
cardTag: card.tagName.toLowerCase(),
|
||||
cardTestId: card.getAttribute('data-test-id'),
|
||||
companionCounts,
|
||||
epochMs: epoch(),
|
||||
pathname: location.pathname,
|
||||
};
|
||||
mutationObserver.disconnect();
|
||||
sendSentinel();
|
||||
state.navigation = readNavigation();
|
||||
if (state.navigation === null) {
|
||||
state.invalidReasons.push('load-event-not-finished-at-first-card');
|
||||
if (startClick === null) {
|
||||
state.navigation = readNavigation();
|
||||
if (state.navigation === null) {
|
||||
state.invalidReasons.push(
|
||||
'load-event-not-finished-at-first-card'
|
||||
);
|
||||
}
|
||||
}
|
||||
const ng = target['ng'] as Record<string, unknown> | undefined;
|
||||
state.capabilities.changeDetectionTicks =
|
||||
@@ -291,6 +400,37 @@ export function journeyRendererProbeScript(
|
||||
childList: true,
|
||||
subtree: true,
|
||||
});
|
||||
if (startClick === null) return;
|
||||
|
||||
// Capture phase on window runs before every listener of the app, so the
|
||||
// start sentinel precedes any bridge call the click causes and the
|
||||
// mutation count starts before the app touches the DOM.
|
||||
const onClick = (event: Event): void => {
|
||||
if (state.start !== null) return;
|
||||
const origin =
|
||||
event.target instanceof Element
|
||||
? event.target.closest(startClick.selector)
|
||||
: null;
|
||||
if (origin === null) return;
|
||||
const listenerEpochMs = epoch();
|
||||
const eventEpochMs = performance.timeOrigin + event.timeStamp;
|
||||
countPreStart(mutationObserver.takeRecords().length);
|
||||
const sentinelStatus = callSentinel(startClick.sentinelId);
|
||||
state.start = {
|
||||
epochMs:
|
||||
Number.isFinite(eventEpochMs) && eventEpochMs <= listenerEpochMs
|
||||
? eventEpochMs
|
||||
: listenerEpochMs,
|
||||
listenerEpochMs,
|
||||
pathname: location.pathname,
|
||||
sentinelStatus,
|
||||
targetTag: origin.tagName.toLowerCase(),
|
||||
targetTestId: origin.getAttribute('data-test-id'),
|
||||
};
|
||||
fromEpochMs = state.start.epochMs;
|
||||
window.removeEventListener('click', onClick, true);
|
||||
};
|
||||
window.addEventListener('click', onClick, true);
|
||||
}
|
||||
|
||||
export function createLaunchJourneyProbeOptions(): JourneyRendererProbeOptions {
|
||||
@@ -306,6 +446,31 @@ export function createLaunchJourneyProbeOptions(): JourneyRendererProbeOptions {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Options for J2 "Open a source": the click on the Xtream portal card (or its
|
||||
* source row) starts the journey; it ends when the section's category list in
|
||||
* the context panel and the first page of its items are visible.
|
||||
*/
|
||||
export function createOpenSourceJourneyProbeOptions(): JourneyRendererProbeOptions {
|
||||
return {
|
||||
// Grid cards (VOD/series, the section a portal opens on), content
|
||||
// cards and live channel rows; skeleton cards are not matched.
|
||||
cardSelector:
|
||||
'app-grid-list mat-card, .content-card, [data-test-id="channel-item"]',
|
||||
companionSelectors: ['app-workspace-context-panel .category-item'],
|
||||
journey: 'open-source',
|
||||
routeFragment: '/workspace/xtreams/',
|
||||
sentinelId: JOURNEY_OPEN_SOURCE_END_SENTINEL_ID,
|
||||
sentinelMethod: JOURNEY_IPC_SENTINEL_METHOD,
|
||||
splashId: 'initial-splash',
|
||||
startClick: {
|
||||
selector: JOURNEY_OPEN_SOURCE_START_SELECTOR,
|
||||
sentinelId: JOURNEY_OPEN_SOURCE_START_SENTINEL_ID,
|
||||
},
|
||||
stateKey: JOURNEY_OPEN_SOURCE_PROBE_STATE_KEY,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Registers the probe on a page that is still parked on `about:blank` by the
|
||||
* journey gate, so it is guaranteed to run at the start of the next document.
|
||||
@@ -317,6 +482,20 @@ export async function installJourneyRendererProbe(
|
||||
await page.addInitScript(journeyRendererProbeScript, options);
|
||||
}
|
||||
|
||||
/**
|
||||
* Arms a click-started probe in the current document. Playwright serializes
|
||||
* the same self-contained script as for `addInitScript`.
|
||||
*/
|
||||
export async function armJourneyRendererProbe(
|
||||
page: Page,
|
||||
options: JourneyRendererProbeOptions
|
||||
): Promise<void> {
|
||||
if (!options.startClick) {
|
||||
throw new Error('journey-renderer-probe-arm-needs-start-click');
|
||||
}
|
||||
await page.evaluate(journeyRendererProbeScript, options);
|
||||
}
|
||||
|
||||
export async function waitForJourneyRendererProbe(
|
||||
page: Page,
|
||||
stateKey: string,
|
||||
@@ -363,6 +542,11 @@ export function assertJourneyRendererProbeState(
|
||||
`journey-renderer-probe-sentinel-${state.sentinel.status}`
|
||||
);
|
||||
}
|
||||
if (state.start !== null && state.start.sentinelStatus !== 'sent') {
|
||||
throw new Error(
|
||||
`journey-renderer-probe-start-sentinel-${state.start.sentinelStatus}`
|
||||
);
|
||||
}
|
||||
// A zero from an observer that never ran is not a measurement; a build
|
||||
// without these entry types must fail the iteration, never lower a
|
||||
// baseline.
|
||||
|
||||
@@ -1,18 +1,22 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdtemp, readFile, rm } from 'node:fs/promises';
|
||||
import { mkdtemp, readdir, readFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import test from 'node:test';
|
||||
|
||||
import {
|
||||
formatJourneyOutputTimestamp,
|
||||
JOURNEY_RUN_STARTED_AT_ENV,
|
||||
JOURNEY_SUMMARY_SCHEMA_VERSION,
|
||||
percentile,
|
||||
recordJourneySummaryEntry,
|
||||
resolveJourneyRunSummaryPath,
|
||||
resolveJourneySummaryPath,
|
||||
summarizeJourneyIterations,
|
||||
writeJourneySummary,
|
||||
type JourneyIterationRecord,
|
||||
type JourneySummary,
|
||||
type JourneySummaryHarness,
|
||||
} from './journey-summary';
|
||||
|
||||
function iteration(
|
||||
@@ -172,6 +176,7 @@ test('writes the summary below dist/performance/journeys/<timestamp> and never o
|
||||
node: 'v22',
|
||||
platform: 'darwin',
|
||||
rendererIndex: 'dist/apps/web/index.html',
|
||||
runId: 'run-a',
|
||||
warmupIterations: 0,
|
||||
},
|
||||
journeys: {
|
||||
@@ -196,3 +201,111 @@ test('writes the summary below dist/performance/journeys/<timestamp> and never o
|
||||
await rm(root, { force: true, recursive: true });
|
||||
}
|
||||
});
|
||||
|
||||
const HARNESS: JourneySummaryHarness = {
|
||||
arch: 'arm64',
|
||||
ci: false,
|
||||
electron: '43.0.0',
|
||||
electronMain: 'dist/apps/electron-backend/main.js',
|
||||
measuredIterations: 1,
|
||||
node: 'v22',
|
||||
platform: 'darwin',
|
||||
rendererIndex: 'dist/apps/web/index.html',
|
||||
runId: 'run-a',
|
||||
warmupIterations: 0,
|
||||
};
|
||||
|
||||
test('one journey run resolves to one summary path from its start time', () => {
|
||||
const root = '/repo';
|
||||
assert.equal(
|
||||
resolveJourneyRunSummaryPath(root, {
|
||||
[JOURNEY_RUN_STARTED_AT_ENV]: '2026-09-27T09:14:01.999Z',
|
||||
}),
|
||||
join(
|
||||
root,
|
||||
'dist',
|
||||
'performance',
|
||||
'journeys',
|
||||
'20260927T091401Z',
|
||||
'summary.json'
|
||||
)
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
resolveJourneyRunSummaryPath(root, {
|
||||
[JOURNEY_RUN_STARTED_AT_ENV]: 'not a date',
|
||||
}),
|
||||
/invalid-date/
|
||||
);
|
||||
});
|
||||
|
||||
test('journeys of one run are merged into one summary and never replaced', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'iptvnator-journey-run-'));
|
||||
try {
|
||||
const summaryPath = resolveJourneySummaryPath(
|
||||
root,
|
||||
new Date('2026-09-27T09:14:01Z')
|
||||
);
|
||||
const launch = summarizeJourneyIterations(
|
||||
[iteration(0, { a: 1 }, { w: 1 })],
|
||||
{}
|
||||
);
|
||||
const openSource = summarizeJourneyIterations(
|
||||
[iteration(0, { b: 2 }, { v: 3 })],
|
||||
{ c: 'reason' }
|
||||
);
|
||||
await recordJourneySummaryEntry(summaryPath, HARNESS, 'launch', launch);
|
||||
const merged = await recordJourneySummaryEntry(
|
||||
summaryPath,
|
||||
HARNESS,
|
||||
'open-source',
|
||||
openSource
|
||||
);
|
||||
const written = JSON.parse(
|
||||
await readFile(summaryPath, 'utf8')
|
||||
) as JourneySummary;
|
||||
assert.deepEqual(Object.keys(written.journeys), [
|
||||
'launch',
|
||||
'open-source',
|
||||
]);
|
||||
assert.deepEqual(written, JSON.parse(JSON.stringify(merged)));
|
||||
assert.equal(written.journeys['open-source']?.counters['b'], 2);
|
||||
assert.equal(written.journeys['launch']?.counters['a'], 1);
|
||||
assert.deepEqual(written.harness, HARNESS);
|
||||
// No temporary file is left next to the summary.
|
||||
assert.deepEqual(await readdir(join(summaryPath, '..')), [
|
||||
'summary.json',
|
||||
]);
|
||||
|
||||
await assert.rejects(
|
||||
recordJourneySummaryEntry(summaryPath, HARNESS, 'launch', launch),
|
||||
/merge-duplicate-launch/
|
||||
);
|
||||
await assert.rejects(
|
||||
recordJourneySummaryEntry(
|
||||
summaryPath,
|
||||
{ ...HARNESS, electron: '44.0.0' },
|
||||
'playback',
|
||||
launch
|
||||
),
|
||||
/merge-harness-mismatch/
|
||||
);
|
||||
// Another invocation in the same second, with journeys this file
|
||||
// does not have yet, must not merge into this run.
|
||||
await assert.rejects(
|
||||
recordJourneySummaryEntry(
|
||||
summaryPath,
|
||||
{ ...HARNESS, runId: 'run-b' },
|
||||
'playback',
|
||||
launch
|
||||
),
|
||||
/merge-other-run/
|
||||
);
|
||||
const unchanged = JSON.parse(
|
||||
await readFile(summaryPath, 'utf8')
|
||||
) as JourneySummary;
|
||||
assert.deepEqual(unchanged, written);
|
||||
} finally {
|
||||
await rm(root, { force: true, recursive: true });
|
||||
}
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
import { mkdir, writeFile } from 'node:fs/promises';
|
||||
import { mkdir, readFile, rename, writeFile } from 'node:fs/promises';
|
||||
import { dirname, join } from 'node:path';
|
||||
|
||||
/**
|
||||
@@ -44,6 +44,8 @@ export interface JourneySummaryHarness {
|
||||
readonly node: string;
|
||||
readonly platform: string;
|
||||
readonly rendererIndex: string;
|
||||
/** Unique per `perf:journeys` invocation; see `JOURNEY_RUN_ID_ENV`. */
|
||||
readonly runId: string;
|
||||
readonly warmupIterations: number;
|
||||
}
|
||||
|
||||
@@ -192,6 +194,90 @@ export function resolveJourneySummaryPath(
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Environment variable that pins one `perf:journeys` invocation to one
|
||||
* summary directory. `playwright.journeys.config.ts` sets it in the runner
|
||||
* before the worker starts, so every journey spec of the run writes to the
|
||||
* same file.
|
||||
*/
|
||||
export const JOURNEY_RUN_STARTED_AT_ENV = 'IPTVNATOR_JOURNEY_RUN_STARTED_AT';
|
||||
/**
|
||||
* Unique id of the invocation, set next to the start time. Two invocations
|
||||
* that start in the same second resolve to the same directory; the id keeps
|
||||
* one from merging into the other's summary.
|
||||
*/
|
||||
export const JOURNEY_RUN_ID_ENV = 'IPTVNATOR_JOURNEY_RUN_ID';
|
||||
|
||||
export function resolveJourneyRunSummaryPath(
|
||||
repositoryRoot: string,
|
||||
environment: NodeJS.ProcessEnv = process.env
|
||||
): string {
|
||||
const raw = environment[JOURNEY_RUN_STARTED_AT_ENV];
|
||||
const startedAt = raw ? new Date(raw) : new Date();
|
||||
return resolveJourneySummaryPath(repositoryRoot, startedAt);
|
||||
}
|
||||
|
||||
function isFileExistsError(error: unknown): boolean {
|
||||
return (
|
||||
typeof error === 'object' &&
|
||||
error !== null &&
|
||||
(error as { code?: unknown }).code === 'EEXIST'
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Adds one journey to the run's summary. The first journey creates the
|
||||
* file; later journeys merge into it only when they belong to the same run
|
||||
* (`harness.runId`) and the rest of the harness is identical. A journey that
|
||||
* is already present fails, so no measurement is ever overwritten.
|
||||
*/
|
||||
export async function recordJourneySummaryEntry(
|
||||
summaryPath: string,
|
||||
harness: JourneySummaryHarness,
|
||||
journeyId: string,
|
||||
entry: JourneySummaryEntry
|
||||
): Promise<JourneySummary> {
|
||||
const created: JourneySummary = {
|
||||
generatedAt: new Date().toISOString(),
|
||||
harness,
|
||||
journeys: { [journeyId]: entry },
|
||||
schemaVersion: JOURNEY_SUMMARY_SCHEMA_VERSION,
|
||||
};
|
||||
try {
|
||||
await writeJourneySummary(summaryPath, created);
|
||||
return created;
|
||||
} catch (error) {
|
||||
if (!isFileExistsError(error)) throw error;
|
||||
}
|
||||
const existing = JSON.parse(
|
||||
await readFile(summaryPath, 'utf8')
|
||||
) as JourneySummary;
|
||||
if (existing.schemaVersion !== JOURNEY_SUMMARY_SCHEMA_VERSION) {
|
||||
throw new Error('journey-summary-merge-schema-mismatch');
|
||||
}
|
||||
if (existing.harness?.runId !== harness.runId) {
|
||||
throw new Error('journey-summary-merge-other-run');
|
||||
}
|
||||
if (JSON.stringify(existing.harness) !== JSON.stringify(harness)) {
|
||||
throw new Error('journey-summary-merge-harness-mismatch');
|
||||
}
|
||||
if (Object.prototype.hasOwnProperty.call(existing.journeys, journeyId)) {
|
||||
throw new Error(`journey-summary-merge-duplicate-${journeyId}`);
|
||||
}
|
||||
const merged: JourneySummary = {
|
||||
...existing,
|
||||
generatedAt: created.generatedAt,
|
||||
journeys: { ...existing.journeys, [journeyId]: entry },
|
||||
};
|
||||
const temporaryPath = `${summaryPath}.${process.pid}.tmp`;
|
||||
await writeFile(temporaryPath, `${JSON.stringify(merged, null, 2)}\n`, {
|
||||
encoding: 'utf8',
|
||||
flag: 'wx',
|
||||
});
|
||||
await rename(temporaryPath, summaryPath);
|
||||
return merged;
|
||||
}
|
||||
|
||||
export async function writeJourneySummary(
|
||||
summaryPath: string,
|
||||
summary: JourneySummary
|
||||
|
||||
@@ -23,6 +23,7 @@ function measurement(
|
||||
domMutations: 480,
|
||||
layoutShiftScore: 0.123456789,
|
||||
longTasks: 2,
|
||||
recentInputLayoutShiftScore: 0,
|
||||
},
|
||||
final: true,
|
||||
firstCardPaintEpochMs: 2_650,
|
||||
@@ -40,24 +41,32 @@ function measurement(
|
||||
domContentLoadedEpochMs: 1_300,
|
||||
loadEventEndEpochMs: 1_400.26,
|
||||
},
|
||||
preStart: { domMutations: 0, lastMutationEpochMs: null },
|
||||
schemaVersion: 1,
|
||||
sentinel: { epochMs: 2_601, status: 'sent' },
|
||||
start: null,
|
||||
terminal: {
|
||||
cardCount: 2,
|
||||
cardTag: 'div',
|
||||
cardTestId: 'dashboard-recent-sources-rail-card',
|
||||
companionCounts: [],
|
||||
epochMs: 2_600.04,
|
||||
pathname: '/dist/apps/web/workspace/dashboard',
|
||||
},
|
||||
};
|
||||
const ipc: JourneyMainIpcCaptureState = {
|
||||
callsAfterSentinel: 3,
|
||||
callsBeforeStart: 0,
|
||||
callsBeforeSentinel: 14,
|
||||
callsByMethod: { dbGetAppPlaylists: 1, getSettings: 13 },
|
||||
inFlightByMethod: {},
|
||||
installedEpochMs: 1_100,
|
||||
malformedEvents: 0,
|
||||
processStartEpochMs: 900,
|
||||
senderIds: [1],
|
||||
sentinel: { occurrences: 1, receivedEpochMs: 2_602 },
|
||||
start: null,
|
||||
unmatchedCompletions: 0,
|
||||
};
|
||||
return {
|
||||
electronVersion: '43.3.0',
|
||||
@@ -146,6 +155,13 @@ test('rejects measurements whose clocks or probes are inconsistent', () => {
|
||||
}),
|
||||
/incomplete-probe/
|
||||
);
|
||||
// A launch that ran without the main-process counters (as J2's do) is
|
||||
// not a J1 measurement.
|
||||
assert.throws(
|
||||
() =>
|
||||
toLaunchIterationRecord(0, false, { ...base, mainCounters: null }),
|
||||
/main-counters-missing/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
toLaunchIterationRecord(0, false, { ...base, spawnEpochMs: 2_700 }),
|
||||
|
||||
@@ -45,7 +45,8 @@ export interface LaunchJourneyMeasurement {
|
||||
readonly electronVersion: string;
|
||||
readonly gate: JourneyRendererGateState;
|
||||
readonly ipc: JourneyMainIpcCaptureState;
|
||||
readonly mainCounters: JourneyMainCountersState;
|
||||
/** Null when the launch ran without the main-process counters (J2). */
|
||||
readonly mainCounters: JourneyMainCountersState | null;
|
||||
readonly pid: number;
|
||||
readonly renderer: JourneyRendererProbeState;
|
||||
readonly spawnEpochMs: number;
|
||||
@@ -61,6 +62,9 @@ export function toLaunchIterationRecord(
|
||||
measurement: LaunchJourneyMeasurement
|
||||
): JourneyIterationRecord {
|
||||
const { ipc, mainCounters, renderer, spawnEpochMs } = measurement;
|
||||
if (mainCounters === null) {
|
||||
throw new Error('launch-journey-record-main-counters-missing');
|
||||
}
|
||||
if (renderer.terminal === null || renderer.navigation === null) {
|
||||
throw new Error('launch-journey-record-incomplete-probe');
|
||||
}
|
||||
|
||||
@@ -0,0 +1,317 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import type { JourneyMainIpcCaptureState } from './journey-main-ipc-capture';
|
||||
import type { JourneyMockRequest } from './journey-mock-request-ledger';
|
||||
import type { JourneyRendererProbeState } from './journey-renderer-probe';
|
||||
import { summarizeJourneyIterations } from './journey-summary';
|
||||
import {
|
||||
OPEN_SOURCE_JOURNEY_UNAVAILABLE_COUNTERS,
|
||||
toOpenSourceIterationRecord,
|
||||
type OpenSourceJourneyMeasurement,
|
||||
} from './open-source-journey-record';
|
||||
|
||||
const SOURCE_PATH = '/dist/apps/web/workspace/xtreams/playlist-1/vod';
|
||||
|
||||
function request(sequence: number, route: string, epochMs: number) {
|
||||
return { epochMs, method: 'GET', route, sequence } as JourneyMockRequest;
|
||||
}
|
||||
|
||||
function measurement(
|
||||
overrides: Partial<OpenSourceJourneyMeasurement> = {}
|
||||
): OpenSourceJourneyMeasurement {
|
||||
const renderer: JourneyRendererProbeState = {
|
||||
capabilities: {
|
||||
changeDetectionTicks: 'unavailable-ng-global-not-published',
|
||||
layoutShift: true,
|
||||
longTask: true,
|
||||
observedTarget: 'documentElement',
|
||||
},
|
||||
counters: {
|
||||
domMutations: 1_596,
|
||||
layoutShiftScore: 0.0004,
|
||||
longTasks: 1,
|
||||
recentInputLayoutShiftScore: 0.22061,
|
||||
},
|
||||
final: true,
|
||||
firstCardPaintEpochMs: 10_090,
|
||||
installed: {
|
||||
bridgePresent: true,
|
||||
documentElementPresent: true,
|
||||
epochMs: 9_000,
|
||||
readyState: 'complete',
|
||||
scriptCount: 12,
|
||||
},
|
||||
invalidReasons: [],
|
||||
journey: 'open-source',
|
||||
longTaskDurationsMs: [61.26],
|
||||
navigation: null,
|
||||
preStart: { domMutations: 4, lastMutationEpochMs: 9_100 },
|
||||
schemaVersion: 1,
|
||||
sentinel: { epochMs: 10_080.5, status: 'sent' },
|
||||
start: {
|
||||
epochMs: 10_000.2,
|
||||
listenerEpochMs: 10_001,
|
||||
pathname: '/dist/apps/web/workspace/dashboard',
|
||||
sentinelStatus: 'sent',
|
||||
targetTag: 'div',
|
||||
targetTestId: 'dashboard-recent-sources-rail-card',
|
||||
},
|
||||
terminal: {
|
||||
cardCount: 50,
|
||||
cardTag: 'mat-card',
|
||||
cardTestId: null,
|
||||
companionCounts: [8],
|
||||
epochMs: 10_078.54,
|
||||
pathname: SOURCE_PATH,
|
||||
},
|
||||
};
|
||||
const ipc: JourneyMainIpcCaptureState = {
|
||||
callsAfterSentinel: 2,
|
||||
callsBeforeStart: 0,
|
||||
callsBeforeSentinel: 17,
|
||||
callsByMethod: { dbGetAppState: 6, dbGetContent: 2, xtreamRequest: 1 },
|
||||
inFlightByMethod: {},
|
||||
installedEpochMs: 9_500,
|
||||
malformedEvents: 0,
|
||||
processStartEpochMs: 1_000,
|
||||
senderIds: [1],
|
||||
sentinel: { occurrences: 1, receivedEpochMs: 10_081 },
|
||||
start: { occurrences: 1, receivedEpochMs: 10_002 },
|
||||
unmatchedCompletions: 0,
|
||||
};
|
||||
return {
|
||||
http: {
|
||||
afterSettleBeforeClick: 0,
|
||||
afterSettled: [
|
||||
request(3, '/player_api.php?action=get_vod_streams', 12_500),
|
||||
],
|
||||
beforeClick: [
|
||||
request(0, '/player_api.php?action=get_account_info', 2_000),
|
||||
],
|
||||
requests: [
|
||||
request(1, '/player_api.php?action=get_account_info', 10_040),
|
||||
request(2, '/assets/marketing/poster/a', 10_200),
|
||||
],
|
||||
},
|
||||
ipc,
|
||||
pid: 4343,
|
||||
renderer,
|
||||
settle: {
|
||||
preStartDomMutations: 4,
|
||||
preStartHttpRequests: 0,
|
||||
preStartIpcCalls: 0,
|
||||
quietMs: 1_000,
|
||||
waitedMs: 1_048,
|
||||
},
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
test('maps the click-started probe, IPC window and mock ledger to exact counters', () => {
|
||||
const record = toOpenSourceIterationRecord(3, false, measurement());
|
||||
assert.equal(record.index, 3);
|
||||
assert.equal(record.warmup, false);
|
||||
assert.equal(record.pid, 4343);
|
||||
assert.deepEqual(record.counters, {
|
||||
'main.mockHttpRequestsToSettled': 2,
|
||||
'renderer.domMutationsToFirstPage': 1_596,
|
||||
'renderer.ipcCallsToFirstPage': 17,
|
||||
'renderer.layoutShiftScore': 0.221,
|
||||
'renderer.longTasks': 1,
|
||||
});
|
||||
assert.deepEqual(record.wallClock, {
|
||||
clickToFirstPageMs: 78.3,
|
||||
clickToFirstPagePaintMs: 89.8,
|
||||
});
|
||||
assert.deepEqual(record.evidence['layoutShift'], {
|
||||
recentInput: 0.221,
|
||||
withoutRecentInput: 0,
|
||||
});
|
||||
assert.deepEqual(record.evidence['httpRequestsByRoute'], {
|
||||
'/assets/marketing/poster/a': 1,
|
||||
'/player_api.php?action=get_account_info': 1,
|
||||
});
|
||||
// Arrived after the post-terminal quiet sample: evidence, not counted.
|
||||
assert.deepEqual(record.evidence['httpRequestsAfterSettledByRoute'], {
|
||||
'/player_api.php?action=get_vod_streams': 1,
|
||||
});
|
||||
assert.deepEqual(record.evidence['httpRequestsBeforeClickByRoute'], {
|
||||
'/player_api.php?action=get_account_info': 1,
|
||||
});
|
||||
assert.equal(record.evidence['httpRequestsToFirstPage'], 1);
|
||||
assert.equal(record.evidence['ipcCallsAfterFirstPage'], 2);
|
||||
assert.deepEqual(record.evidence['firstPage'], {
|
||||
cardCount: 50,
|
||||
cardTag: 'mat-card',
|
||||
cardTestId: null,
|
||||
categoryCount: 8,
|
||||
pathname: SOURCE_PATH,
|
||||
section: 'vod',
|
||||
});
|
||||
assert.deepEqual(record.evidence['start'], {
|
||||
pathname: '/dist/apps/web/workspace/dashboard',
|
||||
targetTag: 'div',
|
||||
targetTestId: 'dashboard-recent-sources-rail-card',
|
||||
});
|
||||
assert.deepEqual(record.evidence['longTaskDurationsMs'], [61.3]);
|
||||
assert.deepEqual(record.evidence['epochs'], {
|
||||
click: 10_000.2,
|
||||
clickListener: 10_001,
|
||||
firstPage: 10_078.54,
|
||||
firstPagePaint: 10_090,
|
||||
mainIpcSentinel: 10_081,
|
||||
mainIpcStart: 10_002,
|
||||
});
|
||||
});
|
||||
|
||||
test('rejects measurements that did not start at the click or did not open the source', () => {
|
||||
const base = measurement();
|
||||
const renderer = base.renderer;
|
||||
assert.throws(
|
||||
() =>
|
||||
toOpenSourceIterationRecord(0, false, {
|
||||
...base,
|
||||
renderer: { ...renderer, start: null },
|
||||
}),
|
||||
/incomplete-probe/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
toOpenSourceIterationRecord(0, false, {
|
||||
...base,
|
||||
renderer: { ...renderer, terminal: null },
|
||||
}),
|
||||
/incomplete-probe/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
toOpenSourceIterationRecord(0, false, {
|
||||
...base,
|
||||
ipc: { ...base.ipc, start: null },
|
||||
}),
|
||||
/ipc-without-start/
|
||||
);
|
||||
const terminal = renderer.terminal as NonNullable<
|
||||
JourneyRendererProbeState['terminal']
|
||||
>;
|
||||
const start = renderer.start as NonNullable<
|
||||
JourneyRendererProbeState['start']
|
||||
>;
|
||||
assert.throws(
|
||||
() =>
|
||||
toOpenSourceIterationRecord(0, false, {
|
||||
...base,
|
||||
renderer: {
|
||||
...renderer,
|
||||
terminal: { ...terminal, pathname: '/workspace/dashboard' },
|
||||
},
|
||||
}),
|
||||
/record-route/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
toOpenSourceIterationRecord(0, false, {
|
||||
...base,
|
||||
renderer: {
|
||||
...renderer,
|
||||
start: { ...start, pathname: SOURCE_PATH },
|
||||
},
|
||||
}),
|
||||
/started-inside-source/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
toOpenSourceIterationRecord(0, false, {
|
||||
...base,
|
||||
renderer: {
|
||||
...renderer,
|
||||
terminal: { ...terminal, epochMs: start.epochMs },
|
||||
},
|
||||
}),
|
||||
/clock-order/
|
||||
);
|
||||
for (const firstCardPaintEpochMs of [null, terminal.epochMs - 1]) {
|
||||
assert.throws(
|
||||
() =>
|
||||
toOpenSourceIterationRecord(0, false, {
|
||||
...base,
|
||||
renderer: { ...renderer, firstCardPaintEpochMs },
|
||||
}),
|
||||
/clock-order/
|
||||
);
|
||||
}
|
||||
assert.throws(
|
||||
() =>
|
||||
toOpenSourceIterationRecord(0, false, {
|
||||
...base,
|
||||
renderer: {
|
||||
...renderer,
|
||||
capabilities: {
|
||||
...renderer.capabilities,
|
||||
changeDetectionTicks: 'hook-present-not-counted',
|
||||
},
|
||||
},
|
||||
}),
|
||||
/cd-hook-hook-present-not-counted/
|
||||
);
|
||||
});
|
||||
|
||||
test('rejects an iteration with activity between the settle snapshot and the click', () => {
|
||||
const base = measurement();
|
||||
assert.doesNotThrow(() => toOpenSourceIterationRecord(0, false, base));
|
||||
// The probe counted DOM mutations after the snapshot (4) but before the
|
||||
// click, e.g. while Playwright ran its actionability checks.
|
||||
assert.throws(
|
||||
() =>
|
||||
toOpenSourceIterationRecord(0, false, {
|
||||
...base,
|
||||
renderer: {
|
||||
...base.renderer,
|
||||
preStart: { domMutations: 5, lastMutationEpochMs: 9_990 },
|
||||
},
|
||||
}),
|
||||
/activity-before-click-dom$/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
toOpenSourceIterationRecord(0, false, {
|
||||
...base,
|
||||
ipc: { ...base.ipc, callsBeforeStart: 1 },
|
||||
}),
|
||||
/activity-before-click-ipc$/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
toOpenSourceIterationRecord(0, false, {
|
||||
...base,
|
||||
http: { ...base.http, afterSettleBeforeClick: 1 },
|
||||
ipc: { ...base.ipc, callsBeforeStart: 2 },
|
||||
}),
|
||||
/activity-before-click-ipc-http$/
|
||||
);
|
||||
});
|
||||
|
||||
test('summarizes under the J2 counters with the unmeasurable ones listed', () => {
|
||||
const entry = summarizeJourneyIterations(
|
||||
[0, 1, 2].map((index) =>
|
||||
toOpenSourceIterationRecord(
|
||||
index,
|
||||
index === 0,
|
||||
measurement({ pid: 5_000 + index })
|
||||
)
|
||||
),
|
||||
OPEN_SOURCE_JOURNEY_UNAVAILABLE_COUNTERS
|
||||
);
|
||||
assert.equal(entry.counters['renderer.ipcCallsToFirstPage'], 17);
|
||||
assert.equal(
|
||||
entry.counterStability['main.mockHttpRequestsToSettled']?.stable,
|
||||
true
|
||||
);
|
||||
assert.equal(entry.wallClock['clickToFirstPageMs.p50'], 78.3);
|
||||
assert.equal(entry.wallClock['clickToFirstPagePaintMs.p90'], 89.8);
|
||||
assert.deepEqual(Object.keys(entry.unavailable).sort(), [
|
||||
'main.sqlStatementsToFirstPage',
|
||||
'renderer.cdTicksToFirstPage',
|
||||
]);
|
||||
});
|
||||
@@ -0,0 +1,212 @@
|
||||
import type { JourneyMainIpcCaptureState } from './journey-main-ipc-capture';
|
||||
import {
|
||||
countJourneyMockRoutes,
|
||||
type JourneyMockRequest,
|
||||
} from './journey-mock-request-ledger';
|
||||
import type { JourneyRendererProbeState } from './journey-renderer-probe';
|
||||
import type { JourneyIterationRecord } from './journey-summary';
|
||||
|
||||
/**
|
||||
* Maps one measured "open a source" (renderer probe armed at the click, main
|
||||
* IPC capture between the start and end sentinels, mock request ledger) to
|
||||
* the journey summary's iteration record for J2.
|
||||
*/
|
||||
export const OPEN_SOURCE_JOURNEY_ID = 'open-source';
|
||||
|
||||
export const OPEN_SOURCE_JOURNEY_COUNTER = {
|
||||
DOM_MUTATIONS: 'renderer.domMutationsToFirstPage',
|
||||
IPC_CALLS: 'renderer.ipcCallsToFirstPage',
|
||||
LAYOUT_SHIFT_SCORE: 'renderer.layoutShiftScore',
|
||||
LONG_TASKS: 'renderer.longTasks',
|
||||
MOCK_HTTP_REQUESTS: 'main.mockHttpRequestsToSettled',
|
||||
} as const;
|
||||
|
||||
export const OPEN_SOURCE_JOURNEY_WALL_CLOCK = {
|
||||
/** Click until the batch that made the first page visible. */
|
||||
CLICK_TO_FIRST_PAGE: 'clickToFirstPageMs',
|
||||
/** Click until the frame that paints it has been committed. */
|
||||
CLICK_TO_FIRST_PAGE_PAINT: 'clickToFirstPagePaintMs',
|
||||
} as const;
|
||||
|
||||
export const OPEN_SOURCE_JOURNEY_UNAVAILABLE_COUNTERS: Readonly<
|
||||
Record<string, string>
|
||||
> = Object.freeze({
|
||||
'main.sqlStatementsToFirstPage':
|
||||
'The main.sqlStatements running total is read from the test process through the journey gate, so it cannot be sampled at the click or at the first-page batch, and the worker count is ordered against worker responses rather than the renderer. A click-to-settled count is a follow-up.',
|
||||
'renderer.cdTicksToFirstPage':
|
||||
'The electron-performance build optimizes scripts (ngDevMode=false), so Angular does not publish window.ng and ɵsetProfiler is unavailable.',
|
||||
});
|
||||
|
||||
/** How long the app was left alone before the click, and what it did. */
|
||||
export interface OpenSourceJourneySettle {
|
||||
readonly preStartDomMutations: number;
|
||||
readonly preStartHttpRequests: number;
|
||||
readonly preStartIpcCalls: number;
|
||||
readonly quietMs: number;
|
||||
readonly waitedMs: number;
|
||||
}
|
||||
|
||||
export interface OpenSourceJourneyMeasurement {
|
||||
readonly http: {
|
||||
/** Mock requests after the app settled but before the click stamp. */
|
||||
readonly afterSettleBeforeClick: number;
|
||||
/**
|
||||
* Requests after the post-terminal quiet sample; outside the window
|
||||
* because their completion was not waited for.
|
||||
*/
|
||||
readonly afterSettled: readonly JourneyMockRequest[];
|
||||
/** Mock requests from the spawn (J1 and settling) until the click. */
|
||||
readonly beforeClick: readonly JourneyMockRequest[];
|
||||
/** Mock requests from the click until the mock was quiet again. */
|
||||
readonly requests: readonly JourneyMockRequest[];
|
||||
};
|
||||
readonly ipc: JourneyMainIpcCaptureState;
|
||||
readonly pid: number;
|
||||
readonly renderer: JourneyRendererProbeState;
|
||||
readonly settle: OpenSourceJourneySettle;
|
||||
}
|
||||
|
||||
const ROUTE_FRAGMENT = '/workspace/xtreams/';
|
||||
|
||||
function roundTenth(value: number): number {
|
||||
return Math.round(value * 10) / 10;
|
||||
}
|
||||
|
||||
function roundThousandth(value: number): number {
|
||||
return Math.round(value * 1_000) / 1_000;
|
||||
}
|
||||
|
||||
export function toOpenSourceIterationRecord(
|
||||
index: number,
|
||||
warmup: boolean,
|
||||
measurement: OpenSourceJourneyMeasurement
|
||||
): JourneyIterationRecord {
|
||||
const { http, ipc, renderer, settle } = measurement;
|
||||
const { start, terminal } = renderer;
|
||||
if (start === null || terminal === null) {
|
||||
throw new Error('open-source-journey-record-incomplete-probe');
|
||||
}
|
||||
if (ipc.start === null) {
|
||||
throw new Error('open-source-journey-record-ipc-without-start');
|
||||
}
|
||||
if (!terminal.pathname.includes(ROUTE_FRAGMENT)) {
|
||||
throw new Error('open-source-journey-record-route');
|
||||
}
|
||||
if (start.pathname.includes(ROUTE_FRAGMENT)) {
|
||||
throw new Error('open-source-journey-record-started-inside-source');
|
||||
}
|
||||
const clickToFirstPageMs = terminal.epochMs - start.epochMs;
|
||||
const paintEpochMs = renderer.firstCardPaintEpochMs;
|
||||
if (
|
||||
clickToFirstPageMs <= 0 ||
|
||||
paintEpochMs === null ||
|
||||
paintEpochMs < terminal.epochMs
|
||||
) {
|
||||
throw new Error('open-source-journey-record-clock-order');
|
||||
}
|
||||
// Activity that started after the settle snapshot but before the click
|
||||
// (while Playwright ran its actionability checks) could complete after
|
||||
// the click and be counted as J2. The probe and the capture keep
|
||||
// counting until the click itself, so they must still match the
|
||||
// snapshot; otherwise the iteration is rejected.
|
||||
const lateActivity = [
|
||||
renderer.preStart.domMutations !== settle.preStartDomMutations
|
||||
? 'dom'
|
||||
: null,
|
||||
ipc.callsBeforeStart !== settle.preStartIpcCalls ? 'ipc' : null,
|
||||
http.afterSettleBeforeClick > 0 ? 'http' : null,
|
||||
].filter((kind) => kind !== null);
|
||||
if (lateActivity.length > 0) {
|
||||
throw new Error(
|
||||
`open-source-journey-record-activity-before-click-${lateActivity.join('-')}`
|
||||
);
|
||||
}
|
||||
if (
|
||||
renderer.capabilities.changeDetectionTicks !==
|
||||
'unavailable-ng-global-not-published'
|
||||
) {
|
||||
throw new Error(
|
||||
`open-source-journey-record-cd-hook-${renderer.capabilities.changeDetectionTicks}`
|
||||
);
|
||||
}
|
||||
// The ledger's clock is the test process's, the terminal's the
|
||||
// renderer's; the split is evidence only.
|
||||
const requestsToFirstPage = http.requests.filter(
|
||||
(entry) => entry.epochMs <= terminal.epochMs
|
||||
).length;
|
||||
const section = terminal.pathname
|
||||
.slice(
|
||||
terminal.pathname.indexOf(ROUTE_FRAGMENT) + ROUTE_FRAGMENT.length
|
||||
)
|
||||
.split('/')[1];
|
||||
return Object.freeze({
|
||||
counters: Object.freeze({
|
||||
[OPEN_SOURCE_JOURNEY_COUNTER.DOM_MUTATIONS]:
|
||||
renderer.counters.domMutations,
|
||||
[OPEN_SOURCE_JOURNEY_COUNTER.IPC_CALLS]: ipc.callsBeforeSentinel,
|
||||
// The whole journey runs inside the 500 ms window after the
|
||||
// click, so shifts flagged hadRecentInput are included.
|
||||
[OPEN_SOURCE_JOURNEY_COUNTER.LAYOUT_SHIFT_SCORE]: roundThousandth(
|
||||
renderer.counters.layoutShiftScore +
|
||||
renderer.counters.recentInputLayoutShiftScore
|
||||
),
|
||||
[OPEN_SOURCE_JOURNEY_COUNTER.LONG_TASKS]:
|
||||
renderer.counters.longTasks,
|
||||
[OPEN_SOURCE_JOURNEY_COUNTER.MOCK_HTTP_REQUESTS]:
|
||||
http.requests.length,
|
||||
}),
|
||||
evidence: Object.freeze({
|
||||
capabilities: renderer.capabilities,
|
||||
epochs: Object.freeze({
|
||||
click: start.epochMs,
|
||||
clickListener: start.listenerEpochMs,
|
||||
firstPage: terminal.epochMs,
|
||||
firstPagePaint: renderer.firstCardPaintEpochMs,
|
||||
mainIpcStart: ipc.start.receivedEpochMs,
|
||||
mainIpcSentinel: ipc.sentinel.receivedEpochMs,
|
||||
}),
|
||||
firstPage: Object.freeze({
|
||||
cardCount: terminal.cardCount,
|
||||
cardTag: terminal.cardTag,
|
||||
cardTestId: terminal.cardTestId,
|
||||
categoryCount: terminal.companionCounts[0] ?? null,
|
||||
pathname: terminal.pathname,
|
||||
section: section ?? null,
|
||||
}),
|
||||
httpRequestsBeforeClickByRoute: countJourneyMockRoutes(
|
||||
http.beforeClick
|
||||
),
|
||||
httpRequestsAfterSettledByRoute: countJourneyMockRoutes(
|
||||
http.afterSettled
|
||||
),
|
||||
httpRequestsByRoute: countJourneyMockRoutes(http.requests),
|
||||
httpRequestsToFirstPage: requestsToFirstPage,
|
||||
ipcCallsAfterFirstPage: ipc.callsAfterSentinel,
|
||||
ipcCallsByMethod: ipc.callsByMethod,
|
||||
layoutShift: Object.freeze({
|
||||
recentInput: roundThousandth(
|
||||
renderer.counters.recentInputLayoutShiftScore
|
||||
),
|
||||
withoutRecentInput: roundThousandth(
|
||||
renderer.counters.layoutShiftScore
|
||||
),
|
||||
}),
|
||||
longTaskDurationsMs: renderer.longTaskDurationsMs.map(roundTenth),
|
||||
settle,
|
||||
start: Object.freeze({
|
||||
pathname: start.pathname,
|
||||
targetTag: start.targetTag,
|
||||
targetTestId: start.targetTestId,
|
||||
}),
|
||||
}),
|
||||
index,
|
||||
pid: measurement.pid,
|
||||
wallClock: Object.freeze({
|
||||
[OPEN_SOURCE_JOURNEY_WALL_CLOCK.CLICK_TO_FIRST_PAGE]:
|
||||
roundTenth(clickToFirstPageMs),
|
||||
[OPEN_SOURCE_JOURNEY_WALL_CLOCK.CLICK_TO_FIRST_PAGE_PAINT]:
|
||||
roundTenth(paintEpochMs - start.epochMs),
|
||||
}),
|
||||
warmup,
|
||||
});
|
||||
}
|
||||
@@ -14,14 +14,14 @@ live in `tools/performance/`.
|
||||
| Journey | Start | End |
|
||||
| ---------------- | -------------------------------------------- | ----------------------------------------------------------------------------- |
|
||||
| J1 `launch` | Electron process spawn | first playlist or portal card rendered on `/workspace`, inline splash removed |
|
||||
| J2 `open-source` | click on a portal card | live category list and first channel page painted |
|
||||
| J2 `open-source` | click on the Xtream portal card | category list and first page of the opened section painted |
|
||||
| J3 `playback` | click on a channel | HTML5 `playing` event |
|
||||
| J4 `search` | six-character query typed into global search | results list settled |
|
||||
|
||||
J1 is instrumented today: `renderer.initialBytes` from the built output, and
|
||||
the runtime counters of the launch benchmark below. J2 to J4 follow the plan
|
||||
in `.plans/` and are added one thread at a time; each thread names its journey
|
||||
and counter in the PR description.
|
||||
J1 is instrumented: `renderer.initialBytes` from the built output, and the
|
||||
runtime counters of the launch benchmark below. J2 is instrumented by its own
|
||||
spec (below). J3 and J4 follow the plan in `.plans/` and are added one thread
|
||||
at a time; each thread names its journey and counter in the PR description.
|
||||
|
||||
## Running the journeys
|
||||
|
||||
@@ -39,7 +39,17 @@ first, starts the Xtream mock server on the dedicated loopback port
|
||||
dist/performance/journeys/<YYYYMMDDTHHMMSSZ>/summary.json
|
||||
```
|
||||
|
||||
The file is never overwritten; a second run in the same second fails instead.
|
||||
Every journey spec (`src/journeys/*.journey.ts`) adds its own
|
||||
`journeys.<id>` entry to that file. The config starts a run only in the
|
||||
Playwright runner (not in a worker, which has `TEST_WORKER_INDEX`): it sets
|
||||
`IPTVNATOR_JOURNEY_RUN_STARTED_AT` and a random `IPTVNATOR_JOURNEY_RUN_ID`
|
||||
before the worker forks, replacing any value left in the environment. All
|
||||
specs of one invocation, including a restarted worker, therefore share the
|
||||
directory and `harness.runId`. The first spec creates the file; a later one
|
||||
merges into it only when `harness.runId` matches and the rest of the harness
|
||||
is identical, through a temporary file and a rename. A journey that is
|
||||
already present fails, so no measurement is ever overwritten; a second
|
||||
invocation in the same second fails instead of merging into the first.
|
||||
`IPTVNATOR_JOURNEY_MEASURED_ITERATIONS` lowers the five measured iterations
|
||||
for a quick local check; the warm-up iteration always runs. Numbers from a
|
||||
laptop are previews: the Linux CI runner is the canonical measurer for
|
||||
@@ -96,7 +106,7 @@ main-process counters below, which exist only with `IPTVNATOR_PERF_CAPTURE=1`:
|
||||
|
||||
| Counter | Source |
|
||||
| ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `renderer.ipcCallsToFirstCard` | `start` trace events the preload emits for every bridge invocation (listener registrations `on*`/`remove*` excluded, as in `wrapElectronApi`). The renderer probe fires one sentinel `dbGetAppPlaylist('__iptvnator-journey-sentinel__')` at the terminal moment; renderer-to-main IPC is ordered, so events before the sentinel are the exact count. |
|
||||
| `renderer.ipcCallsToFirstCard` | `start` trace events the preload emits for every bridge invocation (listener registrations `on*`/`remove*` excluded, as in `wrapElectronApi`). The renderer probe fires one sentinel `cancelSourceProbe('__iptvnator-journey-sentinel__')` at the terminal moment; renderer-to-main IPC is ordered, so events before the sentinel are the exact count. The preload traces the call before forwarding it, and `SOURCE_HEALTH_CANCEL` only looks the id up in an in-memory map, so the sentinel never reaches the database worker. |
|
||||
| `renderer.domMutationsToFirstCard` | `MutationRecord`s (not callback batches) from a `MutationObserver` on the document element with `childList`, `attributes`, `characterData` and `subtree`. When the init script runs before `<html>` exists the observer watches `document`, which the blob reports in `capabilities.observedTarget`. |
|
||||
| `renderer.layoutShiftScore` | Sum of `layout-shift` entries with `hadRecentInput === false`, rounded to three decimals (a shift of 0.0001 flips in and out of the cutoff between runs; the CLS "good" threshold is 0.1, so three decimals keep the counter exact without hiding anything a user could see). The cutoff is sampled in a timer queued from the first `requestAnimationFrame` after the terminal batch, that is after the frame that paints the card has been committed; entries delivered live after the terminal batch are buffered and filtered by the same cutoff. |
|
||||
| `renderer.longTasks` | `longtask` entries over 50 ms up to that same cutoff, which includes the task that rendered the card. The count depends on machine speed, so it is evidence until a run shows it is stable on the CI runner. |
|
||||
@@ -110,8 +120,9 @@ registers the `performance:read-counters` IPC handler. Without the flag
|
||||
nothing is counted, no listener is attached and the handler does not exist;
|
||||
the preload never exposes the channel. SQL statements are counted only with
|
||||
`IPTVNATOR_PERF_COUNT_SQL=1` as well, because the hook wraps every statement
|
||||
execution: the journey sets both, while the M3U, refresh and Xtream
|
||||
benchmarks run with the capture flag alone and keep measuring unwrapped
|
||||
execution: the launch journey sets both (the flags are built in
|
||||
`journey-launch-environment.ts`), while J2's launches and the M3U, refresh
|
||||
and Xtream benchmarks do not set the SQL flag and keep measuring unwrapped
|
||||
statements. A harness test fails if any other source sets the SQL flag. After the renderer probe completes,
|
||||
`journey-main-counters.ts` calls the handler through `electronApp.evaluate`
|
||||
and the gate's tap.
|
||||
@@ -215,6 +226,7 @@ serial-depth counter is a better guardrail candidate than a raw call count.
|
||||
"platform": "darwin",
|
||||
"electron": "43.3.0",
|
||||
"measuredIterations": 5,
|
||||
"runId": "0b6f7f1e-…",
|
||||
"warmupIterations": 1
|
||||
},
|
||||
"journeys": {
|
||||
@@ -252,6 +264,108 @@ numbers so `tools/performance/check-journey-ratchet.mjs` can compare them with
|
||||
once its counter is deterministic on the CI runner; the launch counters are
|
||||
not yet (see [Ratchet](#ratchet)), so the summary is evidence only.
|
||||
|
||||
## J2 `open-source`: open a source to a browsable list
|
||||
|
||||
`open-source.journey.ts` reuses the J1 profile and process pattern: the
|
||||
profile is seeded once through the "Add playlist" dialogs, and every
|
||||
iteration copies it and spawns a fresh process through `runLaunchJourney`,
|
||||
which measures J1 as usual (gate, probe, IPC capture) and then hands the
|
||||
running app to `measureOpenSourceJourney` in
|
||||
`src/journeys/open-source-journey-app.ts`. The click therefore happens after
|
||||
J1's terminal condition and its counters are final, and the two journeys never
|
||||
overlap. One warm-up and five measured iterations, as for J1; the J1 numbers
|
||||
of these launches are not reported again. J2 does not read J1's
|
||||
main-process counters, so its launches run without `IPTVNATOR_PERF_CAPTURE`
|
||||
and `IPTVNATOR_PERF_COUNT_SQL` (`runLaunchJourney` with
|
||||
`mainCounters: false`). The click is not measured under the SQL hook that
|
||||
wraps every statement.
|
||||
|
||||
**Start.** The click on the dashboard card of the Xtream portal
|
||||
(`dashboard-recent-sources-rail-card` with the portal's name; the probe also
|
||||
accepts an `app-playlist-item` row on `/workspace/sources`). Before the
|
||||
click the test hovers the card and waits until the app has been quiet for
|
||||
1 s: no DOM mutation, no new bridge call, no new request to the mock, and
|
||||
neither a bridge call nor a mock request still in flight (30 s timeout, which
|
||||
fails the iteration). Bridge calls in flight come from J1's IPC capture: it
|
||||
was installed before the document loaded, and the preload follows every
|
||||
traced `start` with exactly one `success` or `error`, so a call that is still
|
||||
pending cannot resolve after the click and have its DOM changes or follow-up
|
||||
calls counted as J2. After settling, J1's capture is detached
|
||||
(`detachJourneyMainIpcCapture`), so its listener does not run for every
|
||||
bridge call of the measured click. The settle is a snapshot, and Playwright's
|
||||
actionability checks run between it and the click. The probe and the IPC
|
||||
capture keep counting pre-click activity until the click event itself, and
|
||||
the ledger splits at the click stamp. So the record rejects an iteration
|
||||
whose DOM mutations, bridge calls or mock requests moved after the snapshot
|
||||
(`open-source-journey-record-activity-before-click-*`). The settle wait and what happened during
|
||||
it are kept under `evidence.settle`. The renderer probe is armed in the
|
||||
loaded document with `page.evaluate` (the same self-contained script as J1,
|
||||
with `startClick` set). It registers a capture-phase `click` listener on
|
||||
`window`, which runs before every listener of the app. On the first click
|
||||
inside the start selector it stamps the start at the event's timestamp (or
|
||||
the listener's time if that is earlier) and sends the start sentinel
|
||||
`cancelSourceProbe('__iptvnator-journey-open-source-start__')`, the same
|
||||
no-op marker as J1's. Only then do
|
||||
the counters start.
|
||||
|
||||
**End.** The first `MutationObserver` batch after the start in which the
|
||||
path contains `/workspace/xtreams/`, an item of the first page is visible
|
||||
(`app-grid-list mat-card, .content-card, [data-test-id="channel-item"]`;
|
||||
skeleton cards do not match) and a category of the context panel is visible
|
||||
(`app-workspace-context-panel .category-item`). The probe then sends the end
|
||||
sentinel `cancelSourceProbe('__iptvnator-journey-open-source-end__')` and closes
|
||||
the observers at the same post-paint cutoff as J1. A portal card opens the
|
||||
source's default section, which is VOD (`getPlaylistLink` links to
|
||||
`/workspace/xtreams/<id>/vod`): the category list is the movie category list
|
||||
and the first page is the "All items" grid. The plan's "live category list"
|
||||
would need a second click and is not measured; the landed section is
|
||||
recorded under `evidence.firstPage.section`.
|
||||
|
||||
**HTTP requests to the mock.** The J2 profile is seeded with the origin of a
|
||||
loopback proxy in the test process
|
||||
(`src/performance/journey-mock-request-ledger.ts`) that forwards to the mock
|
||||
and records every request, so requests from the main process (Xtream API,
|
||||
M3U) and from the renderer (artwork served by the mock) are all counted. The
|
||||
default fixture's posters point at `picsum.photos`, so they are neither
|
||||
counted nor blocked: they load after the first page is painted, and on an
|
||||
offline runner they fail instead. Blocking them with `page.route` would put
|
||||
request interception on every renderer request, including the lazy chunks
|
||||
the journey loads. The mock's own
|
||||
`/__control/state` ledger is not used: it exists only in performance-control
|
||||
mode, which disables `/playlist.m3u` and tracks only the 100k scenario, and a
|
||||
Playwright request listener would see renderer traffic only. The ledger stores
|
||||
the method, the path and, for `player_api.php`, the `action` parameter; query
|
||||
strings and stream paths carry credentials and are never stored.
|
||||
|
||||
### Counters
|
||||
|
||||
| Counter | Source |
|
||||
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `renderer.ipcCallsToFirstPage` | Bridge `start` trace events between the start and end sentinels, counted by a second `journey-main-ipc-capture.ts` instance installed with `startSentinelId`. Calls before the start marker are tallied separately (`callsBeforeStart`); a start marker that is missing, repeated or received after the end sentinel fails the iteration. |
|
||||
| `renderer.domMutationsToFirstPage` | `MutationRecord`s from the click until the terminal batch. Records produced before the click (hover, settling) are taken from the observer at the start and counted under `evidence.settle` instead. |
|
||||
| `renderer.layoutShiftScore` | Sum of all `layout-shift` entries from the click until the post-paint cutoff, rounded to three decimals. Unlike J1 it includes entries with `hadRecentInput === true`: the journey is a response to the click and runs inside the 500 ms input window, so the CLS filter would always read 0. The split is under `evidence.layoutShift`. |
|
||||
| `renderer.longTasks` | `longtask` entries over 50 ms whose time range overlaps the window from the click to the cutoff. The task that dispatches the click began before the event's timestamp and still counts; buffered J1 tasks that ended before the click are dropped. Evidence until it is shown to be stable on the CI runner, as for J1. |
|
||||
| `main.mockHttpRequestsToSettled` | Requests the proxy received from the click until, after the terminal batch, no new request had arrived for 1 s and none was in flight (a response slower than that, and what it triggers, stays inside the window). The window ends at the ledger position read by that accepted quiet sample; a request arriving after it was never seen in flight, so it goes to `evidence.httpRequestsAfterSettledByRoute` instead of the counter. The ledger is read 1 s after that sample, so that late traffic is actually observed. The window starts at the renderer's click stamp, the same boundary as every other J2 counter, not when Playwright began its actionability checks; the proxy stamps requests with the test process's wall clock, and both processes read the same host clock. Bounding by the terminal would compare the test process's clock with the renderer's, so the count up to the terminal epoch is evidence only (`evidence.httpRequestsToFirstPage`); `evidence.httpRequestsByRoute` names the requests. |
|
||||
|
||||
Two counters are listed under `unavailable`. `renderer.cdTicksToFirstPage`
|
||||
is missing for the same reason as its J1 counterpart.
|
||||
`main.sqlStatementsToFirstPage` is missing because the running
|
||||
`main.sqlStatements` total that J1 freezes at `ready-to-show` can only be
|
||||
read from the test process through the journey gate. It therefore cannot be
|
||||
sampled at the click or at the first-page batch, and the worker's count is
|
||||
ordered against its responses, not against the renderer. Reading it after
|
||||
the app has settled before the click and again after the first page would
|
||||
give a click-to-settled count; that is left to a follow-up.
|
||||
|
||||
### Wall-clock
|
||||
|
||||
| Entry | Derivation |
|
||||
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `clickToFirstPageMs.p50/.p90` | Terminal epoch minus start epoch: the click until the batch that made the category list and first page visible, the same boundary J1's `spawnToFirstCardMs` uses. |
|
||||
| `clickToFirstPagePaintMs.p50/.p90` | Post-paint cutoff minus start epoch: the click until the frame that paints the first page has been committed (the timer queued from the next `requestAnimationFrame`). This is the "painted" figure of the journey definition. |
|
||||
|
||||
All epochs are taken in the renderer, so neither entry crosses a process clock.
|
||||
|
||||
## `renderer.initialBytes`
|
||||
|
||||
The bytes a browser fetches before Angular can bootstrap, read from the built
|
||||
@@ -492,17 +606,22 @@ reports slow imports of non-Latin playlists.
|
||||
1. Add `apps/electron-backend-e2e/src/journeys/<journey>.journey.ts`. Seed the
|
||||
profile through the app's dialogs, spawn a fresh process per iteration
|
||||
with `measureLaunchJourney` as the model, and drive the journey's start
|
||||
action with Playwright.
|
||||
2. Give the journey its own probe options (`cardSelector`, `routeFragment`,
|
||||
terminal condition) or extend `journey-renderer-probe.ts` when the end
|
||||
condition is not "an element became visible". Keep the probe
|
||||
self-contained: Playwright serializes it with `toString()`.
|
||||
action with Playwright. A journey that starts inside the running app
|
||||
continues from J1 with `runLaunchJourney` and lets the app settle first,
|
||||
as `open-source-journey-app.ts` does.
|
||||
2. Give the journey its own probe options (`cardSelector`,
|
||||
`companionSelectors`, `routeFragment`, `startClick` for a click start) or
|
||||
extend `journey-renderer-probe.ts` when the end condition is not "elements
|
||||
became visible". Use a state key and sentinel ids of its own. Keep the
|
||||
probe self-contained: Playwright serializes it with `toString()`.
|
||||
3. Map the measurement to a `JourneyIterationRecord` in a
|
||||
`<journey>-journey-record.ts` under `src/performance/`; name counters
|
||||
`renderer.*` or `main.*`, and list counters you cannot measure under
|
||||
`unavailable` with the reason.
|
||||
4. Add the journey under `journeys.<id>` in the summary through
|
||||
`summarizeJourneyIterations`; the schema needs no change.
|
||||
4. Add the journey under `journeys.<id>` in the run's summary with
|
||||
`writeJourneyRunEntry` (`src/journeys/journey-run.ts`), which calls
|
||||
`summarizeJourneyIterations` and merges the entry; the schema needs no
|
||||
change.
|
||||
5. Cover the probe with jsdom fixtures and the record and summary code with
|
||||
`node:test` (`pnpm nx run electron-backend-e2e:test-performance-harness`).
|
||||
6. Validate a counter before it becomes a guardrail: one PR must show that
|
||||
|
||||
@@ -248,7 +248,7 @@ pnpm nx build web
|
||||
pnpm run perf:initial-bytes # breakdown only
|
||||
pnpm run perf:initial-bytes:check # measure, then compare with the committed baseline
|
||||
pnpm nx test performance-tools
|
||||
pnpm run perf:journeys # J1 launch benchmark, writes dist/performance/journeys/<timestamp>/summary.json
|
||||
pnpm run perf:journeys # J1 launch + J2 open-source journeys, one dist/performance/journeys/<timestamp>/summary.json
|
||||
```
|
||||
|
||||
`perf:initial-bytes` reads the built `dist/apps/web/index.html` and sums the
|
||||
@@ -257,8 +257,10 @@ bytes on the initial path (the J1 counter `renderer.initialBytes`).
|
||||
`tools/performance/journey-baselines.json`; baselines only move down. CI runs
|
||||
the same check in the `Initial bytes ratchet` job of `ci.yml` for PRs that
|
||||
target `master` and for `master` pushes (dispatch it with
|
||||
`gh workflow run ci.yml --ref <branch>` for a stacked branch). `perf:journeys` builds the `electron-performance` configuration and runs the
|
||||
J1 launch benchmark against the Xtream mock; its probe specs run with
|
||||
`gh workflow run ci.yml --ref <branch>` for a stacked branch). `perf:journeys` builds the `electron-performance` configuration and runs every
|
||||
journey spec against the Xtream mock: J1 launch, then J2 open-source (a
|
||||
second set of launches, each followed by the click on the portal card), both
|
||||
written to the same summary file; its probe specs run with
|
||||
`pnpm nx run electron-backend-e2e:test-performance-harness`. The contract, what
|
||||
counts and how to add a counter or a journey are in the
|
||||
[performance journeys](performance-journeys.md) document.
|
||||
|
||||
Reference in new issue
Block a user