test(performance): harden formal capture validity

This commit is contained in:
4gray committed 2026-07-27 10:05:49 +02:00
1 parent 5891ac0673
commit 77e8fd92e3
14 files changed
+1127 -100

No files matched your search

+13 -4
View File
@@ -89,9 +89,10 @@ Smoke runs may use a dirty worktree and validate only the harness; they cannot
support a performance claim. If another local application owns port 9222, smoke
only may set `IPTVNATOR_PERF_CDP_PORT` to an unused loopback port; formal runs
fail closed unless CDP uses `127.0.0.1:9222`. Raw captures and JSON results stay
under the gitignored `dist/performance/` tree. Headline distributions contain
only the five measured runs: warm-up and diagnostic profiles are never mixed
into them.
under the gitignored `dist/performance/` tree; preflight rejects a symbolic
link in any existing output-path component before creating artifacts. Headline
distributions contain only the five measured runs: warm-up and diagnostic
profiles are never mixed into them.
The benchmark attributes these non-additive intervals:
@@ -129,7 +130,15 @@ must not scan or log playlist payloads to generate metadata. Renderer
long-task, frame-gap, and heartbeat samples are clipped to the measured
operation boundary. Formal comparison fails closed after writing raw results
if exact-window renderer RSS, database-worker peak heap/external samples, or
the database worker's explicit post-GC heap is missing or incoherent.
the database worker's explicit post-GC heap is missing or incoherent. Both
initial-import database requests in every measured run must also contain
coherent event-loop delay, event-loop utilization, and thread-CPU metrics;
the validity record exposes the exact expected and valid request counts rather
than silently dropping nullable samples. An iteration failure idempotently
stops renderer timers, closes trace listeners/output, and starts best-effort
probe/session teardown without waiting on a wedged renderer before Electron is
closed. A partial capture-start failure performs the same rollback before it
escapes to the benchmark lifecycle.
Diagnostic artifacts require separate renderer, main, and database-worker CPU
profiles plus renderer/main/database-worker heap snapshots and a Chromium
trace; raw profiles remain ignored.
+7
View File
@@ -172,6 +172,13 @@ explicit sum is `ipcStructuredCloneProxyMs`. See
[M3U Playlist Module Architecture](./m3u-playlist-module.md#initial-url-import-performance-benchmark-electron)
for the complete cross-process attribution.
Formal initial-import comparison also requires both request-scoped captures in
every measured run to have coherent event-loop delay, event-loop utilization,
and worker-thread CPU values with no unavailable or invalid reason. Summary
validity records the exact expected and valid request counts; nullable metrics
remain in raw results but cannot be silently omitted from comparison
distributions.
The main-process benchmark samples the database worker's V8
`used_heap_size` and `external_memory` independently. Raw output includes a
valid-sample count for each metric. A peak is numeric only after at least one