mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-08 17:06:15 -08:00
feat(embedded-mpv): frame-copy rendering engine (experimental, macOS Apple Silicon) (#1169)
* spike(embedded-mpv): frame-copy pipeline prototype (helper + shm ring + Electron viewer) Standalone macOS spike for the frame-copy unification direction from the 2026-07-10 analysis: a helper process renders mpv offscreen into a GL FBO, reads frames back through an async PBO ring, and publishes BGRA frames into a 3-slot POSIX shm seqlock ring; a minimal Electron viewer copies the newest frame via a plain-C N-API addon and uploads it to a WebGL canvas per rAF. First numbers on M1 Pro (see spike README): 4K60 HEVC hwdec sustained at 60 fps end to end, ~1.2 ms shm copy + ~3.5 ms texture upload, ~10 ms produce-to-upload age, zero torn frames. Remaining gates: weak hardware, long-run pacing, HDR, latency flash test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * spike(embedded-mpv): add RESULTS.md measurement log with M1 Pro baseline Structured per-machine table with repro commands so the pending Intel Mac and Windows iGPU runs can be appended and compared one-to-one. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * spike(embedded-mpv): pacing/judder instrumentation + 50/25 fps and HDR gate results Viewer now measures inter-frame intervals on both clocks (present side and producer side): stddev/p99/max, late-frame counters vs the producer's median interval, and a cumulative LONGRUN summary every 30 s. The addon exposes the producer timestamp (produceMs) for this. Measured on M1 Pro: 50 fps and 25 fps cadences are clean (late frames only at startup; residual jitter is 120 Hz rAF grid quantization, bounded by one display tick), and 4K25 HDR10 PQ/BT.2020 is tonemapped to SDR by mpv before readback at full rate with unchanged copy costs. RESULTS.md carries the tables and HDR-clip repro commands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * spike(embedded-mpv): record 10-minute 4K60 HEVC long-run results Zero dropped frames and zero torn reads after the first-minute warmup over ~8.5 minutes; steady-state late frames (~0.4%) track the 12 s test clip's --loop restarts, not the copy pipeline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * spike(embedded-mpv): viewport-scaling measurement + integration design draft Confirms the render-at-viewport-size claim (4K source in a 720p FBO costs 720p: 0.17 ms readback / 0.16 ms copy / 0.17 ms upload at 60 fps) and adds DESIGN.md — the draft integration architecture: per-session helper process linking bundled libmpv on all platforms (finally full-featured + Wayland- agnostic Linux), JSON-over-stdio control evolving the Linux wid protocol, unchanged EmbeddedMpvSession renderer contract, shm generations for resize, packaging via the existing vendored-runtime tooling, rollout behind its own flag with the docked path as default. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * spike(embedded-mpv): auto-detect Homebrew prefix and Node headers for Intel Macs BREW_PREFIX was hardcoded to /opt/homebrew (Apple Silicon) and NODE_INC to one nvm version; both now resolve via brew --prefix and the PATH node's execPath, so the pending Intel Mac run needs no Makefile edits. README gets a fresh-machine checklist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * spike(embedded-mpv): self-contained measurement bundle for machines without Node/pnpm make-bundle.sh assembles a tarball with the spike sources, vendored N-API headers (Makefile prefers them when present, so no Node install is needed), pre-generated 4K HEVC/HDR10 test clips, and an official Electron dist download for the target arch. collect-results.sh builds and runs the full RESULTS.md scenario suite automatically (plus an optional --long 10-minute run) and writes one results-<host>-<date>.txt to send back. Target-machine prerequisites shrink to Xcode CLT + brew mpv — built for the pending Intel Mac baseline run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * spike(embedded-mpv): support MacPorts libmpv and legacy-macOS bundles Makefile and collect-results.sh now detect libmpv in the Homebrew prefix or MacPorts /opt/local (Homebrew is unsupported on legacy macOS like High Sierra; 'sudo port install mpv +libmpv' provides libmpv there). make-bundle takes ELECTRON_VERSION/BUNDLE_SUFFIX overrides — Electron 27+ needs macOS 10.15, so High Sierra bundles ship Electron 26.6.10 (LSMinimumSystemVersion 10.13). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * spike(embedded-mpv): scope macOS frame-copy engine to Apple Silicon only Owner decision 2026-07-10: skip Intel Mac measurements and gate the future frame-copy engine on arm64. Intel Macs able to run the app at all are a shrinking 2015-2020 cohort and keep the docked/external/web player paths; the macOS hardware gate closes with the M1 Pro numbers, and remaining hardware risk moves to the Windows/Linux ports. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(embedded-mpv): frame-copy helper process and shm frame reader (native layer) iptvnator_mpv_helper: one-process-per-session libmpv host that renders offscreen at viewport size (headless CGL + async PBO ring, validated in spikes/mpv-frame-copy), publishes BGRA frames into a seqlock shm ring with resize generations, plays audio directly, and speaks a stdio protocol — tab-separated commands in, JSON events out. The snapshot event mirrors NativeEmbeddedMpvSessionSnapshot; status semantics (END_FILE reasons, eof-reached with keep-open, pause gated on loaded path, fatal-only status flips) are ported from embedded_mpv.mm. embedded_mpv_frame_reader.node: plain-C N-API reader the preload script uses to memcpy the newest complete frame into a V8 ArrayBuffer (Electron's memory cage forbids zero-copy). Stub exports off macOS. Both build as extra binding.gyp targets through build-embedded-mpv.js; the helper gets the same libmpv dependency-path rewrite + ad-hoc re-sign as the addon and is validated by the forbidden-link check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(embedded-mpv): frame-copy engine wiring in main process and preload EmbeddedMpvFrameCopyAdapter implements the NativeEmbeddedMpvAddon surface over a per-session helper process (spawn, stdio protocol, snapshot cache, graceful quit->SIGTERM->SIGKILL teardown), so EmbeddedMpvNativeService reuses its polling/diff/power-blocker/recording logic unchanged. The IPTVNATOR_ENABLE_EMBEDDED_MPV_FRAME_COPY flag (darwin/arm64 only) routes getAddon() to the adapter and reports engine: 'frame-copy' in support. The preload frame pump loads the shm reader addon, copies the newest frame once per rAF into a reused buffer, and uploads it to WebGL2 on the renderer's canvas — no frame data crosses the contextBridge; the bridge only gains attachEmbeddedMpvFrameView/detachEmbeddedMpvFrameView. The experiment flag relaxes the window sandbox for that native require; contextIsolation and nodeIntegration:false stay on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(embedded-mpv): frame-copy canvas mode in the player component + docs EmbeddedMpvPlayerComponent renders <canvas data-embedded-mpv-frame> when support reports engine 'frame-copy' and the session controller starts/stops the preload frame pump around the session lifecycle. The bounds provider skips HIDDEN_BOUNDS and the popover cutout for this engine — the canvas is ordinary DOM, dialogs and popovers stack above it natively; bounds sync still drives the helper's render size. Adapter unit tests cover spawn args, snapshot caching, shm generations, protocol encoding, unexpected-exit mapping, and dispose escalation. Architecture doc and CLAUDE.md describe the engine, its flag, and the sandbox trade-off. Verified end to end in the built app (M1 Pro): engine detection, helper spawn, lavfi playback onto the canvas via CDP-injected smoke — including an orientation fix (helper FLIP_Y already yields texture-order rows; the pump shader must not flip uv again). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(embedded-mpv): close helper stdin on dispose + lifecycle logging Live testing surfaced a stray idle helper that survived a session switch; until the root cause is pinned down, dispose now also closes the child's stdin (the helper exits on EOF) as a second kill path besides quit -> SIGTERM -> SIGKILL, and spawn/dispose/exit are logged with the session id so leaks are attributable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(embedded-mpv): reap sessions when the renderer reloads or crashes Root cause of the stray idle helper found during live testing: session teardown lives in the renderer's Angular lifecycle, which never runs on a renderer crash or hard reload — the main process kept the session (and its frame-copy helper process / native mpv handle) alive until app shutdown. EmbeddedMpvNativeService now watches the main window's webContents for render-process-gone and did-navigate (full reloads only; in-app Angular routing emits did-navigate-in-page) and disposes every session. Applies to both engines. Verified live: location.reload() during frame-copy playback logs 'Disposing 1 session(s): renderer reloaded' and the helper exits cleanly. Regression test drives both events against the service. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(embedded-mpv): Settings toggle for the frame-copy engine New 'Embedded MPV: frame-copy engine' checkbox in Settings > Playback, shown only when the machine can run it (macOS arm64 with the helper binary present — support now reports frameCopyAvailable). The choice persists to the main-process config store because the engine relaxes the window sandbox for the preload frame pump, which is fixed at window creation: main.ts reads the store before creating the window and sets the engine env var; an explicitly set env var (including '0') always wins, and the UI shows a restart hint while the saved choice differs from the active engine. Localized in all 18 locales. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(embedded-mpv): aspect-fit rendering in the frame-copy helper The helper now observes dwidth/dheight and renders its FBO at the aspect-fit size of the video inside the requested viewport, bumping a shm generation on change — letterbox bars are never baked into frames (the VOD watch shell's ~2:1 box no longer shows black side bars; the canvas background is transparent so the sides show the app surface, while fullscreen keeps its black backdrop). Frames also get smaller than the viewport when aspects differ, trimming copy cost. Aspect override changes refit automatically. Snapshots now carry videoWidth/videoHeight, and the adapter forwards IPTVNATOR_EMBEDDED_MPV_AUDIO_DELAY to mpv's audio-delay for lip-sync tuning until proper calibration lands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(embedded-mpv): require frame-copy artifacts in macOS package validation + docs macOS packages that ship embedded_mpv.node must also ship the iptvnator_mpv_helper binary and the embedded_mpv_frame_reader.node addon — they come out of the same binding.gyp run, and a package missing them would silently lose the frame-copy engine. Covered in the package-identity test. Architecture doc and CLAUDE.md document the Settings toggle, aspect-fit rendering, audio-delay passthrough, and the renderer-reload session reaping. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(electron): inline TS helpers so the sandboxed preload keeps working The frame pump's async/await (target es2015 + importHelpers) made webpack externalize tslib in main.preload.js. Sandboxed preloads can only require Electron's built-in module whitelist, so the entire preload script failed to load and window.electron disappeared for every run without the frame-copy flag. importHelpers:false for electron-backend keeps the preload bundle self-contained — and future async code in preload can no longer silently reintroduce the breakage. Verified live: sandboxed run now has the bridge, reports engine 'native' and frameCopyAvailable true. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(embedded-mpv): commit the frame-copy analysis handoff + source inventory The 2026-07-10 analysis that led to this branch now lives next to the spike (spikes/mpv-frame-copy/ANALYSIS.md), and the architecture doc's What To Commit section lists the frame-copy engine sources. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(embedded-mpv): address review findings on the frame-copy engine - Stale pump attach can no longer win over a newer session: attach/detach bump a shared epoch and async attach waits re-check it after every await, so an attach for a replaced session aborts instead of installing itself (greptile P1). - A failed frame-view attach (no canvas, no WebGL2, reader missing) now disposes the session and surfaces the error UI instead of leaving audio playing behind a black canvas (codex P2). - A stale frame-copy opt-in without the helper binary falls back to the native engine instead of reporting embedded MPV unsupported, and the Settings checkbox stays visible while a saved opt-in exists so it can always be cleared (codex P2). Regression test covers the fallback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(packaging): make the darwin frame-copy packaging test host-agnostic On non-macOS CI hosts validatePackagedEmbeddedMpv also reports that macOS link validation needs a macOS host, so the success-path assertion now checks only the frame-copy artifact requirement instead of expecting an empty error list. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(embedded-mpv): Windows/Linux porting handoff for the frame-copy engine Self-contained entry point for porting sessions on other machines: current state and coordination constraints, per-OS task lists (Linux EGL first, then Windows WGL + named shm — the decisive iGPU perf gate), the hard-won gotchas from the macOS integration (preload/tslib sandbox breakage, V8 memory cage, frame orientation, stale-attach epoch, dispose escalation, node-gyp naming, snapshot protocol semantics), testing recipes, and the suggested milestone order. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(embedded-mpv): branching and merge strategy in the porting handoff Port work goes to stacked branches off the frame-copy branch (PR base = frame-copy branch, sequential merges, stack depth one), never into the frozen PR #1169 branch itself. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(embedded-mpv): drop stale uncommitted note from porting handoff Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(embedded-mpv): harden frame-copy helper startup --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
eb1bfaa474
commit
26271fc076
80 files changed
+5963
-89
No files matched your search
@@ -18,6 +18,15 @@ Source files for the embedded MPV integration:
|
||||
- `libs/shared/interfaces/src/lib/embedded-mpv-session.interface.ts` defines the shared session and audio-track contract.
|
||||
- `libs/ui/playback/src/lib/embedded-mpv-player/` owns the Angular UI and controls.
|
||||
|
||||
Frame-copy engine sources (experimental, macOS Apple Silicon — see the
|
||||
"Frame-Copy Engine" section below):
|
||||
|
||||
- `apps/electron-backend/native/helper/` — `iptvnator_mpv_helper` process (`mpv_frame_helper.cpp`, `frame_helper_render.h`, `frame_helper_io.h`, `frame_shm.h`).
|
||||
- `apps/electron-backend/native/src/embedded_mpv_frame_reader.c` — N-API shm frame reader used by the preload frame pump.
|
||||
- `apps/electron-backend/src/app/services/embedded-mpv-frame-copy.adapter.ts` — helper-process adapter behind the `NativeEmbeddedMpvAddon` surface.
|
||||
- `apps/electron-backend/src/app/api/embedded-mpv-frame-pump.ts` — preload frame pump (shm → WebGL canvas).
|
||||
- `spikes/mpv-frame-copy/` — standalone spike, measurement log (RESULTS.md), integration design (DESIGN.md), and the original analysis (ANALYSIS.md).
|
||||
|
||||
Generated native-addon build output:
|
||||
|
||||
- `apps/electron-backend/native/build/`
|
||||
@@ -89,6 +98,104 @@ The MPV video surface is a native platform view/window, not a normal DOM element
|
||||
|
||||
The dock has a stable reserved height while embedded controls are enabled. Controls fade in and out inside that fixed dock, so normal show/hide behavior does not resize the native MPV viewport or make the video jump. Volume and audio-track panels replace the default transport controls inside the same dock and provide a back button to return to the default controls. Popovers and menus must stay inside that dock unless the native layering strategy changes. The native MPV view deliberately ignores hit testing so mouse movement passes through to Chromium and can reveal Angular controls even when the pointer moves quickly across the video area.
|
||||
|
||||
## Frame-Copy Engine (Experimental, Apple Silicon Only)
|
||||
|
||||
`IPTVNATOR_ENABLE_EMBEDDED_MPV_FRAME_COPY=1` (on top of the regular
|
||||
embedded MPV experiment flag) switches macOS/arm64 to a second rendering
|
||||
engine that replaces the native-view compositing entirely:
|
||||
|
||||
- `apps/electron-backend/native/helper/` — `iptvnator_mpv_helper`, a
|
||||
one-process-per-session libmpv host. It decodes (hwdec), renders
|
||||
offscreen at viewport size (headless CGL + async PBO readback ring),
|
||||
publishes BGRA frames into a POSIX shm seqlock ring
|
||||
(`frame_shm.h`, 3 slots, resize creates a new `-g<N>` generation), and
|
||||
plays audio directly. Control protocol: tab-separated commands on stdin,
|
||||
JSON events on stdout; the `snapshot` event mirrors
|
||||
`NativeEmbeddedMpvSessionSnapshot`. Status semantics are ported from
|
||||
`embedded_mpv.mm`.
|
||||
- `apps/electron-backend/src/app/services/embedded-mpv-frame-copy.adapter.ts` —
|
||||
implements the same `NativeEmbeddedMpvAddon` surface over the helper
|
||||
process, so `EmbeddedMpvNativeService` (polling, diffing, power blocker,
|
||||
recording paths) is reused unchanged. The flag routes `getAddon()` to the
|
||||
adapter and support reports `engine: 'frame-copy'`.
|
||||
- `apps/electron-backend/native/src/embedded_mpv_frame_reader.c` — N-API
|
||||
shm reader loaded by the preload frame pump
|
||||
(`apps/electron-backend/src/app/api/embedded-mpv-frame-pump.ts`): copy
|
||||
the newest complete frame into a reused ArrayBuffer once per rAF and
|
||||
upload it to a WebGL2 texture on the renderer's
|
||||
`<canvas data-embedded-mpv-frame>` (BGRA swizzle in the shader). Frame
|
||||
copies whose post-copy seqlock check reports a writer race are discarded
|
||||
without advancing the consumed sequence, so the next rAF retries instead
|
||||
of uploading partial pixels. Frame data never crosses the contextBridge;
|
||||
the bridge only exposes
|
||||
`attachEmbeddedMpvFrameView`/`detachEmbeddedMpvFrameView`.
|
||||
- Renderer: `EmbeddedMpvPlayerComponent` renders the canvas when
|
||||
`support.engine === 'frame-copy'` and skips the compositor workarounds —
|
||||
no `HIDDEN_BOUNDS` when dialogs open, no popover bottom cutout; dialogs
|
||||
and controls stack above the canvas as ordinary DOM. Bounds sync still
|
||||
runs: the helper re-renders at the new viewport size (device pixels via
|
||||
the display scale factor), including a forced current-frame render when a
|
||||
paused resize creates a fresh shared-memory generation.
|
||||
|
||||
Enabling it: the `Settings > Playback > Embedded MPV: frame-copy engine`
|
||||
checkbox (shown only when support reports `frameCopyAvailable`) persists to
|
||||
the main-process config store (`electron-conf`), which `main.ts` reads
|
||||
before creating the window and translates into the env flag; an explicitly
|
||||
set env var (including `0`) wins over the stored preference, but cannot bypass
|
||||
the platform/runtime safety gate. Frame-copy can relax the window sandbox only
|
||||
when embedded MPV itself is enabled for the current run (packaged app or the
|
||||
regular development experiment flag) and discovery finds both an executable
|
||||
(`X_OK`) helper and a readable regular frame-reader addon in the same native
|
||||
directory. Packaged discovery is limited to packaged resource locations and
|
||||
never falls through to writable cwd/dist development paths. A disabled base
|
||||
experiment keeps the renderer sandbox enabled and embedded MPV unavailable.
|
||||
When the base feature is enabled, a missing, mode-stripped, or incomplete
|
||||
frame-copy runtime keeps the sandbox enabled and falls back to the native
|
||||
engine.
|
||||
Changing the toggle requires an app restart because web preferences are fixed
|
||||
at window creation.
|
||||
|
||||
Rendering size: the helper renders at the **aspect-fit** size of the video
|
||||
(observed `dwidth`/`dheight`) inside the requested viewport and bumps a shm
|
||||
generation when it changes — letterbox bars are never baked into frames,
|
||||
frames stay as small as possible, and the canvas letterboxes with a
|
||||
transparent background (app surface shows at the sides; fullscreen keeps a
|
||||
black backdrop). Snapshots carry `videoWidth`/`videoHeight`.
|
||||
`IPTVNATOR_EMBEDDED_MPV_AUDIO_DELAY=<seconds>` passes through to mpv's
|
||||
`audio-delay` for lip-sync tuning until a calibration flow exists.
|
||||
|
||||
Lifecycle safety: `EmbeddedMpvNativeService` watches the main window for
|
||||
`render-process-gone` and `did-navigate` (full reloads) and disposes every
|
||||
session — Angular teardown never runs on a renderer crash/hard reload, and
|
||||
without the watch helper processes (or native mpv handles) would leak until
|
||||
app shutdown. Unexpected helper exits surface as a session `error`. macOS
|
||||
package validation requires `iptvnator_mpv_helper` and
|
||||
`embedded_mpv_frame_reader.node` next to the addon whenever the addon
|
||||
ships. The after-pack hook restores the helper's executable mode after the
|
||||
asset copy, and optional/skipped native rebuilds remove stale helper/reader
|
||||
artifacts before reporting frame-copy availability. This cleanup prevents
|
||||
known leftover build output; it is not a compatibility check for a complete
|
||||
but version-mismatched runtime pair.
|
||||
|
||||
Trade-offs and constraints:
|
||||
|
||||
- The frame-copy experiment flag can relax the BrowserWindow sandbox only
|
||||
while the base embedded-MPV feature is enabled (preload must
|
||||
`require` the reader addon); `contextIsolation` and
|
||||
`nodeIntegration:false` stay on. The sandbox story must be revisited
|
||||
before this engine can become a default — candidates: utilityProcess +
|
||||
MessagePort (costs one extra copy + GC churn since Electron ports clone
|
||||
ArrayBuffers) or a WebCodecs-based path.
|
||||
- Scope: Apple Silicon only by owner decision (2026-07-10); Intel Macs
|
||||
keep the native-view engine. Windows/Linux ports of the helper (WGL/EGL)
|
||||
are future work — the shm protocol and adapter are platform-agnostic.
|
||||
- Measured baseline (M1 Pro, spikes/mpv-frame-copy/RESULTS.md): 4K60 HEVC
|
||||
sustained end to end, ~1.2 ms shm copy + ~3.5 ms texture upload, ~10 ms
|
||||
produce-to-upload latency, zero torn frames over a 10-minute run.
|
||||
- Helper crash isolation: an unexpected helper exit surfaces as a session
|
||||
`error` (renderer falls back); it can never take down the Electron main
|
||||
process, unlike in-process libmpv.
|
||||
|
||||
## Resume And Track Handling
|
||||
|
||||
`ResolvedPortalPlayback.startTime` is treated as a media offset in seconds for VOD and episodes. The native addon passes it as the `start` option in one MPV `loadfile` options map together with title, user agent, referrer, and HTTP headers.
|
||||
|
||||
Reference in new issue
Block a user