docs(embedded-mpv): document the Linux frame-copy port

- architecture doc: frame-copy section covers Linux (EGL display tiers,
  build deps, package strip), Linux support matrix notes the frame-copy
  exception to the X11 + system-mpv requirements, Linux measured baseline.
- RESULTS.md: Ubuntu 25.04 / i7-1165G7 (Iris Xe) measurement rows via the
  production helper + reader probe; viewport-size claim reproduced.
- PORTING.md: Linux marked done with pointers to what changed; Windows
  remains the open port and its perf gate the open decision.
- CLAUDE.md + tools/embedded-mpv/README.md: platform scope, Linux dev
  build requirements, system-headers fallback, helper strip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
4grayandClaude Fable 5 authored and 4gray committed 2026-07-15 19:52:41 +02:00
1 parent 4316f4d95d
commit c17666ec49
5 files changed
+83 -22

No files matched your search

+1 -1
View File
@@ -617,7 +617,7 @@ This project uses modern Angular signal-based APIs and patterns. **ALWAYS** use
- Built-in HTML5 player with HLS.js or Video.js
- External players: MPV, VLC (via IPC to Electron backend)
- Embedded MPV (experimental, macOS/Windows/Linux): renders mpv video inside the Electron window through a native addon. macOS uses the libmpv render API in an `NSOpenGLView`; Windows uses in-process libmpv with `--wid` against an app-owned child `HWND`; Linux spawns an out-of-process `mpv --wid=<x11-window>` controlled over a JSON IPC socket (X11/XWayland only, requires system `mpv` on PATH; subtitles/speed/aspect/recording are not exported there). mpv's own screensaver inhibition does not apply to any of these paths, so `EmbeddedMpvNativeService` holds an Electron `powerSaveBlocker` (`prevent-display-sleep`) whenever any session's status is `playing`, and releases it on pause, dispose, or shutdown. Service: `apps/electron-backend/src/app/services/embedded-mpv-native.service.ts`; full architecture: `docs/architecture/embedded-mpv-native.md`.
- Embedded MPV frame-copy engine (experimental, macOS Apple Silicon only; enabled via `Settings > Playback > Embedded MPV: frame-copy engine` (restart required) or `IPTVNATOR_ENABLE_EMBEDDED_MPV_FRAME_COPY=1` on top of the embedded MPV experiment flag): a per-session `iptvnator_mpv_helper` process renders mpv offscreen at viewport size and publishes BGRA frames into a shm ring; the preload frame pump uploads them onto a renderer `<canvas data-embedded-mpv-frame>`, so controls/dialogs are ordinary DOM above the video (no native-surface compositing workarounds; the flag relaxes the window sandbox for the preload's native reader addon). Stored and explicit opt-ins relax the sandbox only while the base embedded-MPV feature is enabled and a platform-supported runtime contains both an executable helper and readable regular frame-reader addon; packaged discovery is restricted to packaged resources. A disabled base experiment keeps embedded MPV unavailable with the sandbox intact, while a missing, mode-stripped, or incomplete frame-copy runtime falls back to the native engine without relaxing the sandbox. Adapter: `apps/electron-backend/src/app/services/embedded-mpv-frame-copy.adapter.ts`; helper: `apps/electron-backend/native/helper/`; details in `docs/architecture/embedded-mpv-native.md` ("Frame-Copy Engine").
- Embedded MPV frame-copy engine (experimental, macOS Apple Silicon + Linux; enabled via `Settings > Playback > Embedded MPV: frame-copy engine` (restart required) or `IPTVNATOR_ENABLE_EMBEDDED_MPV_FRAME_COPY=1` on top of the embedded MPV experiment flag): a per-session `iptvnator_mpv_helper` process renders mpv offscreen at viewport size (headless CGL on macOS, headless EGL on Linux — no window embedding, so native Wayland works and the X11/system-mpv requirements of the native Linux engine do not apply) and publishes BGRA frames into a shm ring; the preload frame pump uploads them onto a renderer `<canvas data-embedded-mpv-frame>`, so controls/dialogs are ordinary DOM above the video. Stored and explicit opt-ins relax the sandbox only while the base embedded-MPV feature is enabled and a platform-supported runtime contains both an executable helper and readable regular frame-reader addon; packaged discovery is restricted to packaged resources. A disabled base experiment keeps embedded MPV unavailable with the sandbox intact, while a missing, mode-stripped, or incomplete frame-copy runtime falls back to the native engine without relaxing the sandbox. On Linux the engine is dev-build-only for now: the helper links system libmpv (build deps: `libmpv-dev`, `libegl-dev`, `libgl-dev`, `libgbm-dev`) and is stripped from packaged apps until bundled-runtime staging lands. Adapter: `apps/electron-backend/src/app/services/embedded-mpv-frame-copy.adapter.ts`; helper: `apps/electron-backend/native/helper/`; details in `docs/architecture/embedded-mpv-native.md` ("Frame-Copy Engine").
**VOD/Series Detail Pages (two-state layout)**:
+34 -13
View File
@@ -18,10 +18,10 @@ Source files for the embedded MPV integration:
- `libs/shared/interfaces/src/lib/embedded-mpv-session.interface.ts` defines the shared session and audio-track contract.
- `libs/ui/playback/src/lib/embedded-mpv-player/` owns the Angular UI and controls.
Frame-copy engine sources (experimental, macOS Apple Silicon — see the
"Frame-Copy Engine" section below):
Frame-copy engine sources (experimental, macOS Apple Silicon and Linux —
see the "Frame-Copy Engine" section below):
- `apps/electron-backend/native/helper/` — `iptvnator_mpv_helper` process (`mpv_frame_helper.cpp`, `frame_helper_render.h`, `frame_helper_io.h`, `frame_shm.h`).
- `apps/electron-backend/native/helper/` — `iptvnator_mpv_helper` process (`mpv_frame_helper.cpp`, `frame_helper_render.h`, `frame_helper_gl.h`, `frame_helper_io.h`, `frame_shm.h`).
- `apps/electron-backend/native/src/embedded_mpv_frame_reader.c` — N-API shm frame reader used by the preload frame pump.
- `apps/electron-backend/src/app/services/embedded-mpv-frame-copy.adapter.ts` — helper-process adapter behind the `NativeEmbeddedMpvAddon` surface.
- `apps/electron-backend/src/app/api/embedded-mpv-frame-pump.ts` — preload frame pump (shm → WebGL canvas).
@@ -49,7 +49,9 @@ Linux native Wayland embedding is not implemented. When Electron is started on X
## Linux Support Matrix
Embedded MPV on Linux is supported only for x64 desktop builds where Electron runs under X11 or Xwayland and an `mpv` executable is available on `PATH`. Native Wayland embedding is not supported in this implementation. Packaged Linux launchers pass `--ozone-platform=x11` so Wayland desktops use Xwayland when it is available, and `main.ts` appends the same switch on Linux when it is absent so direct binary/AppImage launches from a terminal behave like launcher starts. Explicit user intent is never overridden: both a user-provided `--ozone-platform` switch and the `ELECTRON_OZONE_PLATFORM_HINT` environment variable suppress the fallback.
Embedded MPV on Linux is supported only for x64 desktop builds where Electron runs under X11 or Xwayland and an `mpv` executable is available on `PATH`. Native Wayland embedding is not supported in this implementation.
The experimental frame-copy engine (below) is the exception to both requirements: it renders offscreen through headless EGL into a renderer canvas — no window embedding — and the helper links libmpv itself, so neither the X11/Xwayland constraint nor the system-`mpv`-on-`PATH` probe applies while it is active. It is currently a dev-build-only engine on Linux (the helper links the build host's system `libmpv` and is stripped from packaged apps until the bundled-runtime staging lands). Packaged Linux launchers pass `--ozone-platform=x11` so Wayland desktops use Xwayland when it is available, and `main.ts` appends the same switch on Linux when it is absent so direct binary/AppImage launches from a terminal behave like launcher starts. Explicit user intent is never overridden: both a user-provided `--ozone-platform` switch and the `ELECTRON_OZONE_PLATFORM_HINT` environment variable suppress the fallback.
When the `mpv` executable probe fails inside a Flatpak or Snap sandbox (`FLATPAK_ID`/`SNAP` env present), the support reason explains that sandboxed packages cannot access a system mpv instead of asking the user to install it.
@@ -98,16 +100,22 @@ The MPV video surface is a native platform view/window, not a normal DOM element
The dock has a stable reserved height while embedded controls are enabled. Controls fade in and out inside that fixed dock, so normal show/hide behavior does not resize the native MPV viewport or make the video jump. Volume and audio-track panels replace the default transport controls inside the same dock and provide a back button to return to the default controls. Popovers and menus must stay inside that dock unless the native layering strategy changes. The native MPV view deliberately ignores hit testing so mouse movement passes through to Chromium and can reveal Angular controls even when the pointer moves quickly across the video area.
## Frame-Copy Engine (Experimental, Apple Silicon Only)
## Frame-Copy Engine (Experimental, Apple Silicon and Linux)
`IPTVNATOR_ENABLE_EMBEDDED_MPV_FRAME_COPY=1` (on top of the regular
embedded MPV experiment flag) switches macOS/arm64 to a second rendering
engine that replaces the native-view compositing entirely:
embedded MPV experiment flag) switches macOS/arm64 and Linux to a second
rendering engine that replaces the native-view compositing entirely
(gate: `isFrameCopyPlatformSupported()` in
`embedded-mpv-frame-copy-platform.util.ts`, shared by `main.ts`, the
service and the adapter):
- `apps/electron-backend/native/helper/` — `iptvnator_mpv_helper`, a
one-process-per-session libmpv host. It decodes (hwdec), renders
offscreen at viewport size (headless CGL + async PBO readback ring),
publishes BGRA frames into a POSIX shm seqlock ring
offscreen at viewport size (async PBO readback ring over a headless GL
context — `frame_helper_gl.h`: CGL on macOS; on Linux EGL, acquiring a
display in order surfaceless-Mesa → default display → GBM render node,
logging the chosen tier to stderr), publishes BGRA frames into a POSIX
shm seqlock ring
(`frame_shm.h`, 3 slots, resize creates a new `-g<N>` generation), and
plays audio directly. Control protocol: tab-separated commands on stdin,
JSON events on stdout; the `snapshot` event mirrors
@@ -175,7 +183,12 @@ ships. The after-pack hook restores the helper's executable mode after the
asset copy, and optional/skipped native rebuilds remove stale helper/reader
artifacts before reporting frame-copy availability. This cleanup prevents
known leftover build output; it is not a compatibility check for a complete
but version-mismatched runtime pair.
but version-mismatched runtime pair. Linux packages deliberately do NOT ship
the helper yet: it links the build host's system `libmpv`, which packaged
apps cannot assume is installed, so the centralized after-pack artifact
preparation strips it and the support probe reports frame-copy unavailable.
The engine is dev-build-only on Linux until bundled-libmpv runtime staging
lands (PORTING.md milestone 4).
Trade-offs and constraints:
@@ -186,12 +199,20 @@ Trade-offs and constraints:
before this engine can become a default — candidates: utilityProcess +
MessagePort (costs one extra copy + GC churn since Electron ports clone
ArrayBuffers) or a WebCodecs-based path.
- Scope: Apple Silicon only by owner decision (2026-07-10); Intel Macs
keep the native-view engine. Windows/Linux ports of the helper (WGL/EGL)
are future work — the shm protocol and adapter are platform-agnostic.
- Scope: on macOS Apple Silicon only by owner decision (2026-07-10);
Intel Macs keep the native-view engine. Linux (any arch) is ported —
headless EGL, works under native Wayland since nothing embeds into a
window; dev builds need `libmpv-dev`, `libegl-dev`, `libgl-dev` and
`libgbm-dev` (the helper links system libmpv, which is legal
out-of-process — the in-process libmpv ban still binds the addon). The
Windows port of the helper (WGL) is future work — the shm protocol and
adapter are platform-agnostic.
- Measured baseline (M1 Pro, spikes/mpv-frame-copy/RESULTS.md): 4K60 HEVC
sustained end to end, ~1.2 ms shm copy + ~3.5 ms texture upload, ~10 ms
produce-to-upload latency, zero torn frames over a 10-minute run.
Linux (i7-1165G7/Iris Xe, same RESULTS.md): 1080p60 sustained with
~1.2 ms copies; 4K rows are software-decode-limited on that hardware;
zero torn frames everywhere.
- Helper crash isolation: an unexpected helper exit surfaces as a session
`error` (renderer falls back); it can never take down the Electron main
process, unlike in-process libmpv.
+11 -4
View File
@@ -13,9 +13,16 @@
restart → helper process renders mpv offscreen → shm ring → preload pump
→ WebGL canvas. Verified live with real IPTV + Stalker VOD.
- Scope decision: macOS = arm64 only (Intel Macs keep the native engine).
Windows/Linux are NOT ported yet — this branch on those OSes behaves
exactly like master (helper doesn't build there, engine can't activate,
env flag falls back to native).
- **UPDATE 2026-07-11: the Linux port below is DONE** (branch
`claude/linux-frame-copy-port-0a871b`, stacked PR on #1169) — headless
EGL helper, portable clock, reader on `__linux__`, TS gates, i18n,
measurements in RESULTS.md. Verified end-to-end in-app on Ubuntu 25.04
(Wayland session) with the xtream mock portal. Dev-build-only on Linux:
the helper links system libmpv and `electron-after-pack.cjs` strips it
from packages until milestone 3 (bundled runtime) — remove that strip
when milestone 3 lands. Windows is NOT ported yet — this branch on
Windows behaves exactly like master (helper doesn't build there, engine
can't activate, env flag falls back to native).
- Coordination: PR #1169 credits larsemig's idea (#1154 comment 4932807350)
and proposes the series merge plan: merge the shared-controls subset
(#1148/#1149/#1152–54) rebased, supersede immersive (#1150/#1151) with
@@ -61,7 +68,7 @@ create/open a Windows twin, flip the TS gates, extend packaging.
## Per-platform task lists
### Linux (do first — much closer to done)
### Linux (DONE 2026-07-11 — see the update in "State" above)
1. **Render backend**: headless EGL (`EGL_PLATFORM_SURFACELESS_MESA` /
`eglGetPlatformDisplay(EGL_PLATFORM_SURFACELESS_MESA)` with fallback to
+33
View File
@@ -107,6 +107,39 @@ would not have been representative. The macOS hardware gate is therefore
closed by the M1 Pro numbers above; the remaining risk hardware is
Windows/Linux.
## Linux mid-range laptop (iGPU) — Ubuntu 25.04, i7-1165G7 / Iris Xe, x64 — 2026-07-11
Source: Linux port branch (headless-EGL `frame_helper_gl.h` backend), system
libmpv 2.5.0 (mpv 0.40), Mesa 25.0 iris. Measured with the production helper
binary + `embedded_mpv_frame_reader.node` in a Node probe loop (the spike
viewer harness is macOS-only), so *age* here is produce→reader-copy and
excludes the renderer texture upload. hwdec was NOT active — this machine
has no VAAPI driver installed (`intel-media-va-driver`), so HEVC rows are
software decode; treat them as a decode-limited floor, not a pipeline
ceiling.
| Scenario | New fps | copy ms avg/p95 | age ms avg/p95 | torn |
| --- | --- | --- | --- | --- |
| 1080p60 testsrc2, sw | 60.1 | 1.16 / 1.37 | 2.26 / 3.21 | 0 |
| 4K60 testsrc2, sw | 50.0 | 5.75 / 6.92 | 7.22 / 8.00 | 0 |
| 4K60 HEVC 25 Mbit, sw decode | 39.9 | 7.62 / 14.6 | 9.06 / 17.0 | 0 |
| 4K60 HEVC 25 Mbit in a 1280×720 viewport | 53.1 | 0.94 / 2.57 | 2.42 / 5.28 | 0 |
Readings:
- 1080p60 — the realistic viewport class for this laptop's 1920×1200
screen — holds a clean 60 fps with ~1 ms copies.
- The 4K rows are stress rows: producers are limited by software
decode/source generation on 4 cores, not by the copy path (copy stays
well under one 60 Hz frame budget even at full 4K).
- The viewport-size claim reproduces on Linux: the same 4K60 HEVC clip in a
720p viewport drops the copy from 7.6 ms to 0.94 ms and lifts fps from
~40 to ~53 (remaining gap = software decode).
- torn=0 across every run; the aspect-fit generation bump was verified
separately (4:3 source in a 16:9 viewport → `-g2` at 960×720).
- EGL display tier used: Mesa surfaceless platform (first tier; no display
server needed).
## Windows mid-range laptop (iGPU) — PENDING
Blocked on the Windows helper port (WGL or D3D11 readback path).
+4 -4
View File
@@ -1,6 +1,6 @@
# Embedded MPV Runtime
This folder contains tooling for preparing MPV runtime/build inputs for IPTVnator's experimental embedded MPV player. macOS and Windows bundle `libmpv`; Linux uses staged MPV headers for compilation and launches the system `mpv` executable at runtime.
This folder contains tooling for preparing MPV runtime/build inputs for IPTVnator's experimental embedded MPV player. macOS and Windows bundle `libmpv`; Linux uses staged MPV headers for compilation and launches the system `mpv` executable at runtime. On Linux, `apps/electron-backend/build-embedded-mpv.js` also falls back to system headers when nothing is staged (`libmpv-dev`; override with `LIBMPV_INCLUDE_DIR`/`LINUX_NATIVE_LIBRARY_DIR`), so a plain distro dev setup builds without staging. The frame-copy helper (`iptvnator_mpv_helper`) additionally needs `libegl-dev`, `libgl-dev`, and `libgbm-dev`, and links the system `libmpv` — allowed because it is a separate process; the in-process-libmpv ban still binds the addon.
## Runtime Policy
@@ -128,7 +128,7 @@ The build manifest records source URLs, downloaded archive SHA-256 values where
## Build Integration
`apps/electron-backend/build-embedded-mpv.js` builds the native addon against the staged runtime/build inputs, copies macOS/Windows runtime libraries into `apps/electron-backend/native/build/Release/lib/`, rewrites macOS Mach-O paths to `@loader_path`, and writes `embedded-mpv-runtime.json`. Linux builds use the staged MPV headers and system X11 development libraries, write an `external-mpv-process` manifest, and must not copy or link directly to `libmpv`; CI validates this with package checks and `ldd`.
`apps/electron-backend/build-embedded-mpv.js` builds the native addon against the staged runtime/build inputs, copies macOS/Windows runtime libraries into `apps/electron-backend/native/build/Release/lib/`, rewrites macOS Mach-O paths to `@loader_path`, and writes `embedded-mpv-runtime.json`. Linux builds use the staged (or system) MPV headers and system X11 development libraries, write an `external-mpv-process` manifest, and the addon must not copy or link directly to `libmpv`; CI validates this with package checks and `ldd`. The frame-copy helper executable built by the same run is the inverse: CI verifies it DOES link `libmpv` (separate process).
For local macOS development with Homebrew `mpv`, use:
@@ -138,7 +138,7 @@ pnpm run serve:backend:embedded-mpv
The script rebuilds the native addon with `IPTVNATOR_EMBEDDED_MPV_ALLOW_HOMEBREW=1` before starting Electron with the experimental player enabled. Use this only for local testing; release packaging rejects the resulting `homebrew-dev` runtime manifest.
The `afterPack` hook copies `dist/apps/electron-backend/native/` into `app.asar.unpacked/electron-backend/native/` so the addon, runtime manifest, and runtime libraries are available as real files where needed. Linux packages include the addon and manifest, but no bundled `libmpv.so`.
The `afterPack` hook copies `dist/apps/electron-backend/native/` into `app.asar.unpacked/electron-backend/native/` so the addon, runtime manifest, and runtime libraries are available as real files where needed. Linux packages include the addon and manifest, but no bundled `libmpv.so`, and the hook strips `iptvnator_mpv_helper` from Linux packages (it links the build host's system libmpv; the frame-copy engine stays dev-build-only on Linux until bundled-runtime staging lands).
During release packaging, `tools/packaging/electron-after-pack.cjs` verifies that macOS/Windows packages use a `vendored-lgpl` runtime/build input set. macOS artifacts additionally verify that Mach-O dependencies have no `/opt/homebrew` or `/usr/local` dynamic links for embedded MPV. Linux artifacts verify that the addon and `external-mpv-process` manifest are present, that no bundled `libmpv.so` files are present, and the runtime support check verifies that `mpv` is available on `PATH`.
@@ -148,4 +148,4 @@ Set `IPTVNATOR_REQUIRE_EMBEDDED_MPV=1` when packaging a release artifact that mu
- macOS keeps the existing libmpv render-context backend because mpv `wid` stays black inside Electron on macOS.
- Windows uses an embedded child `HWND` and passes it to mpv through `wid`.
- Linux uses an X11 child window and starts a system `mpv --wid` process for that window. Native Wayland is not supported in v1; run under X11/Xwayland so `DISPLAY` is set and `mpv` can honor the X11 window id.
- Linux uses an X11 child window and starts a system `mpv --wid` process for that window. Native Wayland is not supported in v1; run under X11/Xwayland so `DISPLAY` is set and `mpv` can honor the X11 window id. The experimental frame-copy engine has no window embedding at all (offscreen EGL into a renderer canvas) and therefore works under native Wayland — dev builds only for now.