Files
4grayandClaude Fable 5 26271fc076 feat(embedded-mpv): frame-copy rendering engine (experimental, macOS Apple Silicon) (#1169)
* spike(embedded-mpv): frame-copy pipeline prototype (helper + shm ring + Electron viewer)

Standalone macOS spike for the frame-copy unification direction from the
2026-07-10 analysis: a helper process renders mpv offscreen into a GL FBO,
reads frames back through an async PBO ring, and publishes BGRA frames into
a 3-slot POSIX shm seqlock ring; a minimal Electron viewer copies the newest
frame via a plain-C N-API addon and uploads it to a WebGL canvas per rAF.

First numbers on M1 Pro (see spike README): 4K60 HEVC hwdec sustained at
60 fps end to end, ~1.2 ms shm copy + ~3.5 ms texture upload, ~10 ms
produce-to-upload age, zero torn frames. Remaining gates: weak hardware,
long-run pacing, HDR, latency flash test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* spike(embedded-mpv): add RESULTS.md measurement log with M1 Pro baseline

Structured per-machine table with repro commands so the pending Intel Mac
and Windows iGPU runs can be appended and compared one-to-one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* spike(embedded-mpv): pacing/judder instrumentation + 50/25 fps and HDR gate results

Viewer now measures inter-frame intervals on both clocks (present side and
producer side): stddev/p99/max, late-frame counters vs the producer's median
interval, and a cumulative LONGRUN summary every 30 s. The addon exposes the
producer timestamp (produceMs) for this.

Measured on M1 Pro: 50 fps and 25 fps cadences are clean (late frames only
at startup; residual jitter is 120 Hz rAF grid quantization, bounded by one
display tick), and 4K25 HDR10 PQ/BT.2020 is tonemapped to SDR by mpv before
readback at full rate with unchanged copy costs. RESULTS.md carries the
tables and HDR-clip repro commands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* spike(embedded-mpv): record 10-minute 4K60 HEVC long-run results

Zero dropped frames and zero torn reads after the first-minute warmup over
~8.5 minutes; steady-state late frames (~0.4%) track the 12 s test clip's
--loop restarts, not the copy pipeline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* spike(embedded-mpv): viewport-scaling measurement + integration design draft

Confirms the render-at-viewport-size claim (4K source in a 720p FBO costs
720p: 0.17 ms readback / 0.16 ms copy / 0.17 ms upload at 60 fps) and adds
DESIGN.md — the draft integration architecture: per-session helper process
linking bundled libmpv on all platforms (finally full-featured + Wayland-
agnostic Linux), JSON-over-stdio control evolving the Linux wid protocol,
unchanged EmbeddedMpvSession renderer contract, shm generations for resize,
packaging via the existing vendored-runtime tooling, rollout behind its own
flag with the docked path as default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* spike(embedded-mpv): auto-detect Homebrew prefix and Node headers for Intel Macs

BREW_PREFIX was hardcoded to /opt/homebrew (Apple Silicon) and NODE_INC to
one nvm version; both now resolve via brew --prefix and the PATH node's
execPath, so the pending Intel Mac run needs no Makefile edits. README gets
a fresh-machine checklist.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* spike(embedded-mpv): self-contained measurement bundle for machines without Node/pnpm

make-bundle.sh assembles a tarball with the spike sources, vendored N-API
headers (Makefile prefers them when present, so no Node install is needed),
pre-generated 4K HEVC/HDR10 test clips, and an official Electron dist
download for the target arch. collect-results.sh builds and runs the full
RESULTS.md scenario suite automatically (plus an optional --long 10-minute
run) and writes one results-<host>-<date>.txt to send back. Target-machine
prerequisites shrink to Xcode CLT + brew mpv — built for the pending Intel
Mac baseline run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* spike(embedded-mpv): support MacPorts libmpv and legacy-macOS bundles

Makefile and collect-results.sh now detect libmpv in the Homebrew prefix or
MacPorts /opt/local (Homebrew is unsupported on legacy macOS like High
Sierra; 'sudo port install mpv +libmpv' provides libmpv there). make-bundle
takes ELECTRON_VERSION/BUNDLE_SUFFIX overrides — Electron 27+ needs macOS
10.15, so High Sierra bundles ship Electron 26.6.10 (LSMinimumSystemVersion
10.13).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* spike(embedded-mpv): scope macOS frame-copy engine to Apple Silicon only

Owner decision 2026-07-10: skip Intel Mac measurements and gate the future
frame-copy engine on arm64. Intel Macs able to run the app at all are a
shrinking 2015-2020 cohort and keep the docked/external/web player paths;
the macOS hardware gate closes with the M1 Pro numbers, and remaining
hardware risk moves to the Windows/Linux ports.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(embedded-mpv): frame-copy helper process and shm frame reader (native layer)

iptvnator_mpv_helper: one-process-per-session libmpv host that renders
offscreen at viewport size (headless CGL + async PBO ring, validated in
spikes/mpv-frame-copy), publishes BGRA frames into a seqlock shm ring with
resize generations, plays audio directly, and speaks a stdio protocol —
tab-separated commands in, JSON events out. The snapshot event mirrors
NativeEmbeddedMpvSessionSnapshot; status semantics (END_FILE reasons,
eof-reached with keep-open, pause gated on loaded path, fatal-only status
flips) are ported from embedded_mpv.mm.

embedded_mpv_frame_reader.node: plain-C N-API reader the preload script
uses to memcpy the newest complete frame into a V8 ArrayBuffer (Electron's
memory cage forbids zero-copy). Stub exports off macOS.

Both build as extra binding.gyp targets through build-embedded-mpv.js; the
helper gets the same libmpv dependency-path rewrite + ad-hoc re-sign as the
addon and is validated by the forbidden-link check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(embedded-mpv): frame-copy engine wiring in main process and preload

EmbeddedMpvFrameCopyAdapter implements the NativeEmbeddedMpvAddon surface
over a per-session helper process (spawn, stdio protocol, snapshot cache,
graceful quit->SIGTERM->SIGKILL teardown), so EmbeddedMpvNativeService
reuses its polling/diff/power-blocker/recording logic unchanged. The
IPTVNATOR_ENABLE_EMBEDDED_MPV_FRAME_COPY flag (darwin/arm64 only) routes
getAddon() to the adapter and reports engine: 'frame-copy' in support.

The preload frame pump loads the shm reader addon, copies the newest frame
once per rAF into a reused buffer, and uploads it to WebGL2 on the
renderer's canvas — no frame data crosses the contextBridge; the bridge
only gains attachEmbeddedMpvFrameView/detachEmbeddedMpvFrameView. The
experiment flag relaxes the window sandbox for that native require;
contextIsolation and nodeIntegration:false stay on.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(embedded-mpv): frame-copy canvas mode in the player component + docs

EmbeddedMpvPlayerComponent renders <canvas data-embedded-mpv-frame> when
support reports engine 'frame-copy' and the session controller starts/stops
the preload frame pump around the session lifecycle. The bounds provider
skips HIDDEN_BOUNDS and the popover cutout for this engine — the canvas is
ordinary DOM, dialogs and popovers stack above it natively; bounds sync
still drives the helper's render size. Adapter unit tests cover spawn args,
snapshot caching, shm generations, protocol encoding, unexpected-exit
mapping, and dispose escalation. Architecture doc and CLAUDE.md describe
the engine, its flag, and the sandbox trade-off.

Verified end to end in the built app (M1 Pro): engine detection, helper
spawn, lavfi playback onto the canvas via CDP-injected smoke — including an
orientation fix (helper FLIP_Y already yields texture-order rows; the pump
shader must not flip uv again).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(embedded-mpv): close helper stdin on dispose + lifecycle logging

Live testing surfaced a stray idle helper that survived a session switch;
until the root cause is pinned down, dispose now also closes the child's
stdin (the helper exits on EOF) as a second kill path besides quit ->
SIGTERM -> SIGKILL, and spawn/dispose/exit are logged with the session id
so leaks are attributable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(embedded-mpv): reap sessions when the renderer reloads or crashes

Root cause of the stray idle helper found during live testing: session
teardown lives in the renderer's Angular lifecycle, which never runs on a
renderer crash or hard reload — the main process kept the session (and its
frame-copy helper process / native mpv handle) alive until app shutdown.
EmbeddedMpvNativeService now watches the main window's webContents for
render-process-gone and did-navigate (full reloads only; in-app Angular
routing emits did-navigate-in-page) and disposes every session. Applies to
both engines. Verified live: location.reload() during frame-copy playback
logs 'Disposing 1 session(s): renderer reloaded' and the helper exits
cleanly. Regression test drives both events against the service.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(embedded-mpv): Settings toggle for the frame-copy engine

New 'Embedded MPV: frame-copy engine' checkbox in Settings > Playback,
shown only when the machine can run it (macOS arm64 with the helper binary
present — support now reports frameCopyAvailable). The choice persists to
the main-process config store because the engine relaxes the window sandbox
for the preload frame pump, which is fixed at window creation: main.ts
reads the store before creating the window and sets the engine env var; an
explicitly set env var (including '0') always wins, and the UI shows a
restart hint while the saved choice differs from the active engine.
Localized in all 18 locales.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(embedded-mpv): aspect-fit rendering in the frame-copy helper

The helper now observes dwidth/dheight and renders its FBO at the
aspect-fit size of the video inside the requested viewport, bumping a shm
generation on change — letterbox bars are never baked into frames (the VOD
watch shell's ~2:1 box no longer shows black side bars; the canvas
background is transparent so the sides show the app surface, while
fullscreen keeps its black backdrop). Frames also get smaller than the
viewport when aspects differ, trimming copy cost. Aspect override changes
refit automatically. Snapshots now carry videoWidth/videoHeight, and the
adapter forwards IPTVNATOR_EMBEDDED_MPV_AUDIO_DELAY to mpv's audio-delay
for lip-sync tuning until proper calibration lands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(embedded-mpv): require frame-copy artifacts in macOS package validation + docs

macOS packages that ship embedded_mpv.node must also ship the
iptvnator_mpv_helper binary and the embedded_mpv_frame_reader.node addon —
they come out of the same binding.gyp run, and a package missing them would
silently lose the frame-copy engine. Covered in the package-identity test.
Architecture doc and CLAUDE.md document the Settings toggle, aspect-fit
rendering, audio-delay passthrough, and the renderer-reload session reaping.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(electron): inline TS helpers so the sandboxed preload keeps working

The frame pump's async/await (target es2015 + importHelpers) made webpack
externalize tslib in main.preload.js. Sandboxed preloads can only require
Electron's built-in module whitelist, so the entire preload script failed
to load and window.electron disappeared for every run without the
frame-copy flag. importHelpers:false for electron-backend keeps the preload
bundle self-contained — and future async code in preload can no longer
silently reintroduce the breakage. Verified live: sandboxed run now has the
bridge, reports engine 'native' and frameCopyAvailable true.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(embedded-mpv): commit the frame-copy analysis handoff + source inventory

The 2026-07-10 analysis that led to this branch now lives next to the spike
(spikes/mpv-frame-copy/ANALYSIS.md), and the architecture doc's What To
Commit section lists the frame-copy engine sources.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(embedded-mpv): address review findings on the frame-copy engine

- Stale pump attach can no longer win over a newer session: attach/detach
  bump a shared epoch and async attach waits re-check it after every await,
  so an attach for a replaced session aborts instead of installing itself
  (greptile P1).
- A failed frame-view attach (no canvas, no WebGL2, reader missing) now
  disposes the session and surfaces the error UI instead of leaving audio
  playing behind a black canvas (codex P2).
- A stale frame-copy opt-in without the helper binary falls back to the
  native engine instead of reporting embedded MPV unsupported, and the
  Settings checkbox stays visible while a saved opt-in exists so it can
  always be cleared (codex P2). Regression test covers the fallback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(packaging): make the darwin frame-copy packaging test host-agnostic

On non-macOS CI hosts validatePackagedEmbeddedMpv also reports that macOS
link validation needs a macOS host, so the success-path assertion now
checks only the frame-copy artifact requirement instead of expecting an
empty error list.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(embedded-mpv): Windows/Linux porting handoff for the frame-copy engine

Self-contained entry point for porting sessions on other machines: current
state and coordination constraints, per-OS task lists (Linux EGL first,
then Windows WGL + named shm — the decisive iGPU perf gate), the
hard-won gotchas from the macOS integration (preload/tslib sandbox
breakage, V8 memory cage, frame orientation, stale-attach epoch, dispose
escalation, node-gyp naming, snapshot protocol semantics), testing
recipes, and the suggested milestone order.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(embedded-mpv): branching and merge strategy in the porting handoff

Port work goes to stacked branches off the frame-copy branch (PR base =
frame-copy branch, sequential merges, stack depth one), never into the
frozen PR #1169 branch itself.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(embedded-mpv): drop stale uncommitted note from porting handoff

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(embedded-mpv): harden frame-copy helper startup

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 19:48:07 +02:00

145 lines
5.4 KiB
Bash
Executable File

#!/usr/bin/env bash
# Build the spike and run the full RESULTS.md scenario suite automatically,
# writing all metrics into one results-<host>-<date>.txt file to send back.
#
# ./collect-results.sh # quick suite (~4 min, 30 s per scenario)
# ./collect-results.sh --long # + 10-minute 4K60 HEVC long run
#
# Prereqs on the target machine: Xcode Command Line Tools and Homebrew mpv
# (`brew install mpv`). Electron and media clips are bundled (or resolved
# from the repo / generated with ffmpeg when running from a checkout).
# Viewer windows will open and close on their own during the run.
set -euo pipefail
DIR="$(cd "$(dirname "$0")" && pwd)"
cd "$DIR"
DURATION="${DURATION:-30}"
LONG=0
[[ "${1:-}" == "--long" ]] && LONG=1
if ! xcode-select -p >/dev/null 2>&1; then
echo "Xcode Command Line Tools required: run 'xcode-select --install'" >&2
exit 1
fi
MPV_PREFIX=""
for prefix in "$(brew --prefix 2>/dev/null || true)" /opt/local; do
if [[ -n "$prefix" ]] && ls "$prefix"/lib/libmpv*.dylib >/dev/null 2>&1; then
MPV_PREFIX="$prefix"
break
fi
done
if [[ -z "$MPV_PREFIX" ]]; then
echo "libmpv not found. Install it with Homebrew ('brew install mpv') or," >&2
echo "on legacy macOS (e.g. High Sierra), MacPorts: 'sudo port install mpv +libmpv'" >&2
exit 1
fi
make
if [[ -z "${ELECTRON:-}" ]]; then
for candidate in \
"$DIR/electron/Electron.app/Contents/MacOS/Electron" \
"$DIR/../../node_modules/.bin/electron" \
"$DIR/../../../../../node_modules/.bin/electron"; do
if [[ -x "$candidate" ]]; then
ELECTRON="$candidate"
break
fi
done
fi
if [[ -z "${ELECTRON:-}" ]]; then
echo "Electron not found; set ELECTRON=/path/to/electron" >&2
exit 1
fi
if [[ "$ELECTRON" == "$DIR/electron/"* ]]; then
# Bundled runtime may carry a quarantine attribute after transfer.
xattr -cr "$DIR/electron/Electron.app" 2>/dev/null || true
fi
# Media clips: bundled under media/, otherwise generate with ffmpeg.
mkdir -p media
if [[ ! -f media/spike-4k-hevc.mp4 ]] && command -v ffmpeg >/dev/null; then
echo "generating media/spike-4k-hevc.mp4 ..."
ffmpeg -v error -y -f lavfi -i "testsrc2=size=3840x2160:rate=60" -t 12 \
-c:v hevc_videotoolbox -b:v 25M -tag:v hvc1 -pix_fmt yuv420p \
media/spike-4k-hevc.mp4
fi
if [[ ! -f media/spike-4k-hdr10.mp4 && -f media/spike-4k-hevc.mp4 ]] &&
command -v ffmpeg >/dev/null; then
echo "generating media/spike-4k-hdr10.mp4 ..."
ffmpeg -v error -y -f lavfi -i "testsrc2=size=3840x2160:rate=25" -t 10 \
-c:v hevc_videotoolbox -profile:v main10 -pix_fmt p010le -b:v 30M \
-tag:v hvc1 /tmp/spike-hdr-base.mp4
ffmpeg -v error -y -i /tmp/spike-hdr-base.mp4 -c:v copy \
-bsf:v "hevc_metadata=colour_primaries=9:transfer_characteristics=16:matrix_coefficients=9" \
media/spike-4k-hdr10.mp4
fi
OUT="$DIR/results-$(hostname -s)-$(date +%Y%m%d-%H%M).txt"
{
echo "# mpv frame-copy spike results"
echo "date: $(date)"
echo "cpu: $(sysctl -n machdep.cpu.brand_string 2>/dev/null || echo unknown)"
echo "os: $(sw_vers -productName) $(sw_vers -productVersion)"
system_profiler SPDisplaysDataType 2>/dev/null |
grep -E "Chipset Model|Resolution|UI Looks like" | sed 's/^ *//' || true
} > "$OUT"
run_scenario() { # name media size extra_helper_args duration
local name="$1" media="$2" size="$3" extra="$4" dur="$5"
echo "== $name (${dur}s) =="
printf '\n## %s\n' "$name" >> "$OUT"
local hlog vlog
hlog="$(mktemp /tmp/spike-helper.XXXXXX)"
vlog="$(mktemp /tmp/spike-viewer.XXXXXX)"
# shellcheck disable=SC2086
./build/mpv_helper "$media" --size "$size" --no-audio $extra \
> "$hlog" 2>&1 &
local hpid=$!
disown "$hpid"
sleep 2
SPIKE_SHM=/mpv-frame-spike "$ELECTRON" "$DIR/viewer/main.js" \
> "$vlog" 2>&1 &
local vpid=$!
disown "$vpid"
sleep "$dur"
kill "$vpid" 2>/dev/null || true
kill -INT "$hpid" 2>/dev/null || true
sleep 1
kill -9 "$hpid" "$vpid" 2>/dev/null || true
grep -E "PIXELPROBE|STATS|LONGRUN" "$vlog" |
sed 's/^\[viewer\] //' | tail -24 >> "$OUT"
grep -E "^\[helper\] (GL renderer|fps=)" "$hlog" | tail -6 >> "$OUT"
rm -f "$hlog" "$vlog"
}
run_scenario "1080p60 testsrc2 sw" \
'av://lavfi:testsrc2=size=1920x1080:rate=60' 1920x1080 "" "$DURATION"
run_scenario "4K60 testsrc2 sw" \
'av://lavfi:testsrc2=size=3840x2160:rate=60' 3840x2160 "" "$DURATION"
run_scenario "1080p50 cadence" \
'av://lavfi:testsrc2=size=1920x1080:rate=50' 1920x1080 "" "$DURATION"
if [[ -f media/spike-4k-hevc.mp4 ]]; then
run_scenario "4K60 HEVC hwdec=videotoolbox" \
media/spike-4k-hevc.mp4 3840x2160 "--hwdec videotoolbox --loop" "$DURATION"
run_scenario "4K60 HEVC -> 720p viewport" \
media/spike-4k-hevc.mp4 1280x720 "--hwdec videotoolbox --loop" "$DURATION"
else
echo "WARN: media/spike-4k-hevc.mp4 missing, HEVC scenarios skipped" | tee -a "$OUT"
fi
if [[ -f media/spike-4k-hdr10.mp4 ]]; then
run_scenario "4K25 HDR10 tonemap hwdec" \
media/spike-4k-hdr10.mp4 3840x2160 "--hwdec videotoolbox --loop" "$DURATION"
else
echo "WARN: media/spike-4k-hdr10.mp4 missing, HDR scenario skipped" | tee -a "$OUT"
fi
if [[ "$LONG" == 1 && -f media/spike-4k-hevc.mp4 ]]; then
run_scenario "LONGRUN 10min 4K60 HEVC hwdec" \
media/spike-4k-hevc.mp4 3840x2160 "--hwdec videotoolbox --loop" 600
fi
echo
echo "Done. Send this file back:"
echo " $OUT"