Files
iptvnator/tools/embedded-mpv/README.md
4gray 3ed612ba65 fix(release): repair Snap uploads and retry published releases (#1691)
* fix(release): allow Snapcraft scratch extraction and retry public releases

* test(release): detect local Snap permission test prerequisites
2026-09-25 23:34:44 +02:00

478 lines
25 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Embedded MPV Runtime
This directory owns the source builders, staging, manifests, and archive
helpers for IPTVnator's experimental Embedded MPV runtime.
The Linux architecture has a strict process boundary:
- Electron, `embedded_mpv.node`, and `embedded_mpv_frame_reader.node` must not
load or link libmpv.
- Native-view starts a separate system `mpv --wid` process.
- Frame-copy starts `iptvnator_mpv_helper`; only that helper may link libmpv.
Do not weaken this boundary to simplify packaging. A missing helper/runtime
must make frame-copy unavailable and leave native-view as the safe x64
fallback.
## Runtime Policy
Release builds use an LGPL-compatible, dynamically linked runtime:
- FFmpeg is built without `--enable-gpl` and `--enable-nonfree`.
- mpv is built with `-Dlibmpv=true` and `-Dgpl=false`.
- Bundled libraries remain individually replaceable under `native/lib`.
- Exact source URLs, versions, checksums or git commits, submodules, licenses,
build flags, local patches, and build scripts are published with the release.
Homebrew mpv is local-development-only. It requires
`IPTVNATOR_EMBEDDED_MPV_ALLOW_HOMEBREW=1`, and release validation rejects it.
## Generated Layout
```text
vendor/embedded-mpv/
darwin-arm64/
include/mpv/client.h
lib/*.dylib
runtime-manifest.json
darwin-x64/
include/mpv/client.h
lib/*.dylib
runtime-manifest.json
win32-x64/
include/mpv/client.h
lib/libmpv-2.dll # accepted basename variants are preserved
lib/libmpv.dll.a # or an MSVC import library
runtime-manifest.json
linux-x64/
include/mpv/client.h
lib/libmpv.so
lib/libmpv.so.2
lib/<declared closure>
notices/embedded-mpv-notices.json
notices/THIRD_PARTY_NOTICES.txt
notices/licenses/<package>/<upstream path>
runtime-manifest.json
```
These directories are generated release inputs and are ignored by git.
`runtime-manifest.json` is the profile-neutral source/build manifest. Packaging
writes a normalized `embedded-mpv-runtime.json` beside the native artifacts.
Bundled Linux profiles flatten the three notice entries from `notices/` into
that same native directory; system and marker-only profiles remove them.
## Building And Staging
Stage an existing compatible prefix with:
```bash
pnpm embedded-mpv:stage-runtime -- darwin arm64 /path/to/prefix
pnpm embedded-mpv:stage-runtime -- darwin x64 /path/to/prefix
pnpm embedded-mpv:stage-runtime -- win32 x64 /path/to/prefix
pnpm embedded-mpv:stage-runtime -- linux x64 /path/to/prefix
```
Build the pinned macOS or Linux source runtime first when no prefix exists:
```bash
pnpm embedded-mpv:build-runtime -- arm64 /tmp/macos-prefix
pnpm embedded-mpv:stage-runtime -- darwin arm64 /tmp/macos-prefix
pnpm embedded-mpv:build-runtime:linux -- /tmp/linux-prefix
pnpm embedded-mpv:stage-runtime -- linux x64 /tmp/linux-prefix
```
### Windows CI pin lifecycle
The PAT-backed refresh job must pin every third-party action to a full commit.
Windows package builds consume the one validated record in
`windows-runtime-pin.json`; URL and checksum repository variables are not build
inputs. Check it locally with:
```bash
pnpm embedded-mpv:windows-runtime-pin:check
```
The weekly `refresh-windows-embedded-mpv-runtime.yaml` workflow opens a bot PR
when the upstream asset is unavailable or 14 days old, well before zhongfly's
30-day retention boundary. A manual refresh uses the same dependency-free
updater:
```bash
pnpm embedded-mpv:windows-runtime-pin:refresh -- --force
```
That workflow validates its own result with
`windows-runtime-pin.test.mjs`, so nothing in those refresh tests may read the
CHECKED-IN pin's `publishedAt`. Building a fixture from it makes the outcome a
function of the very data the job replaces, and the assertion then fails
exactly when the job succeeds: a rotation asserted that the pin it had just
written was already past the 14-day threshold, the validation step went red,
the bot PR was never opened, and the pin sat until upstream retention deleted
its asset — turning every Windows build red on a cold cache. Refresh tests
build their own pin at a chosen age (`createPinFixture({ ageDays })`);
`CURRENT_PIN` is for the schema, naming and licence-statement checks, which
hold for any pin.
The upstream archive is checksum- and layout-verified, not independently
certified as a complete LGPL closure. It contains no corresponding source or
license notices, so IPTVnator does not mirror it. Any future stable mirror must
ship complete corresponding source, exact build scripts and patches, notices,
and a validated transitive license record beside the binary.
The macOS builder verifies every downloaded archive against its pinned
SHA-256 digest before extraction. FreeType uses its official SourceForge
distribution as the primary source and the official Savannah distribution as
a fallback; a failed or mismatched download is discarded before the next
mirror is attempted. The runtime manifest records the selected URL for a new
download and the complete ordered candidate list for every archive, so
fallback use remains visible in the source provenance. Changes to the
downloader participate in the runtime cache key, so cached native artifacts
cannot outlive source-acquisition policy changes.
The Linux builder runs only on Linux x64. It requires the tool versions and
system development interfaces declared in `build-linux-runtime.cjs`, including
Meson 1.6 or newer, gperf 3.1 or newer, Ninja, CMake, NASM, pkg-config,
patchelf, and `readelf`.
It builds into an owned staging directory and publishes atomically, so it will
not delete or overwrite an arbitrary destination.
The Linux builder acquires its archives through the same
`downloadPinnedSource` helper, so a pin whose primary host is unavailable
falls through to its pinned `mirrors` before the build fails; every candidate
is verified against the pinned SHA-256, and a mismatch is discarded rather
than used. FreeType, Fontconfig, and libdisplay-info carry mirrors because
their primary hosts are single points of failure — freedesktop.org in
particular answers GitHub runners with HTTP 418 under load. Unlike the macOS
manifest, the Linux manifest keeps `sourceUrl` at the canonical pinned value
even when a mirror served the bytes: notice generation and the Snap
publication boundary compare that field against the immutable pin. A used
mirror is reported in the build log instead. `download-pinned-source.mjs` is
part of the released source-archive tooling set and of the Linux runtime
cache key.
The pinned Linux source stack currently includes FFmpeg 8.1, mpv 0.41.0,
libplacebo 7.360.1, libass 0.17.3, FreeType 2.13.3, FriBidi 1.0.16,
HarfBuzz 8.5.0, Expat 2.8.2, Fontconfig 2.16.0, OpenSSL 3.5.7, hwdata
0.409, and libdisplay-info 0.1.1. The builder stages a private pinned
`pnp.ids`/`hwdata.pc`; libdisplay-info is not allowed to consume the build
host's `/usr/share/hwdata`.
Before publication, the Linux builder verifies:
- every archive digest and git/submodule commit;
- the exact FFmpeg/mpv flags and LGPL policy;
- an exact `libmpv.so.2` SONAME and complete reachable shared-library closure;
- `$ORIGIN` RUNPATHs with no build-prefix paths or undeclared host fallback;
- the external system-library allowlist;
- `GLIBC_2.35` and `GLIBCXX_3.4.30` ABI ceilings;
- file hashes, byte sizes, build inputs, licenses, and source obligations.
## Linux Package Profiles
Set one exact `IPTVNATOR_LINUX_FRAME_COPY_PROFILE` per packaging pass:
| Profile | Formats | Runtime handling |
| ---------- | ---------------- | ---------------------------------------------------------------------------- |
| `system` | DEB, RPM, Pacman | Remove `native/lib`; require the format-specific system runtime listed below |
| `portable` | AppImage, Snap | Retain the pinned LGPL closure under `native/lib` |
| `flatpak` | Flatpak | Retain the same pinned LGPL closure under `native/lib` |
The system helper directly links libmpv, EGL, GL, and GBM. Package metadata
therefore declares the full interface set:
- DEB: `libmpv2`, `libegl1`, `libgl1`, `libgbm1`
- RPM: `mpv-libs`, `libglvnd-egl`, `libglvnd-glx`, `mesa-libgbm`
- Pacman: `mpv`, `libglvnd`, `mesa`
The helper links `libGL.so.1` (`-lGL`) rather than `libOpenGL.so.0`; the
former is the direct GL interface supplied by all three system contracts and
Snap's `mesa-core22`.
The DEB metadata is release-tested on Ubuntu 24.04 (Noble). Ubuntu 22.04
(Jammy) only provides `libmpv1`; use the x64 AppImage on that distribution
rather than relaxing the runtime contract. CI explicitly installs the distro
Mesa software renderer for headless smoke. IPTVnator does not add DRI-driver
packages as direct dependencies; any transitive graphics-driver stack remains
under the distro's dependency policy.
The Snap is `base: core22` with strict confinement. It retains Electron
Builder's default plugs and adds an auto-connected private `shared-memory`
plug plus `graphics-core22`, targeting a real empty mode-0755 `$SNAP/graphics`
with external `mesa-core22` as default provider. The graphics provider supplies
EGL/GL/GLX/GBM/DRM/VA, while Electron Builder's exact GNOME content runtime
supplies ALSA/PulseAudio. Neither provider is bundled into IPTVnator's Snap,
source archive, notices, or package-size accounting. The package hook creates
the empty content target because core22 does not synthesize one; the extracted
artifact verifier rejects a missing, redirected, non-empty, or wrongly
permissioned target. Snap metadata must also contain exactly the canonical
graphics-provider layouts: bind `/usr/share/libdrm` from
`$SNAP/graphics/libdrm`, and symlink `/usr/share/drirc.d` to
`$SNAP/graphics/drirc.d`.
The bounded probe and every playback helper share one sanitized loader
environment derived from the validated, cached runtime mode. Ambient
ELF audit/preload/origin/library overrides, direct EGL/GBM/GL/VA/Vulkan paths,
shell startup/options, tracing hooks, exported Bash functions, and
caller-provided architecture triplets are removed or replaced. The
extracted-artifact verifier uses the same deny-set for its direct helper smoke
and preserves feature/debug selectors such as `LIBGL_ALWAYS_SOFTWARE`. System
packages then use the default loader; bundled packages put their validated
`native/lib` first. Packaged addon/helper lookup is package-owned
`app.asar.unpacked` only; cwd/dist candidates are development-only.
AppImage and Flatpak use normal host/sandbox lookup for the declared external
interfaces. Inside the exact packaged Flatpak `/app` context, the helper
reconstructs only Freedesktop Platform 24.08's immutable
`__EGL_EXTERNAL_PLATFORM_CONFIG_DIRS` value; the GL extension's
`add-ld-path` remains available through the sandbox loader cache. Flatpak CI
therefore invokes `flatpak run com.fourgray.iptvnator
--embedded-mpv-runtime-probe` instead of executing the helper around the
application gate. In a genuine Snap mount, filtered `SNAP_LIBRARY_PATH` GL roots
under `/var/lib/snapd/lib/gl` come next, then the fixed x64
`$SNAP/graphics` roots, then the core22 base
`/usr/lib/x86_64-linux-gnu`, exact `$SNAP/gnome-platform` graphics/audio roots,
and finally generic `$SNAP` library roots. Keeping the base ABI ahead of the
older GNOME content runtime prevents its `libedit.so.2` from injecting an
unavailable `libtinfo.so.5` dependency into mesa-core22's software renderer.
The helper rebuilds GBM, GL/VA driver, EGL vendor/platform, and Vulkan layer
variables from those trusted locations. A Linux session without the validated
cached mode is rejected before spawn.
Both the bounded probe and playback execute through
`$SNAP/graphics/bin/graphics-core22-provider-wrapper`. The graphics mount must
be a real directory and the wrapper a regular, non-symlinked, readable
executable. Otherwise the gate returns the stable
`snap-graphics-provider-unavailable` reason before spawning the helper. The
wrapper child also drops shell startup/options, tracing hooks, and exported
`BASH_FUNC_*` functions, and uses a fixed core22 system `PATH`; ambient Bash
configuration therefore cannot replace the probe before helper execution.
Installed-Snap CI disconnects `graphics-core22`, requires the application-level
diagnostic to emit `snap-graphics-provider-unavailable` and exit with the
controlled status `1`, then reconnects the provider and requires a successful
diagnostic. This keeps the canonical layouts and missing-provider fallback in
the same regression contract.
Flatpak is an isolated packaging pass and keeps `iptvnator` as the real
Electron ELF so Electron Builder's `electron-wrapper` passes it directly to
Zypak. Other Linux targets retain the conditional `iptvnator` wrapper and
`iptvnator.bin`. Mixed Flatpak/non-Flatpak target sets fail before mutation. A
missing or unsupported profile, or a target from another profile, fails
packaging.
Linux frame-copy release artifacts are x64-only. Non-x64 packages are always
marker-only even if environment variables point at the x64 staged runtime.
## Build Integration
`apps/electron-backend/build-embedded-mpv.js` builds the addon, frame reader,
and helper against the staged inputs. On Linux it links the helper to the
verified staged libmpv path rather than a generic host `-lmpv`, then checks
with `readelf` that:
- the helper has exactly the declared libmpv `DT_NEEDED`;
- the helper RUNPATH is `$ORIGIN/lib`;
- the addon and frame reader have no libmpv dependency;
- no runtime dependency contains an absolute/build-prefix loader path.
The package hook copies native artifacts into
`app.asar.unpacked/electron-backend/native/`, selects the system or bundled
layout, restores exact file modes, writes the packaged manifest, and validates
the bundled legal payload. AppImage, Snap, and Flatpak receive
`embedded-mpv-notices.json`, `THIRD_PARTY_NOTICES.txt`, and
`licenses/<package>/**`; DEB, RPM, Pacman, and marker-only packages must not
retain them. Package validation also scans the Electron executable and all
shipped Electron libraries for a direct libmpv dependency. Before target
packaging, that scan is recursive over the pristine Electron tree. After Snap
has merged its template runtime into the payload root, the post-target scan
excludes exactly its package-manager `lib/**` and `usr/lib/**` trees while
remaining recursive everywhere else.
At startup, Linux x64 frame-copy is advertised only after the main process
validates that manifest/files and successfully executes:
```bash
iptvnator_mpv_helper --runtime-probe
```
The bounded probe initializes idle libmpv plus EGL/OpenGL and mpv render
contexts, then creates, maps, validates, and destroys a minimal `16x16`
shared-memory ring named `/impv-fc-runtime-probe-<pid>`. It does not open media
or enter media/command loops. A timeout, loader failure, malformed protocol,
missing file, hash mismatch, unusable graphics path, or shm lifecycle failure
returns a stable reason and keeps the BrowserWindow sandbox enabled. The
application diagnostic retains `helper-probe-failed` as the top-level reason
for nonzero helper exits and adds `helperReason` only when the helper emitted
one exact protocol-v1 line with a fixed allowlisted reason. Its optional
`helperDetail` is restricted to 1–1024 printable ASCII characters; invalid
detail suppresses both helper fields. Every probe has the same explicit 16 MiB
aggregate captured-output ceiling, regardless of tracing. With
`IPTVNATOR_TRACE_PLAYER=1`, non-empty captured helper stderr is written
separately as one JSON-escaped stderr line: its `stderr` field contains at most
the first 16,384 characters and its `truncated` boolean is always explicit.
Empty captures, disabled tracing, and trace-writer failures do not emit a
record or alter availability. The installed-Snap probe therefore tests the
private shared-memory confinement
needed by playback rather than only loader and graphics startup.
Packaging CI invokes the same gate through
`snap run iptvnator --embedded-mpv-runtime-probe`. This packaging-only
application switch runs before BrowserWindow startup, emits one availability
JSON line, and returns zero only for a usable runtime; it never directly loads
libmpv in Electron. The installed-Snap smoke adds `EGL_LOG_LEVEL=debug` and
`LIBGL_DEBUG=verbose` under that bounded trace channel to expose GLVND/Mesa
loader failures without weakening the hostile-environment gate.
Electron Builder excludes `electron-backend/native{,/**/*}` from `app.asar`.
Only `afterPack` writes the profile-normalized
`app.asar.unpacked/electron-backend/native` tree. Layout and final-artifact
verification enumerate `app.asar` and reject any stale native entry, preventing
hidden x64 helpers, bundled libraries, or notices in system and marker-only
packages.
## CI And Source Distribution
Linux CI builds or restores the pinned source runtime once, then packages and
verifies `system`, `portable`, and `flatpak` independently. Every artifact is
extracted for manifest, mode, package-metadata, ELF-isolation, and helper-probe
checks. System formats are probed after their declared dependency is installed;
Snap and Flatpak also require a sandboxed probe where the runner supports it.
`tools/packaging/verify-linux-frame-copy-runtime.mjs` bounds its helper probe at
`PACKAGE_VERIFICATION_PROBE_TIMEOUT_MS` (15 s), not the application gate's
`RUNTIME_PROBE_TIMEOUT_MS` (3 s): the short budget exists because the app's
decision blocks the Electron main process, whereas nothing waits on the
packaging probe but the CI job's own timeout, and a premature kill would call a
healthy package broken. A hard timeout is the one outcome that says nothing
about the payload, so the verifier repeats it up to
`PACKAGE_VERIFICATION_PROBE_MAX_ATTEMPTS` (2) times with the identical bounded
launch and announces the retry on stderr. Every other outcome — spawn error,
signal, nonzero exit, malformed protocol line — stays fail-closed on the first
attempt, and a helper that keeps hanging still fails once the attempts are
spent. Both constants live in `runtime-probe-contract.cjs`.
For a locally installed `--dangerous` Snap, CI explicitly installs and
connects `mesa-core22` and `gnome-3-28-1804`, verifies both connections, and
then runs the application-level diagnostic under Xvfb.
The Linux packaging matrix alone depends on the runtime-builder job. macOS and
Windows use an independent matrix, while both matrices share the same anchored
step list; draft release assembly remains atomic and requires both matrices.
The Linux runtime cache contains only staged headers/libraries/manifest plus
immutable source inputs: exact downloaded archives (including hwdata), a clean
recursive libplacebo checkout, and collected license files. It never caches
finished notices or the compliance tarball. After either a build or cache hit,
CI revalidates those inputs, regenerates `vendor/embedded-mpv/linux-x64/notices`
for the current runtime manifest, and creates
`linux-frame-copy-runtime-sources.tar.xz` for the current repository
revision/diff. Before archiving, the clean cached libplacebo checkout is
converted into a non-dereferenced working-tree snapshot with every `.git`
entry removed; the validated main/submodule commits remain in the source
index.
That source-compliance archive uses normalized tar metadata and contains the
exact unique archive hash set, VCS-free libplacebo sources and the exact pinned
six recursive submodule records, license inputs, generated notices,
runtime/source index metadata, and the builder, stager, manifest,
notice-generator, and source-snapshot code.
Submodule identity is canonicalized as `full-commit safe/path`; optional
clone-depth-dependent `git describe` annotations are discarded.
The source index carries a globally sorted inventory of every libplacebo
directory, file, and symlink. Regular-file hashes, sizes, normalized executable
bits, exact safe link targets, aggregate counts/bytes, and the canonical
inventory digest are checked against the trusted pinned v7.360.1 checkout. The
tar has an exact member/type layout, and `metadata/archive-sha256.txt` is
checked against the actual source archive bytes. Listing continues past every
tar end marker so concatenated xz streams cannot hide undeclared members.
The notice generator rejects missing, undeclared, symlinked, size-mismatched,
or hash-mismatched license files.
Once CI creates the final `linux-frame-copy-runtime-sources.tar.xz`, it writes
`source-archive-binding.json` beside the staged runtime with the archive's
SHA-256 and repository revision. Bundled x64 AppImage, Snap, and Flatpak
manifests copy that exact object as `sourceArchive`; system packages and
marker-only non-x64 packages omit it.
The packaged x64 Playwright smoke depends on its fixture-contract target and
passes Chromium `--ignore-gpu-blocklist` so Mesa llvmpipe can provide WebGL2 in
CI. This affects only Chromium's software-renderer admission; the manifest,
hash, loader, and helper probes still fail closed, and `--no-sandbox` remains
root-only.
Snap publication is a separate `release.published` workflow for public stable
GitHub releases, with a `workflow_dispatch` retry from `master` for an existing
public stable tag. Both paths resolve the release through the API before
checking out its tag; draft, prerelease and mismatched event IDs are rejected.
It verifies that the public release already contains at least
one Snap and exactly one non-empty
`linux-frame-copy-runtime-sources.tar.xz` before uploading anything. The
release verifier hashes the downloaded archive, checks its clean released
revision, exact member/type layout and safe link targets, source checksum
metadata, source index, actual pinned source-member hashes, six recursive
libplacebo submodule records, legal payload, exact trusted libplacebo tree
inventory/digest, released tooling, and runtime manifest. Checkout and both
artifact-transfer actions use full pinned commits, and checkout does not
persist its repository credential. The verifier bounds
source members, the archive, SquashFS listing, extracted size, entry count,
command time, and job time; every Snap must use the canonical snap-root
layout (Electron app at `/`, so `/iptvnator.bin` and `/resources/**`) and
pass the existing static package validator.
The public-release boundary also reapplies the exact strict
`meta/snap.yaml` graphics/shared-memory/layout contract and enumerates the
extracted `resources/app.asar`, rejecting any archived
`electron-backend/native/**` payload. Its bounded ASAR header reader depends
only on Node built-ins and released local tooling, so verification remains
runnable in the clean tag checkout without `node_modules`.
Exactly one x64 Snap is accepted, and only when its exact `sourceArchive` and
`sourceRuntime` match the downloaded archive; any non-x64 Snap must be
marker-only. A secretless job copies each asset through a no-follow descriptor,
checks hashes before and after inspection, writes an exact receipt, fully
reverifies a root-owned read-only snapshot, and transfers only that data
through the pinned artifact service while publishing the exact receipt digest
separately as a job output.
The dependent publish job runs on a bounded GitHub-hosted `ubuntu-latest`
runner with no checkout or release-tag code. It verifies that separate digest,
the exact receipt schema, every asset size/hash, and the expected regular-file
layout, rejects links and extras, root-seals the transferred data again, and
installs the official stable Snapcraft snap. Snapcraft creates temporary
metadata-extraction siblings beside its input, so the publisher creates
root-owned read-only hard links in a separate root-owned sticky directory.
The uploader can create temporary siblings but cannot modify or replace those
inputs; the original sealed snapshot supplies the upload filename list.
Only its final fixed shell step
receives the Store credential; it executes no released code, resolves no PATH
command, and passes the credential only to each exact
`/snap/bin/snapcraft upload --release=edge` process. GitHub credentials remain
scoped to asset selection/download. Candidate/stable
promotion is manual after installed-Snap frame-copy and missing-runtime
fallback smoke; GitHub Actions never promotes automatically.
Windows CI stages the x64 LGPL archive selected by the checked-in validated pin
described above. PR, master, and tag builds consume that same record; repository
variables and fallback URLs are not build inputs. The DLL basename encoded in
the archive's import library is preserved and must be present beside
`iptvnator_mpv_helper.exe`. The weekly workflow rotates the reviewed pin before
the upstream's latest-30-build retention removes it. A permanent mirror still
requires the complete corresponding source/build records, license notices, and
validated transitive license closure beside the binary.
## Local Development
Linux can use distribution development packages for an unshipped local build
(`libmpv-dev`, EGL/GL/GBM development files, and X11 headers). Overrides:
`LIBMPV_INCLUDE_DIR` selects the header root. `LINUX_NATIVE_LIBRARY_DIR`
selects a link-time library directory that must already be visible to the
system dynamic loader; it is not inherited as a helper `LD_LIBRARY_PATH`.
Required/release package builds must use the pinned staged runtime and manifest.
On macOS:
```bash
pnpm run serve:backend:embedded-mpv
```
This explicitly permits Homebrew for the local native build and enables the
experiment. It is not a release path.
See `docs/architecture/embedded-mpv-native.md` for the runtime capability,
fallback, controls, and packaged-release contracts.