Files
iptvnator/apps/electron-backend/native/helper/frame_helper_render.h
T
4grayandClaude Fable 5 7d75d989e8 feat(embedded-mpv): Linux port of the frame-copy rendering engine (headless EGL) (#1171)
* feat(embedded-mpv): Linux frame-copy helper via headless EGL

Port the frame-copy engine's native layer to Linux (PORTING.md items 1-4):

- frame_helper_gl.h: platform GlContext abstraction. macOS keeps the CGL
  path (moved verbatim); Linux acquires an EGL display in order
  surfaceless-Mesa -> default display -> GBM render node, binds a 3.2 core
  desktop-GL context surfaceless (1x1 pbuffer fallback), and hands mpv
  eglGetProcAddress. The helper's own GL calls link against glvnd
  libOpenGL, so no display server is required.
- frame_shm.h: portable frame_shm_now_ns() (CLOCK_MONOTONIC) shared by the
  helper and the reader addon, replacing the macOS-only
  clock_gettime_nsec_np(CLOCK_MONOTONIC_RAW); producer and consumer stay on
  the same clock.
- embedded_mpv_frame_reader.c: real implementation now also on __linux__
  (the code was already POSIX apart from the clock call).
- binding.gyp: OS==linux executable branch for iptvnator_mpv_helper linking
  system libmpv (-lmpv) + EGL/OpenGL/gbm, with rpaths for $ORIGIN/lib and
  the build-time library dir. The in-process addon still does not link
  libmpv - the ban only binds in-process, the helper is out of process.
- build-embedded-mpv.js: system-dev fallback on Linux (LIBMPV_INCLUDE_DIR
  or /usr/include) so a distro libmpv-dev install builds without staging a
  vendored runtime; a pre-set LINUX_NATIVE_LIBRARY_DIR now wins over the
  vendored lib dir.

Verified on Ubuntu 25.04 / i7-1165G7 (Iris Xe): lavfi smoke per PORTING.md
(idle->loading->playing snapshots at 4 Hz, aspect-fit generation bump
g1 1280x720 -> g2 960x720 for a 4:3 source), reader probe 60 fps at
1080p60 with 0 torn reads, clean quit with no leaked processes or shm.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(embedded-mpv): enable the frame-copy engine gates on Linux

Flip the TypeScript side of the Linux port (PORTING.md item 5). A shared
dependency-free predicate, isFrameCopyPlatformSupported() (linux any-arch,
darwin arm64-only), now backs all four gates so they cannot drift:

- main.ts: the persisted Settings toggle promotes to the env flag on Linux
  too (this runs before window creation and controls the sandbox relax).
- EmbeddedMpvNativeService.isFrameCopyEngineActive/isFrameCopyAvailable.
- EmbeddedMpvFrameCopyAdapter.isSupported.

getSupport() ordering: the frame-copy branch moves above the Linux-only
native-engine prerequisites - the X11/Xwayland display-server check and
the system-mpv-on-PATH probe only bind the --wid native engine, while the
frame-copy helper renders offscreen (headless EGL) and links libmpv
itself. createSession() also skips resolving the native window handle for
frame-copy sessions, which the adapter ignores anyway, so native-Wayland
sessions no longer trip the window-handle assertion.

Settings copy: the i18n frame-copy description now says macOS (Apple
Silicon) and Linux in all 18 languages; stale macOS-only doc comments in
the settings/support interfaces updated alongside.

Tests: platform-gate matrix for the adapter (darwin arm64/x64, linux
x64/arm64, win32) and service specs covering Linux activation under
native Wayland, macOS arm64 staying active, macOS x64 staying native, and
the skipped window handle for frame-copy sessions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(packaging): CI + package guards for the Linux frame-copy helper

- build-and-make.yaml: install libegl-dev/libgl-dev/libgbm-dev on the
  Linux runner (the helper's EGL backend needs them now that the helper
  target builds on Linux), and verify the built helper exists and DOES
  link libmpv - the inverse of the addon's no-libmpv rule, which still
  holds and stays validated.
- electron-after-pack.cjs: strip iptvnator_mpv_helper from packaged Linux
  apps. It links the build host's system libmpv, which end-user systems
  cannot be assumed to have; the support probe treats the missing helper
  as frame-copy-unavailable (dev-build-only engine until the
  bundled-runtime staging milestone).
- frame_helper_gl.h: log the chosen EGL display tier to stderr (the
  adapter mirrors helper stderr), so bring-up problems on exotic setups
  are diagnosable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(embedded-mpv): document the Linux frame-copy port

- architecture doc: frame-copy section covers Linux (EGL display tiers,
  build deps, package strip), Linux support matrix notes the frame-copy
  exception to the X11 + system-mpv requirements, Linux measured baseline.
- RESULTS.md: Ubuntu 25.04 / i7-1165G7 (Iris Xe) measurement rows via the
  production helper + reader probe; viewport-size claim reproduced.
- PORTING.md: Linux marked done with pointers to what changed; Windows
  remains the open port and its perf gate the open decision.
- CLAUDE.md + tools/embedded-mpv/README.md: platform scope, Linux dev
  build requirements, system-headers fallback, helper strip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(embedded-mpv): commit the Linux frame-copy measurement probe

linux-frame-probe.mjs reproduces the RESULTS.md Linux rows: spawns the
production helper, attaches the frame-reader addon to the announced shm
generation, and reports new-frame fps, copy wall time, produce->copy age,
torn reads and pixel spread. Usage documented in RESULTS.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(embedded-mpv): address multi-agent review findings on the Linux port

Confirmed findings (each verified by 3 adversarial reviewers):

- CI would fail to link the helper: -lOpenGL needs the unversioned glvnd
  libOpenGL.so, shipped only by libopengl-dev, which neither the runner
  images nor the previous apt line provide. Added to the workflow and to
  every documented Linux build-dep list.
- The new 'test -x' dist guard could never pass: webpack's dist asset
  copy drops file modes (helper arrives as 0644). The guard is now
  'test -f'; electron-after-pack.cjs restores the execute bit on packaged
  helpers (also fixes packaged-macOS spawns); the support probe now
  requires X_OK, so a mode-stripped helper reads as frame-copy-unavailable
  and falls back to native instead of failing spawn with EACCES.
- The Settings frame-copy toggle was unreachable in exactly the Linux
  states the port targets: the native-Wayland and missing-system-mpv
  unsupported payloads omitted frameCopyAvailable, and toggle visibility
  derives solely from it. Both returns now advertise availability.

Also from review:

- build-embedded-mpv.js keeps the old graceful-skip contract when the new
  system-dev fallback finds libmpv-dev but the GL/EGL/gbm dev stack is
  missing (previously such machines skipped; a hard electron-build
  failure was a regression).
- createSession derives the window-handle skip from the dispatched addon
  instead of re-evaluating the engine gate, so the two cannot disagree.
- The render thread logs the GL renderer string (surfaceless Mesa can
  silently pick llvmpipe on non-Mesa-primary systems; now diagnosable —
  verified 'Mesa Intel Iris Xe' on this machine).
- Specs pin the new semantics: frameCopyAvailable advertised while native
  is unsupported (Wayland / no mpv), frame-copy supported without a
  system mpv, and the handle-skip test disposes its session through the
  owning adapter.
- Docs: PORTING.md file map reflects the frame_helper_gl.h seam for the
  Windows porter; helper-strip removal correctly gated on milestone 4
  (bundled libmpv), not milestone 3; RESULTS.md preamble notes the
  RAW->MONOTONIC clock change; stale '(macOS)' scope comments updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(embedded-mpv): address Greptile/Codex review comments

- Sandbox gate requires a usable helper (Greptile P1, security): the
  main.ts env promotion now also probes for an executable
  iptvnator_mpv_helper before relaxing the window sandbox — a stale
  opt-in on packaged Linux (helper deliberately stripped) or after a
  cleaned native build no longer costs a sandboxless launch for an
  engine that cannot activate. Helper discovery (addon candidate paths +
  X_OK probe) moved into embedded-mpv-frame-copy-platform.util.ts,
  shared by main.ts and the service; the service keeps thin instance
  wrappers so tests can stub per scenario. New util spec pins the
  platform matrix, candidate resolution, and the execute-bit semantics.
- Stale frame-copy artifacts on skipped builds (Codex P2): cleanOutput()
  now also removes iptvnator_mpv_helper and
  embedded_mpv_frame_reader.node, so a failed/skipped rebuild cannot
  leave a previous helper advertising frame-copy support against a
  runtime the build just declared unavailable.
- Multiarch default lib dir (Greptile P1, partially refuted): -l
  resolution never depended on our -L (the compiler's built-in search
  paths include the Debian/Ubuntu multiarch dir — proven by the green CI
  run linking with a nonexistent -L dir), but the system-dev fallback
  now defaults to /usr/lib/<multiarch-triple> when present so the -L
  flag and the helper's baked rpath point somewhere real.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(embedded-mpv): harden Linux frame-copy port

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 20:42:50 +02:00

379 lines
12 KiB
C++

/*
* Offscreen render + readback pipeline for the frame-copy helper.
*
* Headless GL context (per-platform, see frame_helper_gl.h), mpv render API
* into an FBO sized to the viewport, async PBO readback ring, BGRA frames
* published into the FrameShm ring. Validated in spikes/mpv-frame-copy
* (4K60 sustained on M1 Pro).
*
* Threading: everything here runs on the render thread except
* requestResize()/stop(), which only touch atomics/mutex-guarded state.
*/
#pragma once
#include <mpv/client.h>
#include <mpv/render_gl.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>
#include <atomic>
#include <chrono>
#include <condition_variable>
#include <cstring>
#include <functional>
#include <mutex>
#include <string>
#include "frame_helper_gl.h"
#include "frame_helper_io.h"
#include "frame_shm.h"
namespace frame_helper {
inline uint64_t nowNs() { return frame_shm_now_ns(); }
struct ShmRing {
FrameShmHeader* header = nullptr;
uint8_t* base = nullptr;
size_t size = 0;
std::string name;
bool create(const std::string& shmName, int width, int height,
uint32_t generation) {
destroy();
shm_unlink(shmName.c_str());
const int fd = shm_open(shmName.c_str(), O_CREAT | O_EXCL | O_RDWR, 0600);
if (fd < 0) return false;
const uint64_t frameBytes = (uint64_t)width * 4u * (uint64_t)height;
const uint64_t dataOffset =
(sizeof(FrameShmHeader) + FRAME_SHM_DATA_ALIGN - 1) &
~(uint64_t)(FRAME_SHM_DATA_ALIGN - 1);
const size_t total =
(size_t)(dataOffset + FRAME_SHM_RING_SLOTS * frameBytes);
if (ftruncate(fd, (off_t)total) != 0) {
close(fd);
shm_unlink(shmName.c_str());
return false;
}
void* mapped =
mmap(nullptr, total, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
close(fd);
if (mapped == MAP_FAILED) {
shm_unlink(shmName.c_str());
return false;
}
std::memset(mapped, 0, sizeof(FrameShmHeader));
auto* hdr = static_cast<FrameShmHeader*>(mapped);
hdr->version = FRAME_SHM_VERSION;
hdr->width = (uint32_t)width;
hdr->height = (uint32_t)height;
hdr->stride = (uint32_t)width * 4u;
hdr->generation = generation;
hdr->frame_bytes = frameBytes;
hdr->data_offset = dataOffset;
std::atomic_thread_fence(std::memory_order_release);
hdr->magic = FRAME_SHM_MAGIC;
header = hdr;
base = static_cast<uint8_t*>(mapped);
size = total;
name = shmName;
return true;
}
uint8_t* slotData(uint64_t seq) const {
return base + header->data_offset +
(seq % FRAME_SHM_RING_SLOTS) * header->frame_bytes;
}
void destroy() {
if (base) {
munmap(base, size);
shm_unlink(name.c_str());
}
header = nullptr;
base = nullptr;
size = 0;
name.clear();
}
};
class RenderPipeline {
public:
/* Called from the render thread after a resize created a new shm
* generation, so the main protocol layer can announce it. */
std::function<void(const std::string& name, int width, int height,
uint32_t generation)>
onGenerationChanged;
bool start(mpv_handle* mpv, const std::string& shmBaseName, int width,
int height, std::string& errorOut);
void requestResize(int width, int height);
void stop();
void notifyUpdate(); /* mpv render update callback -> wake render loop */
void runLoop(); /* render thread body */
mpv_render_context* renderContext() { return renderContext_; }
/* 0 = initializing, 1 = ready, -1 = failed */
int initState() const { return initState_.load(); }
private:
bool setupGl(std::string& errorOut);
bool rebuildTargets(int width, int height);
void publishPending();
void renderFrame();
mpv_handle* mpv_ = nullptr;
GlContext gl_;
mpv_render_context* renderContext_ = nullptr;
std::string shmBaseName_;
ShmRing ring_;
uint32_t generation_ = 0;
int width_ = 0;
int height_ = 0;
GLuint texture_ = 0;
GLuint fbo_ = 0;
GLuint pbos_[FRAME_SHM_RING_SLOTS] = {0};
int cursor_ = 0;
int pendingPbo_ = -1;
uint64_t nextSeq_ = 1;
std::mutex mutex_;
std::condition_variable cv_;
bool updatePending_ = false;
bool stopRequested_ = false;
int pendingWidth_ = 0;
int pendingHeight_ = 0;
std::atomic<int> initState_{0};
};
inline bool RenderPipeline::start(mpv_handle* mpv,
const std::string& shmBaseName, int width,
int height, std::string& errorOut) {
mpv_ = mpv;
shmBaseName_ = shmBaseName;
width_ = width;
height_ = height;
return gl_.create(errorOut);
}
inline bool RenderPipeline::setupGl(std::string& errorOut) {
if (!gl_.makeCurrent(errorOut)) return false;
/* Keep the accepted renderer diagnosable. Linux already rejected earlier
* software tiers when a later hardware-backed EGL candidate was usable. */
const GLubyte* renderer = glGetString(GL_RENDERER);
if (renderer) {
std::fprintf(stderr, "gl renderer: %s\n",
reinterpret_cast<const char*>(renderer));
}
if (!rebuildTargets(width_, height_)) {
errorOut = "framebuffer setup failed";
return false;
}
mpv_opengl_init_params glInit = {gl_.procLoader(), gl_.procLoaderCtx()};
mpv_render_param params[] = {
{MPV_RENDER_PARAM_API_TYPE,
const_cast<char*>(MPV_RENDER_API_TYPE_OPENGL)},
{MPV_RENDER_PARAM_OPENGL_INIT_PARAMS, &glInit},
{MPV_RENDER_PARAM_INVALID, nullptr},
};
const int result = mpv_render_context_create(&renderContext_, mpv_, params);
if (result < 0) {
errorOut = mpv_error_string(result);
return false;
}
return true;
}
inline bool RenderPipeline::rebuildTargets(int width, int height) {
if (texture_) glDeleteTextures(1, &texture_);
if (fbo_) glDeleteFramebuffers(1, &fbo_);
if (pbos_[0]) glDeleteBuffers(FRAME_SHM_RING_SLOTS, pbos_);
pendingPbo_ = -1;
cursor_ = 0;
width_ = width;
height_ = height;
const GLsizeiptr frameBytes = (GLsizeiptr)width * 4 * height;
glGenTextures(1, &texture_);
glBindTexture(GL_TEXTURE_2D, texture_);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8, width, height, 0, GL_RGBA,
GL_UNSIGNED_BYTE, nullptr);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glGenFramebuffers(1, &fbo_);
glBindFramebuffer(GL_FRAMEBUFFER, fbo_);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D,
texture_, 0);
glGenBuffers(FRAME_SHM_RING_SLOTS, pbos_);
for (GLuint pbo : pbos_) {
glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo);
glBufferData(GL_PIXEL_PACK_BUFFER, frameBytes, nullptr, GL_STREAM_READ);
}
glBindBuffer(GL_PIXEL_PACK_BUFFER, 0);
if (glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE) {
return false;
}
generation_ += 1;
const std::string shmName =
shmBaseName_ + "-g" + std::to_string(generation_);
if (!ring_.create(shmName, width, height, generation_)) {
return false;
}
if (onGenerationChanged) {
onGenerationChanged(shmName, width, height, generation_);
}
return true;
}
inline void RenderPipeline::notifyUpdate() {
{
std::lock_guard<std::mutex> lock(mutex_);
updatePending_ = true;
}
cv_.notify_all();
}
inline void RenderPipeline::requestResize(int width, int height) {
{
std::lock_guard<std::mutex> lock(mutex_);
pendingWidth_ = width;
pendingHeight_ = height;
updatePending_ = true;
}
cv_.notify_all();
}
inline void RenderPipeline::stop() {
{
std::lock_guard<std::mutex> lock(mutex_);
stopRequested_ = true;
}
cv_.notify_all();
}
inline void RenderPipeline::publishPending() {
if (pendingPbo_ < 0 || !ring_.header) return;
glBindBuffer(GL_PIXEL_PACK_BUFFER, pbos_[pendingPbo_]);
const void* mapped = glMapBufferRange(GL_PIXEL_PACK_BUFFER, 0,
(GLsizeiptr)ring_.header->frame_bytes,
GL_MAP_READ_BIT);
if (mapped) {
const uint64_t seq = nextSeq_++;
FrameShmSlot& slot =
ring_.header->slots[seq % FRAME_SHM_RING_SLOTS];
slot.seq.store(0, std::memory_order_release);
std::memcpy(ring_.slotData(seq), mapped,
(size_t)ring_.header->frame_bytes);
slot.produce_time_ns = nowNs();
slot.seq.store(seq, std::memory_order_release);
ring_.header->latest_seq.store(seq, std::memory_order_release);
glUnmapBuffer(GL_PIXEL_PACK_BUFFER);
}
glBindBuffer(GL_PIXEL_PACK_BUFFER, 0);
pendingPbo_ = -1;
}
inline void RenderPipeline::renderFrame() {
mpv_opengl_fbo fbo = {(int)fbo_, width_, height_, 0};
int flipY = 1;
mpv_render_param params[] = {
{MPV_RENDER_PARAM_OPENGL_FBO, &fbo},
{MPV_RENDER_PARAM_FLIP_Y, &flipY},
{MPV_RENDER_PARAM_INVALID, nullptr},
};
mpv_render_context_render(renderContext_, params);
glBindFramebuffer(GL_READ_FRAMEBUFFER, fbo_);
glReadBuffer(GL_COLOR_ATTACHMENT0);
glPixelStorei(GL_PACK_ALIGNMENT, 1);
glPixelStorei(GL_PACK_ROW_LENGTH, 0);
publishPending();
glBindBuffer(GL_PIXEL_PACK_BUFFER, pbos_[cursor_]);
glReadPixels(0, 0, width_, height_, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV,
nullptr);
glBindBuffer(GL_PIXEL_PACK_BUFFER, 0);
pendingPbo_ = cursor_;
cursor_ = (cursor_ + 1) % FRAME_SHM_RING_SLOTS;
mpv_render_context_report_swap(renderContext_);
}
inline void RenderPipeline::runLoop() {
std::string glError;
if (!setupGl(glError)) {
emitLine(JsonWriter()
.str("event", "fatal")
.str("error", "render init failed: " + glError)
.finish());
initState_.store(-1);
gl_.destroy();
ring_.destroy();
return;
}
initState_.store(1);
while (true) {
int resizeWidth = 0;
int resizeHeight = 0;
bool targetsRebuilt = false;
{
std::unique_lock<std::mutex> lock(mutex_);
cv_.wait_for(lock, std::chrono::milliseconds(100), [&] {
return updatePending_ || stopRequested_;
});
if (stopRequested_) break;
updatePending_ = false;
resizeWidth = pendingWidth_;
resizeHeight = pendingHeight_;
pendingWidth_ = 0;
pendingHeight_ = 0;
}
if (resizeWidth > 0 && resizeHeight > 0 &&
(resizeWidth != width_ || resizeHeight != height_)) {
if (!rebuildTargets(resizeWidth, resizeHeight)) {
emitLine(JsonWriter()
.str("event", "fatal")
.str("error", "resize target rebuild failed")
.finish());
break;
}
targetsRebuilt = true;
}
const uint64_t flags = mpv_render_context_update(renderContext_);
/* A resize installs an empty shm ring. Redraw the current frame even
* while paused; the async PBO is published on the next loop. */
if (targetsRebuilt || (flags & MPV_RENDER_UPDATE_FRAME)) {
renderFrame();
} else {
/* Flush a stranded readback so pause keeps the last frame. */
publishPending();
}
if (ring_.header) {
ring_.header->heartbeat_ns.store(nowNs(),
std::memory_order_relaxed);
}
}
mpv_render_context_set_update_callback(renderContext_, nullptr, nullptr);
mpv_render_context_free(renderContext_);
renderContext_ = nullptr;
if (texture_) glDeleteTextures(1, &texture_);
if (fbo_) glDeleteFramebuffers(1, &fbo_);
if (pbos_[0]) glDeleteBuffers(FRAME_SHM_RING_SLOTS, pbos_);
gl_.destroy();
ring_.destroy();
}
} // namespace frame_helper