refactor(mediabunny): [PKG-MB-05] own video iterators, frame pools and rendering

This commit is contained in:
Harvey Zhao committed 2026-09-12 19:47:46 +08:00
1 parent 5d0b128483
commit 9d037dcbf9
27 files changed
+3652 -595

No files matched your search

@@ -19,18 +19,22 @@ declarations, the factory signature, canvas forwarding, or package entrypoints.
| `playback.ts` | Playback intent, cancellable play/seek/replacement operations and shared pending audio resume |
| `readiness.ts` | One metadata publication after both participants, guarded event sequences |
| `hls-selection.ts` | Source-bound track queries, pairing and active replacement errors |
| `VideoEngine.js` / `AudioEngine.js` | Current native decoding, rendering, seeking, Web Audio clock and scheduling |
| `VideoEngine.ts` | Video capability/setup, source and seek ownership, first frames and terminal cleanup |
| `video-frames.ts` | Shared pending iterator reads, idempotent iterator release and existing frame timing policy |
| `video-renderer.ts` | Owned RAF, stale frame rejection, clock-driven draw/stall/end events |
| `video-poster.ts` | Per-request image callbacks, source guards and handler cleanup |
| `AudioEngine.js` | Remaining Web Audio clock, audio decoding and scheduling implementation |
| `VideoShim.ts` / `EventTarget.ts` | Video-like descriptors, synchronous/Promise forwarding and owned event lifecycle |
| `shim-values.ts` | Existing volume coercion and synthetic TimeRanges values |
| `shim-frames.ts` | Synthetic frame callbacks, per-instance RAF ownership and terminal cleanup |
| `engine-types.ts` / `engine-ports.ts` | Internal shim, coordinator and decoder interfaces |
| `AudioEngine.d.ts` / `VideoEngine.d.ts` | Temporary boundaries for JS decoders, removed in MB-06/05 |
| `AudioEngine.d.ts` | Temporary boundary for the remaining JS audio engine, removed in MB-06 |
| `m3u8.js` | Current ArtPlayer control/setting integration |
The sixteen production TypeScript modules use strict checking with `skipLibCheck: false`
and no ambient Node types. The coordinator is an actual checked implementation. The two
adjacent decoder declarations are temporary boundaries, not checking of their JavaScript
implementations. Remaining JavaScript is scheduled in PKG-MB-05/06/07/08.
The twenty production TypeScript modules use strict checking with `skipLibCheck: false`
and no ambient Node types. The coordinator and video engine are checked implementations.
The adjacent audio declaration is a temporary boundary, not checking of its JavaScript
implementation. Remaining JavaScript is scheduled in PKG-MB-06/07/08.
Do not describe the whole package as migrated yet.
The shim keeps its existing own-property order and direct prototype surface because
@@ -82,7 +86,7 @@ for the seek completion signal, resolved before optional resume to avoid a cycle
A pending AudioContext resume is shared across play-pause-play; only the current operation
starts video and emits play/playing. Source change and destroy invalidate both operations.
Cancellation settles the public operation while observing the underlying late result;
actual late decoder work still belongs to MB-05/06.
video frame ownership is detailed below; remaining audio work belongs to MB-06.
Normal play, pause, seek/resume and HLS replacement event order is retained. Before each
event the coordinator rechecks ownership, since a listener may synchronously pause, seek,
@@ -99,6 +103,43 @@ This corrects the old false-ready state; normal valid input event order is prese
Existing-track unsupported-codec handling remains a decoder capability task, not evidence
that every capability failure has been fixed.
## Video frame ownership
VideoEngine keeps source/load generation separate from seek iteration and renderer activity.
A new source calls cancelPending before track selection starts, so an old seek cannot draw
while the new source is still parsing. Seek during capability detection waits for that load
without canceling its metadata; only the latest seek may install or draw its frames.
Ordinary paused seek still displays its first target frame and queues the second.
Iterator detachment is synchronous. The old return Promise is observed without blocking a
new load on obsolete reads; repeated release calls return the same cleanup Promise. Late
completion cannot clear the current iterator or its fetching flag. Stop/start shares an
already pending next() for the same iterator instead of issuing concurrent reads. Pause
invalidates render work; it does not cancel a legitimate paused seek's target frame.
Each seek after the initial iterator creates a fresh CanvasSink with its own two-canvas pool.
Concurrent obsolete decoders can otherwise overwrite the pixels of a queued frame even when
the proxy discards their returned objects. Sink identity is internal and may change on seek;
public canvas identity stays stable. Tests intercept sink prototypes to follow new instances.
CanvasSink retains poolSize 2, contain fit and track transparency. Its implementation closes
VideoSample after drawing into the pooled canvas. The proxy releases iterators and references;
WrappedCanvas is not a VideoFrame and must not receive close(). Default dropLateFrames remains
false, with the same tolerance/rate formula when enabled. Start preserves the queued second
frame until it is consumed instead of replacing it through premature prefetch. There is no
added frame copy.
Each renderer owns one RAF, including ID zero. Stop/destroy and event-listener reentry make
queued callbacks inert; an instance that never scheduled a frame cannot cancel another
instance's frame. Normal timeupdate, stall recovery and ended ordering is preserved, while
reentrant pause/source/destroy/restart stops the obsolete event sequence. Poster image
callbacks have source and request identity and are detached on replacement, draw or destroy.
Background decoder failure stops rendering and reports one media error through the coordinator,
even if audio cleanup fails. Iterator cleanup failures are observed and warned; explicit
stopIterator still rejects. Missing 2D context fails at use with a descriptive error, rather
than becoming falsely ready. Capability handling for existing undecodable tracks remains
under MB-CAP-01/MB-READY-01; the current no-video fallback is preserved for that separate review.
## Types and dependencies
The locked runtime SDK remains MediaBunny 1.56.1. Its pinned WebCodecs declaration
@@ -118,6 +159,7 @@ node node_modules/typescript/bin/tsc -p packages/artplayer-proxy-mediabunny/tsco
yarn build artplayer-proxy-mediabunny
yarn test:browser test/browser/mediabunny-inputs.spec.js test/browser/mediabunny.spec.js test/browser/mediabunny-load.spec.js
yarn test:browser test/browser/mediabunny-shim.spec.js
yarn test:browser test/browser/mediabunny-video.spec.js
```
Node tests use actual SDK parsing and controlled lifecycle interleavings. Browser tests