refactor(mediabunny): [PKG-MB-05] own video iterators, frame pools and rendering

2026-10-18 02:21:20 -08:00 · 2026-09-12 19:47:46 +08:00
1 parent 5d0b128483
commit 9d037dcbf9
@@ -0,0 +1,10 @@
+{
+  "task": "PKG-MB-05",
+  "sourceCommit": "5d0b128483a78a5374bb7163adc1421ba6a4d4c8",
+  "sourceSha256LF": {
+    "packages/artplayer-proxy-mediabunny/src/VideoEngine.js": "5ac862b0fd9c196dc6656f8908a214c1b3a0a53a7e85ff845d9068888831986b",
+    "packages/artplayer-proxy-mediabunny/dist/artplayer-proxy-mediabunny.js": "01b88c915804e1c42b90c8c37aa948029472c7f47f6eb3a266629ed945be9fb1",
+    "node_modules/mediabunny/src/media-sink.ts": "9fbec4b6b0ec6c613d8f546fa91d8f14aac1c7c4a53ec7ed14b31319d43510f2"
+  },
+  "sdkNote": "Installed 1.56.1 CanvasSink closes each VideoSample after drawing into its pooled canvas. Consumer owns the iterator return, not VideoFrame.close on WrappedCanvas."
+}
@@ -0,0 +1,80 @@
+# PKG-MB-05：视频迭代器、seek、晚帧与RAF
+
+任务通过完整CI后标记done，并立即以专用本地提交交付。冻结来源为5d0b1284，源码与实际main哈希见
+mb-video-source.json；SDK仍为已锁定MediaBunny 1.56.1，未升级依赖。
+
+## 实际实现
+
+VideoEngine.js已迁移为严格TS并删除VideoEngine.d.ts桥。拆出video-frames.ts、
+video-renderer.ts、video-poster.ts，包内共20个生产TS模块；仅AudioEngine.d.ts仍是
+显式的临时JS边界。ARCHITECTURE.md记录文件地图和状态/资源归属。
+
+- source/load、seek迭代和render活动分别拥有身份；旧load、第一/第二seek帧、能力查询和
+  异步frame读取结果不得覆盖新source或seek。新source开始解析时即取消旧视频操作。
+- 加载能力检测期间seek等待该加载，不能把合法metadata一起取消；暂停状态的正常seek
+  仍显示目标第一帧并保留第二帧。load/reset不等待废弃迭代器无限期返回。
+- 迭代器先同步脱离当前字段，再观察return；每个迭代器只释放一次。stop/start对同一
+  pending next共享读取，旧finally不清掉新fetch状态。
+- RAF每实例单一归属，ID 0有效；暂停、销毁与事件回调重入后不再排程旧任务。
+  未创建RAF的实例不能取消其他实例的ID 0。正常timeupdate/ended/stall事件顺序保持。
+- 海报请求有源身份和请求身份，替换、真正绘制帧、销毁时解除回调。
+- 活动后台解码失败停止渲染并通过协调器报告一次MediaError，音频清理异常不覆盖原错误。
+  后台iterator return拒绝有观察和警告；显式stopIterator保持Promise拒绝。
+- 无2D context时在实际使用阶段报明确错误，替代旧的空值TypeError；不会伪造就绪。
+
+默认dropLateFrames=false、容差/倍速公式、CanvasSink poolSize=2、contain、透明度保持。
+SDK的CanvasSink把VideoSample画入池化canvas后自行close；代理负责迭代器，不对WrappedCanvas
+调用VideoFrame.close，不增加每帧拷贝。公开factory/类型/Canvas门面与三种产物入口保持。
+
+## 证据
+
+38项新视频断言对5d0b1284实际main为7通过/31失败；候选全部通过。包专项146项通过。
+main/legacy各110项通过，106候选断言+4冻结正常对照。三格式构建与严格类型检查通过。
+最终浏览器结果归档并记录在mb-video-validation.json。54项由12普通真实播放、10真实
+解码帧竞态、9原生Stream取消、9synthetic RAF、11Windows WebKit能力对照、3无轨道拒绝组成。
+其中10个新竞态场景对旧main全部失败，另5能力对照通过；失败trace与报告完整保留。
+新场景使用实际SDK解析/解码、实际CanvasSink帧和浏览器时钟，只延迟一帧的交付来控制
+暂停/连续seek/切源时序。timeupdate回调暂停检查真实RAF，不是mock codec或纯事件出口测试。
+
+开发时修正了测试宿主：控制音频seek来隔离视频测试；两个实例的RAF测试直接创建两个实际
+VideoEngine，避免测试helper重复返回同一canvas。旧32项测试中25失败，扩展到37项后30失败。
+初始复现读取必须在下一次返回done，避免旧实现缺陷造成测试自身无限循环；没有扩大超时。
+
+## 后续
+
+MB-06继续实际音频解码、时钟与节点释放；MB-07继续并发HLS选轨和UI拓扑；MB-08完善
+消费者声明矩阵。MB-09仍需要长播放、AV同步和完整核心/插件/设备组合。既有轨道但不可解码
+时的fallback还需MB-CAP-01/MB-READY-01评审，不以这次晚帧修复宣布所有能力问题已关闭。
+MPL通知、8082示例与真实安装包在MB-10；未推送、打tag或发布。
+
+最终复核另复现了启动预取覆盖已排队第二帧，修复为消费该帧后再取下一帧，保持默认
+不丢帧意图及容差策略。新增第38项回归，旧main为7通过/31失败。首轮完整CI1306通过
+之后因这项实际源码修复重新构建、验证产物/浏览器并运行完整CI，不复用旧的绿色结果。
+
+进一步使用两个实际解码VideoSample在进入CanvasSink池化转换前暂停交付，连续seek
+1→3→5后释放旧样本：当前帧timestamp仍属于5秒，但排队canvas像素校验从1220654181
+变成2730873815。此缺陷在已通过第二轮CI的实现上也复现，报告为mb05-pool-before-results。
+已改为每个seek独立CanvasSink池（仍2个canvas，无每帧拷贝），保留公开canvas实例；
+内部sink身份可变化。对应浏览器测试跟踪SDK原型以覆盖新sink，而非绑定一次旧实例。
+修复后像素稳定；最终54项浏览器及对应旧main15项对照另行归档。
+
+构建中两次发生Windows copyfile UNKNOWN，仅覆盖已有docs/compiled legacy目标失败；
+源码和目标读取、r+打开及复制到cache均成功，磁盘空间充足。备份并移除该可再生目标，
+由正常yarn build重新生成后三格式构建成功，没有手改产物内容或扩大文件删除范围。
+首次54项浏览器所用main与重新完整构建的main哈希相同，仍有效；legacy重新运行110项检查。
+
+进一步补充新迭代器保持断言：Firefox旧切源在某次运行中未多画旧帧，却已被旧清理
+清空新迭代器，原画面断言不足以发现。新增断言后最终旧main10个原生竞态均失败，
+5个能力对照通过；先前9失败/6通过报告另存，不把同一副作用的不同调度结果隐去。
+
+最终浏览器清理断言改为按现有expect超时等待AudioContext真实closed状态；close本来是
+异步操作，固定两帧不足以判定失败。此处只调整测试等待，生产代码未改。末次完整CI
+1307项覆盖最终生产源码；之后单独重跑浏览器spec lint和全部54浏览器用例。旧main
+同样重跑15项以保留清理后的原始视频断言结论，未提高超时或跳过失败。
+
+最终完整CI1307项（1152单元+14工程+141基线）通过，另44重复契约；310生产TS。
+源码、测试、产物与报告哈希已核验，三格式docs/compiled与dist逐字节一致。
+38视频断言在旧main为7通过/31失败；最终包专项146、main/legacy各110、浏览器54均通过。
+旧产物10个原生帧竞态失败均为绘制计数、帧像素、迭代器保留或RAF状态断言，不是超时：暂停后绘制2→10，
+旧seek结果在新seek/source后绘制2→3，timeupdate暂停后残留1个RAF；候选均无此残留。
+回退可revert专用提交，原始5d0b1284及基线保持可追溯。下一项PKG-MB-06继续音频引擎。
@@ -54,3 +54,10 @@ VideoShim/EventTarget与主协调器已拆为严格TS模块，详情见
 无轨道容器由历史readyState 4改为code 4错误；有效媒体的正常顺序保持，重复就绪和
 失效操作结果被阻止。不要把synthetic RAF、受控竞态或Windows WebKit能力对照计为真实播放。
 音视频解码器仍通过两个显式声明桥连接，MB-05/06继续实际实现迁移和底层晚操作清理。
+
+## MB-05 video ownership
+
+VideoEngine与迭代器/渲染/海报职责已实际迁移TS，删除视频声明桥；只剩音频声明桥。
+执行yarn test:mediabunny及新增mediabunny-video.spec.js，按实际报告区分普通播放、
+真实帧竞态、能力对照和synthetic RAF。[变更](changes/2026-09-12-PKG-MB-05-video.md)与
+[验证](baselines/mb-video-validation.json)记录38个新断言、旧产物复现及真实解码场景。
@@ -4,7 +4,7 @@

 基线：`40fcda6a37d0049d42e49c1e64e70d4fd9ba5f7f`。总任务 222 项，范围 22 个包及工作区/示例。

-状态：todo 114 / doing 7 / blocked 0 / done 101 / deferred 0。风险 L/M/H 表示兼容风险，不表示工期。
+状态：todo 113 / doing 7 / blocked 0 / done 102 / deferred 0。风险 L/M/H 表示兼容风险，不表示工期。

 前置依赖是启动条件；验收是完成条件。任务可以继续拆分，但不能复用或悄悄删除旧 ID。

@@ -333,7 +333,7 @@
 | PKG-MB-02 | artplayer-proxy-mediabunny<br>建立真实媒体与事件基线 | PKG-MB-01, ENG-05, ENG-08 | MP4/WebM/HLS/Blob/Stream、seek/错误/轨道切换样本 | ready 事件顺序、音画同步、超时和释放可重跑 | H | done |
 | PKG-MB-03 | artplayer-proxy-mediabunny<br>整理 input 与加载取消 | PKG-MB-02, CORE-09 | input.js 源检测、输入资源、Range 与取消 | HLS 检测在 input 边界，旧来源类型和超时保持 | H | done |
 | PKG-MB-04 | artplayer-proxy-mediabunny<br>整理 VideoShim/EventTarget/Engine 协调 | PKG-MB-03, CORE-06, CORE-11 | shim 描述符、事件桥、加载状态和操作隔离 | 公开属性/同步异步返回与事件重入顺序保持 | H | done |
-| PKG-MB-05 | artplayer-proxy-mediabunny<br>整理视频解码、seek 和帧释放 | PKG-MB-04 | VideoEngine 调度、队列、晚帧处理和资源释放 | 暂停/seek/切源无旧帧覆盖，默认丢帧策略不变 | H | todo |
+| PKG-MB-05 | artplayer-proxy-mediabunny<br>整理视频解码、seek 和帧释放 | PKG-MB-04 | VideoEngine 调度、队列、晚帧处理和资源释放 | 暂停/seek/切源无旧帧覆盖，默认丢帧策略不变 | H | done |
 | PKG-MB-06 | artplayer-proxy-mediabunny<br>整理音频解码、时钟与同步 | PKG-MB-04 | AudioEngine 时钟、缓冲和 AudioContext 归属 | AV sync、倍速、无音轨、静音/音量和暂停恢复通过 | H | todo |
 | PKG-MB-07 | artplayer-proxy-mediabunny<br>整理 HLS 配对轨道与 selector | PKG-MB-05, PKG-MB-06, CORE-14 | m3u8 配对、质量/音频选择及拓扑清理 | 实际选择高亮、切到无轨道来源清理、无重复 readiness | H | todo |
 | PKG-MB-08 | artplayer-proxy-mediabunny<br>完成 TS 与媒体能力声明 | PKG-MB-07, ENG-04, ENG-06 | 8 个自有 JS 模块迁移与 Result/shim 类型 | 解码器/Stream/DOM 类型清楚，旧 Option 和 art.mediabunny 使用保持 | H | todo |
@@ -524,4 +524,5 @@
 - PKG-MB-02: [记录](changes/2026-09-12-PKG-MB-02-checkpoint.md) [记录](baselines/mb-media-checkpoint.json) [记录](mb-validation.md) [记录](changes/2026-09-12-PKG-MB-02-baseline.md) [记录](baselines/mb-behavior-validation.json)
 - PKG-MB-03: [记录](changes/2026-09-12-PKG-MB-03-input.md) [记录](baselines/mb-input-validation.json)
 - PKG-MB-04: [记录](baselines/mb-coordination-source.json) [记录](changes/2026-09-12-PKG-MB-04-shim-checkpoint.md) [记录](baselines/mb-shim-checkpoint.json) [记录](changes/2026-09-12-PKG-MB-04-coordination.md) [记录](baselines/mb-coordination-validation.json)
+- PKG-MB-05: [记录](baselines/mb-video-source.json) [记录](changes/2026-09-12-PKG-MB-05-video.md) [记录](baselines/mb-video-validation.json)
 - PKG-FACTORY-01: [记录](baselines/factory-assignment-gaps.json) [记录](baselines/factory-compatibility-proposals.json) [记录](factory-compatibility-decision.md) [记录](changes/2026-09-12-PKG-FACTORY-01-decision.md)
@@ -1,5 +1,19 @@
 # 进度与证据

+## PKG-MB-05完成：视频迭代器、seek、晚帧与RAF
+
+VideoEngine实际迁移TS，拆出frame读取/释放、renderer与poster职责，删除视频声明桥。
+新source立即取消旧视频操作；seek与加载、暂停分别管理，旧帧不覆盖新状态；单实例RAF
+与海报回调有清理；保留默认丢帧策略并修复启动预取覆盖第二帧，以及多seek共享
+Canvas池导致排队帧像素被旧样本改写。包内20生产TS，
+只剩AudioEngine显式声明桥。38项新断言对旧main7通过/31失败、候选全通过；包专项
+146、main/legacy各110通过。最终54浏览器：12普通播放+10真实解码帧竞态+9Stream
+取消+9synthetic RAF+11能力对照+3无轨道拒绝。旧main10项真实帧竞态全部失败，
+候选全部通过，完整报告/trace归档。最终完整CI1307通过，另44重复契约，310生产TS。
+见[变更](changes/2026-09-12-PKG-MB-05-video.md)及[证据](baselines/mb-video-validation.json)。
+222项：102 done、7 doing、113 todo；立即专用本地提交，下一项MB-06音频引擎。
+音频、HLS拓扑、长播放/AV同步与codec能力剩余项继续，不构成npm发布验收。
+
 ## PKG-MB-04完成：主协调器TS与操作隔离

 MediaBunnyEngine实际迁移TS，并拆分playback/readiness/HLS查询职责；正常接口和事件顺序
@@ -3648,7 +3648,7 @@
        "PKG-MB-07",
        "PKG-MB-09"
      ],
-      "compatibleResolution": "Reproduce pending operations and destroy/source-switch races before migrating each owner; preserve public event and promise behavior while preventing stale resource use. MB-02 now reproduces late load rejection and uncancelled waiting/loadstart timers after destroy in all three historical implementations using a controlled host. Other interleavings and full-core native behavior are not yet verified. Expanded MB-02 baseline reproduces play-after-pause/destroy, rejected-play false state, seek-after-destroy and retained successful-load timeout timers in all three actual historical implementations; Range and active-timeout normal controls remain separate. MB-03 candidate owns pending Input before track selection and cancels load/HEAD/timers on source replacement, timeout and destroy; source/main/legacy regressions pass. Pending play/seek, late decoder work and full readiness coordination remain MB-04/05/06, so this risk stays open. MB-04 shim checkpoint fixes synthetic RAF retention and terminal event-listener cleanup, including engine cleanup exceptions. Closing the event bridge does not prove pending engine operations stopped; coordination remains in progress. MB-04 now guards play/seek/replacement completion, preserves explicit pause, observes stale failures and isolates reentrant event sequences. Decoder-internal late frames/audio and HLS UI races remain MB-05/06/07; risk stays open.",
+      "compatibleResolution": "Reproduce pending operations and destroy/source-switch races before migrating each owner; preserve public event and promise behavior while preventing stale resource use. MB-02 now reproduces late load rejection and uncancelled waiting/loadstart timers after destroy in all three historical implementations using a controlled host. Other interleavings and full-core native behavior are not yet verified. Expanded MB-02 baseline reproduces play-after-pause/destroy, rejected-play false state, seek-after-destroy and retained successful-load timeout timers in all three actual historical implementations; Range and active-timeout normal controls remain separate. MB-03 candidate owns pending Input before track selection and cancels load/HEAD/timers on source replacement, timeout and destroy; source/main/legacy regressions pass. Pending play/seek, late decoder work and full readiness coordination remain MB-04/05/06, so this risk stays open. MB-04 shim checkpoint fixes synthetic RAF retention and terminal event-listener cleanup, including engine cleanup exceptions. Closing the event bridge does not prove pending engine operations stopped; coordination remains in progress. MB-04 now guards play/seek/replacement completion, preserves explicit pause, observes stale failures and isolates reentrant event sequences. Decoder-internal late frames/audio and HLS UI races remain MB-05/06/07; risk stays open. MB-05 migrates VideoEngine and owns iterators, late first/second seek frames, pending reads, per-instance RAF and poster callbacks; same native race assertions fail on old main and pass on candidate. Independent CanvasSink pools also prevent multiple obsolete seek samples from overwriting queued current-frame pixels. Audio nodes, long-run AV sync and HLS UI topology remain MB-06/07/09.",
      "closureCriteria": "Historical failing interleavings and candidate cancellation/cleanup pass with real media and event evidence, including long playback and track topology.",
      "status": "open",
      "evidence": [
@@ -3662,7 +3662,9 @@
        "refactor/changes/2026-09-12-PKG-MB-03-input.md",
        "refactor/changes/2026-09-12-PKG-MB-04-shim-checkpoint.md",
        "refactor/changes/2026-09-12-PKG-MB-04-coordination.md",
-        "refactor/baselines/mb-coordination-validation.json"
+        "refactor/baselines/mb-coordination-validation.json",
+        "refactor/changes/2026-09-12-PKG-MB-05-video.md",
+        "refactor/baselines/mb-video-validation.json"
      ]
    },
    {
@@ -3482,11 +3482,15 @@
      "dependsOn": [
        "PKG-MB-04"
      ],
-      "status": "todo",
+      "status": "done",
      "risk": "H",
      "deliverable": "VideoEngine 调度、队列、晚帧处理和资源释放",
      "acceptance": "暂停/seek/切源无旧帧覆盖，默认丢帧策略不变",
-      "evidence": []
+      "evidence": [
+        "baselines/mb-video-source.json",
+        "changes/2026-09-12-PKG-MB-05-video.md",
+        "baselines/mb-video-validation.json"
+      ]
    },
    {
      "id": "PKG-MB-06",