test(auto-thumbnail): [PKG-AUTO-THUMB-03] distinguish render and data readiness

This commit is contained in:
Harvey Zhao committed 2026-09-14 04:48:39 +08:00
1 parent f30c701d88
commit 3e39fc745f
10 files changed
+2948 -5

No files matched your search

+2 -1
View File
@@ -107,7 +107,8 @@
"test:danmuku-types-package": "node refactor/scripts/danmuku-package-types.mjs",
"test:danmuku-mask-types-package": "node refactor/scripts/danmuku-mask-package-types.mjs",
"test:auto-thumbnail-types": "node --test refactor/scripts/auto-thumbnail-types.test.mjs",
"test:auto-thumbnail-types-package": "node refactor/scripts/auto-thumbnail-package-types.mjs"
"test:auto-thumbnail-types-package": "node refactor/scripts/auto-thumbnail-package-types.mjs",
"probe:auto-thumbnail-rendering": "node refactor/scripts/auto-thumbnail-rendering-probe.mjs"
},
"browserslist": "last 1 Chrome version",
"devDependencies": {
@@ -139,6 +139,17 @@ close this risk based on the later-cell assertions or replace it with a nonblack
test; legitimate black frames have opaque pixels. Frame timing, additional
reentry/cleanup boundaries and resource budgets still need the remaining task03 work.
`yarn probe:auto-thumbnail-rendering` captures an isolated diagnostic matrix from
the current built ESM and the frozen timeline media. It compares intrinsic-size
hidden/transparent/clipped/visible elements with current readiness, a readyState
gate, and waiting for the first loadeddata event. The captured Windows WebKit
profiles all miss the unique first frame; the Chromium/Firefox controls retain it.
Seek records distinguish an already-ready element from a delivered loadeddata
event. A zero getVideoPlaybackQuality().totalVideoFrames is also observed in the
correct Firefox control, so it is not a portable frame-readiness gate. The command
reports observations, not a passing acceptance suite, and never edits production
bundles. See refactor's rendering-readiness record before repeating these options.
The frozen eight-second timeline has one unique purple first frame followed by
red, black, blue and yellow sections. Its command and fingerprint are in
`refactor/baselines/auto-thumbnail-timeline-media.json`. Reproduce into a new
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,55 @@
# PKG-AUTO-THUMB-03 渲染与首帧等待对照
状态仍为 doing。这是排除候选原因的检查点,没有修改生产抽帧算法,没有将
AUTO-THUMB-PIXEL-01 或包迁移标为完成。起点为
`f30c701d88b3c5210b7fa6ddb024b55eeeaf8206`。
已有 opacity 探针同时把 video 缩成 1px,不能单独回答完整尺寸隐藏方式的影响。
本次保持实际 320×180 媒体尺寸,分别使用 visibility:hidden、opacity:0、
clip-path:inset(50%)、完全可见元素,保留原始首帧时间 0 和渐进编码流程。
运行输入是正式 ESM 的内存副本,改变 CSS 与等待条件只发生在诊断页面中。
第一轮只把无帧回调分支也改为检查 readyState>=2。事件证据显示 metadata
交付时往往已经是 readyState=4,因此这并不等于真正等到了 loadeddata 事件。
第二轮明确让第一帧等待 loadeddata,并记录每次 seek 前累计交付的事件数;
后续帧沿用正常流程。最终脚本同时保留 current、ready-state、loaded-event
三种策略,避免后续 AI 把不同的等待语义混为一谈。
## 观察与下一步影响
- Windows WebKit 26.6 的 12 种组合均完成五次更新,但没有一组取得时间 0
应有的唯一紫色帧。初次 draw 可能是透明像素,也可能已是后续红色;全可见
元素也失败。因此不能靠“非黑/不透明”判断已经抽到正确首帧。
- 四个 loaded-event 变体第一次 seek 前均已交付一次 loadeddata,仍不准确;
它们实际执行了事件等待,不是仅根据 readyState 推测已等待。
- Chromium 153.0.8010.12 与 Firefox 155.0 的当前代码对照均取得唯一紫色首帧;
后续颜色与正常黑色内容也被保留。它们具有原生帧呈现回调,WebKit 此环境没有。
- Firefox 对照画面正确时 totalVideoFrames 仍为 0;Windows WebKit 也为 0。
不能将该计数 >0 作为跨浏览器 fallback 的完成条件,否则会卡住正常提取。
- 三个 WebKit 组合在请求 1.6 秒后又交付了约 0.002 秒的旧 seeked,触发已有
同目标重试,最终仍只产生五次 draw。记录保留这六次 seek,不能把采样数五
误当作原生事件和重试数也必须为五;这也不解决首帧内容错误。
- 本次没有实现这些未经验证的 CSS/事件/计数修复;也没有删除或放宽现有像素
检查。后续需要从当前 WebKit 媒体路径或实际支持设备取得更直接证据,不能
继续把上述样式/readyState/loadeddata 的简单变体当作尚未尝试的新方案。
具体结果、事件次序、代码与媒体指纹见
[验证记录](../baselines/auto-thumbnail-rendering-readiness.json)。首轮/事件补充
缓存报告也保留指纹,最终十四组来自提交脚本,而不是用文字推测先前探针结果。
这些是一次版本固定的观测,不能外推为所有 WebKit/Safari 版本或物理设备缺陷。
## 重跑与维护
```sh
yarn build artplayer-plugin-auto-thumbnail
yarn probe:auto-thumbnail-rendering
```
第二条只读取产物和固定媒体,启动 loopback 临时端口,依次启动三个测试浏览器,
输出到新的 `.cache/auto-thumbnail-rendering-*`。脚本检查精确可替换代码位置,
记录各变体 hash,结束时检查源产物没有变化。它是诊断采集命令,不加入默认
通过测试列表,也不把成功退出解释为像素已通过。八秒是探针采集上限,不是
新生产超时。使用现有 Playwright 依赖,没有增加或升级依赖、版本或发布产物。
本项只有诊断脚本、入口、说明和证据变更;不声称修复所有旧类型/首帧问题。
回退该检查点会移除诊断入口和记录,不影响 07/08/09 的生产修复。
+1 -1
View File
@@ -517,7 +517,7 @@
- PKG-AUDIO-05: [记录](changes/2026-09-12-PKG-AUDIO-05-combinations.md) [记录](baselines/audio-combinations-first.json) [记录](baselines/audio-combinations-checkpoint.json) [记录](audio-validation.md)
- PKG-AUTO-THUMB-01: [记录](baselines/auto-thumbnail-release.json) [记录](baselines/auto-thumbnail-contract.md) [记录](changes/2026-09-13-PKG-AUTO-THUMB-01-contract.md) [记录](baselines/auto-thumbnail-contract-validation.json)
- PKG-AUTO-THUMB-02: [记录](baselines/auto-thumbnail-failures.md) [记录](baselines/auto-thumbnail-failures-validation.json) [记录](changes/2026-09-13-PKG-AUTO-THUMB-02-failures.md)
- PKG-AUTO-THUMB-03: [记录](changes/2026-09-13-PKG-AUTO-THUMB-03-lifecycle-checkpoint.md) [记录](baselines/auto-thumbnail-lifecycle-checkpoint.json) [记录](changes/2026-09-13-PKG-AUTO-THUMB-03-hidden-renderer.md) [记录](baselines/auto-thumbnail-hidden-renderer.json) [记录](baselines/auto-thumbnail-timeline-media.json) [记录](changes/2026-09-13-PKG-AUTO-THUMB-03-frame-presentation.md) [记录](baselines/auto-thumbnail-frame-presentation.json)
- PKG-AUTO-THUMB-03: [记录](changes/2026-09-13-PKG-AUTO-THUMB-03-lifecycle-checkpoint.md) [记录](baselines/auto-thumbnail-lifecycle-checkpoint.json) [记录](changes/2026-09-13-PKG-AUTO-THUMB-03-hidden-renderer.md) [记录](baselines/auto-thumbnail-hidden-renderer.json) [记录](baselines/auto-thumbnail-timeline-media.json) [记录](changes/2026-09-13-PKG-AUTO-THUMB-03-frame-presentation.md) [记录](baselines/auto-thumbnail-frame-presentation.json) [记录](changes/2026-09-14-PKG-AUTO-THUMB-03-rendering-readiness.md) [记录](baselines/auto-thumbnail-rendering-readiness.json)
- PKG-VTT-THUMB-01: [记录](baselines/vtt-thumbnail-release.json) [记录](baselines/vtt-thumbnail-contract.md) [记录](changes/2026-09-13-PKG-VTT-THUMB-01-contract.md)
- PKG-VTT-THUMB-02: [记录](baselines/vtt-thumbnail-failures.json) [记录](changes/2026-09-13-PKG-VTT-THUMB-02-failures.md)
- PKG-VTT-THUMB-03: [记录](changes/2026-09-13-PKG-VTT-THUMB-03-resources.md) [记录](baselines/vtt-thumbnail-resources.json) [记录](changes/2026-09-13-PKG-VTT-THUMB-03-parser.md) [记录](baselines/vtt-thumbnail-parser.json)
+15
View File
@@ -1,5 +1,20 @@
# 进度与证据
## PKG-AUTO-THUMB-03 渲染/等待诊断检查点(仍 doing)
新增可重跑的 `yarn probe:auto-thumbnail-rendering`,用固定实际 ESM/时间轴媒体
对照真实尺寸下四种可见性样式及 current/readyState/loadeddata 事件三种策略。
最终 Windows WebKit 12 组均未取得唯一紫色首帧;Chromium/Firefox 两个对照正常。
首个 loadeddata 的交付计数被直接记录,避免将 readyState>=2 误称为已等待事件。
正确 Firefox 帧的 totalVideoFrames 仍为 0,不能用该计数 >0 构造通用等待条件。
另外捕获三组旧 seeked 导致的同目标重试,保留全部事件,不把它算作额外采样。
生产源码、dist 和普通像素断言均未改变;没有采用未通过验证的 CSS/等待修复。
见[诊断记录](changes/2026-09-14-PKG-AUTO-THUMB-03-rendering-readiness.md)与
[证据](baselines/auto-thumbnail-rendering-readiness.json)。脚本/lint/工具链及计划
检查通过只说明诊断可运行,不代表首帧通过。PIXEL-01 和 03 保持未完成,计划
仍为 237 项:160 done、15 doing、62 todo。后续不再重复这些已否定的简单策略,
继续定位当前媒体路径和补齐独立的历史消费验证;VAST 默认初始化选择仍待答复。
## PKG-AUTO-THUMB-09 历史 default 调用兼容完成
通过同一函数上的 default 自引用补齐 1.0.1 的 JavaScript 调用,继续支持 1.1.0
+4 -2
View File
@@ -4166,9 +4166,11 @@
"refactor/baselines/auto-thumbnail-hidden-renderer.json",
"refactor/baselines/auto-thumbnail-timeline-media.json",
"test/auto-thumbnail-frames.test.js",
"refactor/baselines/auto-thumbnail-frame-presentation.json"
"refactor/baselines/auto-thumbnail-frame-presentation.json",
"refactor/changes/2026-09-14-PKG-AUTO-THUMB-03-rendering-readiness.md",
"refactor/baselines/auto-thumbnail-rendering-readiness.json"
],
"compatibleResolution": "Establish actual decoded-frame readiness for the existing timeline/grid and preserve silent independent extraction; do not waive colored pixel checks as unsupported media.",
"compatibleResolution": "Establish actual decoded-frame readiness for the existing timeline/grid and preserve silent independent extraction; do not waive colored pixel checks as unsupported media. Intrinsic-size hidden/opacity/clip/visible rendering crossed with current/readyState/first-loadeddata-event strategies still misses the unique first frame in all 12 Windows WebKit profiles. Correct Firefox frames retain totalVideoFrames=0, so that counter cannot be a portable readiness gate. Do not reintroduce these unvalidated workarounds.",
"closureCriteria": "Candidate source and artifacts generate usable sampled pixels on native browser controls with lifecycle checks; supported Safari and core combinations remain explicit final gates.",
"workspaceState": "The frames module now waits for native presentation and seek completion on capable browsers. Chromium/Firefox source/main/legacy assert all five cells including first purple frame, second red cell, legitimate black and presentation times. Callback-less Windows WebKit first two cells remain diagnostic, later cells asserted; physical Safari/core combinations remain open. 84 native lifecycle/pixel cases passed with explicit held native-frame vs Blob-fallback paths; 19 new Node cases fail 17 ways on prior checkpoint and all 52 current cases pass per candidate. Risk and task03 remain open."
},
@@ -0,0 +1,135 @@
import assert from 'node:assert/strict'
import { createHash } from 'node:crypto'
import fs from 'node:fs'
import http from 'node:http'
import process from 'node:process'
import { chromium, firefox, webkit } from '@playwright/test'
const bundle = fs.readFileSync('packages/artplayer-plugin-auto-thumbnail/dist/artplayer-plugin-auto-thumbnail.mjs', 'utf8')
const bytes = fs.readFileSync('test/browser/media/auto-thumbnail-timeline.mp4')
assert.equal(process.version.slice(1), fs.readFileSync('.node-version', 'utf8').trim())
assert.equal(process.env.npm_config_user_agent?.split(' ')[0], 'yarn/1.22.22', 'Use yarn probe:auto-thumbnail-rendering')
const output = fs.mkdtempSync('refactor/.cache/auto-thumbnail-rendering-')
console.log(`Rendering diagnostics: ${output}`)
const fingerprint = data => createHash('sha256').update(data).digest('hex')
assert.equal(fingerprint(bytes), JSON.parse(fs.readFileSync('refactor/baselines/auto-thumbnail-timeline-media.json')).file.sha256, 'The unique-frame diagnostic requires the frozen timeline fixture')
assert(bundle.includes('const ready = !presentedFrames || video.readyState >= 2;'))
assert(bundle.includes('visibility:hidden;'))
assert.equal(bundle.match(/let pending;/g)?.length, 1)
assert.equal(bundle.match(/video.currentTime = target;/g)?.length, 1)
const server = http.createServer((request, response) => {
if (request.url !== '/video.mp4') {
response.writeHead(200, { 'Content-Type': 'text/html' })
response.end('<!doctype html><title>Frame rendering probe</title>')
return
}
const match = /^bytes=(\d+)-(\d*)$/.exec(request.headers.range || '')
const start = match ? Number(match[1]) : 0
const end = match?.[2] ? Math.min(Number(match[2]), bytes.length - 1) : bytes.length - 1
response.writeHead(match ? 206 : 200, { 'Content-Type': 'video/mp4', 'Content-Length': end - start + 1, 'Accept-Ranges': 'bytes', ...(match ? { 'Content-Range': `bytes ${start}-${end}/${bytes.length}` } : {}) })
response.end(bytes.subarray(start, end + 1))
})
await new Promise(resolve => server.listen(0, '127.0.0.1', resolve))
const styles = { hidden: 'visibility:hidden;', opacity: 'visibility:visible;opacity:0;', clip: 'visibility:visible;clip-path:inset(50%);', visible: 'visibility:visible;' }
const results = []
function purple([red, green, blue, alpha]) {
return alpha === 255 && Math.min(red, blue) > 100 && Math.abs(red - blue) < 30 && green < 40
}
try {
for (const [engineName, engine] of Object.entries({ webkit, chromium, firefox })) {
const browser = await engine.launch()
try {
const variants = engineName === 'webkit'
? Object.entries(styles).flatMap(([style, css]) => ['current', 'ready-state', 'loaded-event'].map(readiness => ({ style, css, readiness })))
: [{ style: 'hidden', css: styles.hidden, readiness: 'current' }]
for (const variant of variants) {
const page = await browser.newPage()
await page.goto(`http://127.0.0.1:${server.address().port}`)
let code = bundle.replace('visibility:hidden;', variant.css)
if (variant.readiness === 'ready-state')
code = code.replace('const ready = !presentedFrames || video.readyState >= 2;', 'const ready = video.readyState >= 2;')
if (variant.readiness === 'loaded-event')
code = code.replace('let pending;', 'let pending; let firstFrame = true;').replace('const ready = !presentedFrames || video.readyState >= 2;', 'const ready = !firstFrame; firstFrame = false;')
code = code.replace('video.currentTime = target;', '{ window.__thumbnailSeeks.push({target, loadedData: window.__thumbnailData, readyState: video.readyState}); video.currentTime = target; }')
const result = await page.evaluate(async (code) => {
const draws = []
const events = []
const updates = []
const warnings = []
const videos = []
window.__thumbnailData = 0
window.__thumbnailSeeks = []
const create = document.createElement.bind(document)
const draw = CanvasRenderingContext2D.prototype.drawImage
const originalWarn = console.warn
console.warn = (...args) => warnings.push(args.map(String))
document.createElement = (...args) => {
const element = create(...args)
if (args[0] === 'video') {
videos.push(element)
element.addEventListener('loadeddata', () => window.__thumbnailData++)
for (const name of ['loadedmetadata', 'loadeddata', 'canplay', 'seeked', 'playing']) element.addEventListener(name, () => events.push({ name, time: element.currentTime, readyState: element.readyState, decoded: element.webkitDecodedFrameCount, total: element.getVideoPlaybackQuality?.().totalVideoFrames }))
}
return element
}
CanvasRenderingContext2D.prototype.drawImage = function (source, ...args) {
const result = draw.call(this, source, ...args)
if (source instanceof HTMLVideoElement) {
const [x, y, width, height] = args
draws.push({ time: source.currentTime, rgba: [...this.getImageData(x + Math.floor(width / 2), y + Math.floor(height / 2), 1, 1).data], readyState: source.readyState, decoded: source.webkitDecodedFrameCount, total: source.getVideoPlaybackQuality?.().totalVideoFrames, box: { width: source.getBoundingClientRect().width, height: source.getBoundingClientRect().height } })
}
return result
}
const listeners = new Map()
let finish
const done = new Promise(resolve => finish = resolve)
const art = {
option: { url: '/video.mp4' },
on(name, fn) {
if (!listeners.has(name))
listeners.set(name, new Set())
listeners.get(name).add(fn)
},
off(name, fn) { listeners.get(name)?.delete(fn) },
get thumbnails() { return updates.at(-1) },
set thumbnails(value) {
updates.push(value)
if (updates.length === 5)
finish('completed')
},
}
const emit = (name) => {
for (const fn of [...(listeners.get(name) || [])]) fn()
}
let timer
try {
const { default: factory } = await import(`data:text/javascript,${encodeURIComponent(code)}`)
await factory({ width: 80, number: 5 })(art)
timer = setTimeout(() => finish('timeout'), 8000)
emit('video:loadedmetadata')
const outcome = await done
return { outcome, draws, events, warnings, seeks: window.__thumbnailSeeks, supportsFrames: typeof videos[0]?.requestVideoFrameCallback === 'function' && typeof videos[0]?.cancelVideoFrameCallback === 'function' }
}
finally {
clearTimeout(timer)
emit('destroy')
document.createElement = create
CanvasRenderingContext2D.prototype.drawImage = draw
console.warn = originalWarn
}
}, code)
const firstFrameMatches = result.draws[0] ? purple(result.draws[0].rgba) : false
const entry = { engine: engineName, browser: browser.version(), ...variant, codeSha256: fingerprint(code), firstFrameMatches, ...result }
results.push(entry)
console.log(JSON.stringify({ engine: engineName, ...variant, outcome: result.outcome, firstFrameMatches, seeks: result.seeks, draws: result.draws.map(item => ({ time: item.time, rgba: item.rgba, total: item.total })) }))
await page.close()
}
}
finally { await browser.close() }
}
}
finally {
await new Promise(resolve => server.close(resolve))
assert.equal(fs.readFileSync('packages/artplayer-plugin-auto-thumbnail/dist/artplayer-plugin-auto-thumbnail.mjs', 'utf8'), bundle)
fs.writeFileSync(`${output}/report.json`, `${JSON.stringify({ capturedAt: new Date().toISOString(), environment: { node: process.version, platform: process.platform, arch: process.arch }, sourceSha256: fingerprint(bundle), mediaSha256: fingerprint(bytes), results, scope: 'Diagnostic capture, not acceptance. Four intrinsic-size rendering modes and three first-frame readiness strategies are compared in WebKit on the recorded host. Chromium/Firefox current-code controls retain native presentation callbacks. No production file is changed; missing or incorrect pixels remain explicit.' }, null, 2)}\n`)
}
+3 -1
View File
@@ -1797,7 +1797,9 @@
"baselines/auto-thumbnail-hidden-renderer.json",
"baselines/auto-thumbnail-timeline-media.json",
"changes/2026-09-13-PKG-AUTO-THUMB-03-frame-presentation.md",
"baselines/auto-thumbnail-frame-presentation.json"
"baselines/auto-thumbnail-frame-presentation.json",
"changes/2026-09-14-PKG-AUTO-THUMB-03-rendering-readiness.md",
"baselines/auto-thumbnail-rendering-readiness.json"
]
},
{
+7
View File
@@ -15,6 +15,13 @@ runtime types describe the alias without changing the historical root declaratio
This does not decode video or close all older type-shape, native pixel, or device
acceptance gaps. Candidate entry types use the same runtime files.
`yarn probe:auto-thumbnail-rendering` is an optional diagnosis command, separate
from passing test suites. It runs four intrinsic-size rendering modes and three
readiness strategies in Windows WebKit, plus Chromium/Firefox controls. It records
raw draw pixels, seek/event ordering, frame counters and exact code/media hashes
in a new cache directory. Completion of the command does not imply correct pixels:
the current WebKit first-frame discrepancy remains an open release/refactor risk.
`yarn test:danmuku-mask` runs candidate run cancellation/resource ownership plus
frozen historical defects and public contracts. Candidate tests also run in
`test:unit`; their controlled SDK, RAF and canvas hosts do not prove native model