test(asr): [PKG-ASR-05] verify initial media failure and first-source recovery

This commit is contained in:
Harvey Zhao committed 2026-09-15 14:39:50 +08:00
1 parent 9cf4a58480
commit f37658d657
10 files changed
+10251 -4

No files matched your search

@@ -169,7 +169,7 @@ recognizer service as a prerequisite for local audio verification.
## Shared installed browser validation ## Shared installed browser validation
Eight browser files select verified installed ASR bytes through browser-candidate.js. The Audio Track combination independently verifies both installed plugins. Published ASR controls retain their frozen archives; attachments distinguish selected published, installed and source inputs. Tests cover native local PCM, direct/capture routing, CORS, video-only input recovery, volume/mute and ownership. Deliberately forced Firefox binding rejection remains labeled; no recognition service or physical speaker output is claimed. Nine browser files select verified installed ASR bytes through browser-candidate.js. The Audio Track combination independently verifies both installed plugins. Published ASR controls retain their frozen archives; attachments distinguish selected published, installed and source inputs. Tests cover native local PCM, direct/capture routing, CORS, video-only input recovery, volume/mute and ownership. Deliberately forced Firefox binding rejection remains labeled; no recognition service or physical speaker output is claimed.
Use `yarn test:package --browser`, then `yarn test:browser:installed`. The common Use `yarn test:package --browser`, then `yarn test:browser:installed`. The common
roster in scripts/browser-validation/scope.ts drives both preparation and required roster in scripts/browser-validation/scope.ts drives both preparation and required
@@ -201,3 +201,10 @@ are rejected and normal audio recognition resumes on a valid source. It does not
simulate a network outage or every decoder error. `test/asr-media-error.test.js` simulate a network outage or every decoder error. `test/asr-media-error.test.js`
also covers errors during Worklet loading and delayed capture closure. Both are also covers errors during Worklet loading and delayed capture closure. Both are
required when changing capture cancellation or error listener ownership. required when changing capture cancellation or error listener ownership.
`asr-initial-error.spec.js` separately checks failure before the first metadata:
native play rejects, no Context or recognition is allocated, and stop/destroy are
safe even when the player never became ready. Recovery is a separate test, requiring
actual nonzero PCM from the first valid source and final Context closure. The
failure/cleanup test also runs without WebAudio; only audio recovery is capability
skipped. This keeps initial-failure evidence distinct from successful playback.
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,49 @@
# PKG-ASR-05 首次媒体失败与恢复
来源 HEAD:`9cf4a58480118f92b1ae58e5d8ae6664961d1f70`。接续任务10的播放中错误
修复,验证此前没有覆盖的首次 metadata 之前失败;任务05保持 doing。
## 新增验证
`asr-initial-error.spec.js` 从构造起就使用不可解码的本地 HTTP JSON 内容,等待
原生 MediaError code 4,再通过真实按钮调用 play,确认原生 NotSupportedError
返回给调用方。播放器未 ready、零 Context、零识别回调、没有字幕。
按 default/capture 和核心 5.3.0/5.4.0/候选拆成两个独立用例:
1. 在失败状态下调用公开 stop、destroy,始终不创建音频资源并清理播放器 DOM。
这部分不要求 WebAudio,所以 Chromium/Firefox/Windows WebKit 都实际执行。
2. 同一实例切换到首个有效 AAC 后,真实按钮播放、至少三块非零 PCM、字幕更新,
一个 Context 从 running 到最终 closed。缺少 WebAudio 时仅跳过此恢复用例,
不把已完成的失败路径观察算成音频恢复通过。
没有注入 native error、模拟 AudioContext/PCM,识别回调只把实际 PCM 统计并返回
本地测试文字。没有调用外部识别服务或麦克风。该用例证明首次失败的有限路径,
不声称覆盖每种网络中断、codec、设备或错误时序。
## 安装与维护
本次运行代码、类型、根依赖/锁均未改,复用任务10的已隔离安装 `run-gTeZ8R`。
main/legacy 两份映射经既有验证器检查当前源码、构建输入与 tarball 字节身份,
没有缺包时退回源码。没有重新打包或把这次新增 ARCHITECTURE 文字说成已包含在
旧安装归档;发布候选最终仍需重新打包。源代码一致性与完整包内容新鲜度是两件事。
CI installed 清单新增该文件,ASR 共九个浏览器文件;清单用例保护它不能被遗漏。
包内架构说明和环境矩阵 ASR 行同步,避免初始 BASE-08 的“待音频验证”描述被
当作当前进度。其他行仍按矩阵开头的时间范围说明读取,不在本轮代为宣称完成。
## 结果与限制
Node 24.21.0、Yarn 1.22.22;源码与 installed main/legacy 的实际报告见
[机器证据](../baselines/asr-initial-error-validation.json)。每轮包含18项首次失败
清理和12项实际恢复,另6项 Windows WebKit 恢复因缺WebAudio跳过。浏览器为
Chromium 153.0.8010.12、Firefox 155.0、WebKit 26.6。无重试,没有新生产缺陷。
定向 lint、docs-tools 类型、6项清单回归和严格工具链通过。本次不重复上一任务
已通过的234项生产专项,也不把重复使用安装归档算作一次新打包。原生媒体请求
取消/错误日志保留,不把测试通过写成所有请求都正常完成。
执行 `yarn test:browser asr-initial-error.spec.js --workers=1`;指定经验证的
`ARTPLAYER_BROWSER_ARTIFACTS` 则运行安装模式。源码/安装 scope 的其余规则不变。
没有推送或远端CI执行;设备、外部调用方服务、完整分发和发布复盘仍归既有任务。
回退只撤销新增测试、清单项和文档,不回退任务10的媒体错误修复。
+1 -1
View File
@@ -44,7 +44,7 @@ Node10 是 TS 模块解析模式名,不代表支持 Node.js 10。现代 es2020
| artplayer / 5.4.1 | 原生媒体、字幕、切源、控件/设置、全屏/PiP、SSR;ENV-MEDIA + VTT/SRT | 发布 5.4.0 已冻结;桌面部分基线,移动/原生全屏/PiP/多引擎待测 | CORE-07/16/17/18/22/23、ENG-05 | | artplayer / 5.4.1 | 原生媒体、字幕、切源、控件/设置、全屏/PiP、SSR;ENV-MEDIA + VTT/SRT | 发布 5.4.0 已冻结;桌面部分基线,移动/原生全屏/PiP/多引擎待测 | CORE-07/16/17/18/22/23、ENG-05 |
| artplayer-plugin-ads / npm 1.0.6、工作区 2.1.0 未发布 | 五个 1.0.x 归档及实际核心 4.5.5 归档已冻结;HTML/视频、本片恢复、跳过、错误/销毁;本地短广告和主片 | 4.5.5/5.4.1/候选核心组合、三引擎原生全屏/双实例/UI的main和legacy各27项;本机Chrome152真实标签暂停/视频恢复/后台销毁各6项见ads-validation.md。实际设备/最小化及完整分发仍待,不扩大到全部历史核心 | PKG-ADS-01/02/03/05/06 | | artplayer-plugin-ads / npm 1.0.6、工作区 2.1.0 未发布 | 五个 1.0.x 归档及实际核心 4.5.5 归档已冻结;HTML/视频、本片恢复、跳过、错误/销毁;本地短广告和主片 | 4.5.5/5.4.1/候选核心组合、三引擎原生全屏/双实例/UI的main和legacy各27项;本机Chrome152真实标签暂停/视频恢复/后台销毁各6项见ads-validation.md。实际设备/最小化及完整分发仍待,不扩大到全部历史核心 | PKG-ADS-01/02/03/05/06 |
| artplayer-plugin-ambilight / 1.1.0 | Canvas 采样、CORS、resize、后台/销毁;同源及受控跨源 MP4 | 待真实画布和资源测量 | PKG-AMBILIGHT-01/02/05/06 | | artplayer-plugin-ambilight / 1.1.0 | Canvas 采样、CORS、resize、后台/销毁;同源及受控跨源 MP4 | 待真实画布和资源测量 | PKG-AMBILIGHT-01/02/05/06 |
| artplayer-plugin-asr / 2.1.0 | PCM 音频输出/暂停/销毁;AudioContext + 调用方服务 | 待音频环境与调用方 WebSocket 验证;示例服务版本未知,不在 PR 调线上服务 | PKG-ASR-01/02/05/06、EX-03 | | artplayer-plugin-asr / 2.1.0 | 原生 PCM、direct/capture 归属、暂停/切源/错误/销毁;AudioContext + 调用方回调 | 2026-09-15 补充:5.3.0/5.4.0/候选核心的本地 PCM、音量/静音、跨域、无音轨、原生错误与首次失败恢复已有分段证据,见 [ASR 维护说明](../packages/artplayer-plugin-asr/ARCHITECTURE.md) 和 [错误修复](changes/2026-09-15-PKG-ASR-10-media-error.md)。Windows WebKit 无 WebAudio,不等于 Safari;真机、调用方外部服务和完整分发仍按05/06与EX-03验收,不在PR调用线上服务 | PKG-ASR-01/02/05/06/10、EX-03 |
| artplayer-plugin-audio-track / 1.1.0 | 独立外部 Audio 与主视频时钟同步;本地 MP4+AAC,偏移/切源/缓冲/失败样本 | 1.1.0 发布及调用契约见 [包基线](baselines/audio-track-contract.md);旧核心 5.4.0 为具体对照点,桌面/Safari 实测待完成 | PKG-AUDIO-01/02/05/06 | | artplayer-plugin-audio-track / 1.1.0 | 独立外部 Audio 与主视频时钟同步;本地 MP4+AAC,偏移/切源/缓冲/失败样本 | 1.1.0 发布及调用契约见 [包基线](baselines/audio-track-contract.md);旧核心 5.4.0 为具体对照点,桌面/Safari 实测待完成 | PKG-AUDIO-01/02/05/06 |
| artplayer-plugin-auto-thumbnail / 1.1.0 | 帧提取/跳转、CORS、回收;本地 MP4+跨源失败 | 待 Canvas/seek/移动内存行为 | PKG-AUTO-THUMB-01/02/05/06 | | artplayer-plugin-auto-thumbnail / 1.1.0 | 帧提取/跳转、CORS、回收;本地 MP4+跨源失败 | 待 Canvas/seek/移动内存行为 | PKG-AUTO-THUMB-01/02/05/06 |
| artplayer-plugin-chapter / 1.1.0 | 区间、边界/重叠、hover/update、进度条组合;MP4+固定章节 | 发布 1.1.0 已冻结;最小注册/两段 DOM 已验,边界和新旧组合待 | PKG-CHAPTER-01/02/04/05/06、PILOT-01 | | artplayer-plugin-chapter / 1.1.0 | 区间、边界/重叠、hover/update、进度条组合;MP4+固定章节 | 发布 1.1.0 已冻结;最小注册/两段 DOM 已验,边界和新旧组合待 | PKG-CHAPTER-01/02/04/05/06、PILOT-01 |
+1 -1
View File
@@ -598,7 +598,7 @@
- PKG-ASR-02: [记录](changes/2026-09-13-PKG-ASR-02-audio-baseline.md) [记录](baselines/asr-audio-validation.json) - PKG-ASR-02: [记录](changes/2026-09-13-PKG-ASR-02-audio-baseline.md) [记录](baselines/asr-audio-validation.json)
- PKG-ASR-03: [记录](changes/2026-09-13-PKG-ASR-03-audio-ownership.md) [记录](baselines/asr-ownership-validation.json) - PKG-ASR-03: [记录](changes/2026-09-13-PKG-ASR-03-audio-ownership.md) [记录](baselines/asr-ownership-validation.json)
- PKG-ASR-04: [记录](changes/2026-09-13-PKG-ASR-04-public-types.md) [记录](baselines/asr-types-validation.json) [记录](baselines/asr-type-diagnostics.json) - PKG-ASR-04: [记录](changes/2026-09-13-PKG-ASR-04-public-types.md) [记录](baselines/asr-types-validation.json) [记录](baselines/asr-type-diagnostics.json)
- PKG-ASR-05: [记录](baselines/asr-volume-validation.json) [记录](baselines/asr-fallback-validation.json) [记录](baselines/asr-core.json) [记录](baselines/asr-local-core-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-local-core-checkpoint.md) [记录](baselines/asr-combinations-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-cors-combinations.md) [记录](baselines/asr-explicit-capture-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-editor-generation.md) [记录](changes/2026-09-15-PKG-ASR-05-no-audio.md) [记录](baselines/asr-no-audio-validation.json) - PKG-ASR-05: [记录](baselines/asr-volume-validation.json) [记录](baselines/asr-fallback-validation.json) [记录](baselines/asr-core.json) [记录](baselines/asr-local-core-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-local-core-checkpoint.md) [记录](baselines/asr-combinations-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-cors-combinations.md) [记录](baselines/asr-explicit-capture-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-editor-generation.md) [记录](changes/2026-09-15-PKG-ASR-05-no-audio.md) [记录](baselines/asr-no-audio-validation.json) [记录](changes/2026-09-15-PKG-ASR-05-initial-error.md) [记录](baselines/asr-initial-error-validation.json)
- PKG-ADS-01: [记录](baselines/ads-release.json) [记录](baselines/ads-contract.md) [记录](baselines/ads-contract-validation.json) [记录](changes/2026-09-12-PKG-ADS-01-contract.md) [记录](scripts/ads-contract.test.mjs) - PKG-ADS-01: [记录](baselines/ads-release.json) [记录](baselines/ads-contract.md) [记录](baselines/ads-contract-validation.json) [记录](changes/2026-09-12-PKG-ADS-01-contract.md) [记录](scripts/ads-contract.test.mjs)
- PKG-ADS-02: [记录](changes/2026-09-12-PKG-ADS-02-tests.md) [记录](ads-validation.md) [记录](baselines/ads-validation.json) - PKG-ADS-02: [记录](changes/2026-09-12-PKG-ADS-02-tests.md) [记录](ads-validation.md) [记录](baselines/ads-validation.json)
- PKG-ADS-03: [记录](changes/2026-09-12-PKG-ADS-03-lifecycle.md) [记录](ads-validation.md) [记录](baselines/ads-lifecycle-validation.json) - PKG-ADS-03: [记录](changes/2026-09-12-PKG-ADS-03-lifecycle.md) [记录](ads-validation.md) [记录](baselines/ads-lifecycle-validation.json)
+13
View File
@@ -1,5 +1,18 @@
# 进度与证据 # 进度与证据
## PKG-ASR-05 首次媒体失败与恢复
首次metadata之前的原生媒体失败、play拒绝、零音频资源、stop/destroy以及首个
合法来源恢复已验证。源码/复用的已安装main和legacy三轮各30通过、6恢复能力
跳过,共90通过(54失败清理、36真实PCM恢复),18 Windows WebKit恢复跳过。
失败清理用例在WebKit实际执行,不把部分检查或缺WebAudio算成恢复成功。
见[检查点](changes/2026-09-15-PKG-ASR-05-initial-error.md)及
[证据](baselines/asr-initial-error-validation.json)。6清单回归、类型/lint/工具链通过;
新增CI清单、包内说明和ASR环境行。运行字节/依赖/锁未改,没有重打包,旧归档
不包含本次新增文档。203 done、21 doing、43 todo不变;任务05的设备、外部服务
及完整分发要求继续开放。后续转向其他包的剩余验收。
## PKG-ASR-10 修复原生媒体错误后的过期识别 ## PKG-ASR-10 修复原生媒体错误后的过期识别
复现真实 `art.video.src` 不可解码输入产生 error 而没有 pause:旧识别结果仍显示、 复现真实 `art.video.src` 不可解码输入产生 error 而没有 pause:旧识别结果仍显示、
+3 -1
View File
@@ -2924,7 +2924,9 @@
"baselines/asr-explicit-capture-validation.json", "baselines/asr-explicit-capture-validation.json",
"changes/2026-09-13-PKG-ASR-05-editor-generation.md", "changes/2026-09-13-PKG-ASR-05-editor-generation.md",
"changes/2026-09-15-PKG-ASR-05-no-audio.md", "changes/2026-09-15-PKG-ASR-05-no-audio.md",
"baselines/asr-no-audio-validation.json" "baselines/asr-no-audio-validation.json",
"changes/2026-09-15-PKG-ASR-05-initial-error.md",
"baselines/asr-initial-error-validation.json"
] ]
}, },
{ {
+1
View File
@@ -43,6 +43,7 @@ export const installedTests = [
'asr-capture-cors.spec.js', 'asr-capture-cors.spec.js',
'asr-no-audio.spec.js', 'asr-no-audio.spec.js',
'asr-media-error.spec.js', 'asr-media-error.spec.js',
'asr-initial-error.spec.js',
'asr-audio-track.spec.js', 'asr-audio-track.spec.js',
'chromecast-lifecycle.spec.js', 'chromecast-lifecycle.spec.js',
'mediabunny-audio.spec.js', 'mediabunny-audio.spec.js',
+1
View File
@@ -22,6 +22,7 @@ test('Source invocation clears inherited artifact selection without changing cal
assert(installedTests.includes('canvas-lifecycle.spec.js')) assert(installedTests.includes('canvas-lifecycle.spec.js'))
assert(installedTests.includes('asr-no-audio.spec.js')) assert(installedTests.includes('asr-no-audio.spec.js'))
assert(installedTests.includes('asr-media-error.spec.js')) assert(installedTests.includes('asr-media-error.spec.js'))
assert(installedTests.includes('asr-initial-error.spec.js'))
for (const file of installedTests) for (const file of installedTests)
assert(fs.existsSync(path.resolve('test/browser', file))) assert(fs.existsSync(path.resolve('test/browser', file)))
}) })
+107
View File
@@ -0,0 +1,107 @@
import process from 'node:process'
import { browserCandidate } from '../helpers/browser-candidate.js'
import { expect, test } from './fixtures.js'
let candidate
test.beforeAll(async () => {
candidate = await browserCandidate('artplayer-plugin-asr', process.env.ARTPLAYER_ASR_ARTIFACT)
})
async function createFailedPlayer(page, core, mode, testInfo) {
await page.goto(`/test/player.html?core=${core}`)
const capabilities = await page.evaluate(() => ({ context: typeof (window.AudioContext || window.webkitAudioContext), worklet: typeof window.AudioWorkletNode }))
await testInfo.attach('asr-initial-error-inputs', { contentType: 'application/json', body: JSON.stringify({ core, mode, capabilities, provenance: candidate.provenance, errorSource: '/test/manifest.json', initialReady: false, forcedFailure: false }) })
await page.addScriptTag({ content: candidate.code })
await page.evaluate((mode) => {
const Context = window.AudioContext || window.webkitAudioContext
const contexts = []
const chunks = []
if (Context) {
window.AudioContext = class extends Context {
constructor(options) {
super(options)
contexts.push(this)
}
}
}
window.art = new window.Artplayer({
container: '.player',
url: '/test/manifest.json?asr-initial-error=unsupported',
plugins: [window.artplayerPluginAsr({
...(mode === 'capture' ? { audioInput: { type: 'capture' } } : {}),
onAudioChunk({ pcm }) {
let peak = 0
for (const sample of new Int16Array(pcm))
peak = Math.max(peak, Math.abs(sample))
chunks.push({ peak, source: window.art.video.currentSrc })
return 'Recovered audio'
},
})],
})
window.initialErrorState = () => ({ ready: window.art.isReady, destroyed: window.art.isDestroy, error: window.art.video.error?.code || null, paused: window.art.video.paused, contexts: contexts.map(context => context.state), chunks, playResult: window.playResult, subtitle: document.querySelector('.art-layer-asr')?.textContent || '' })
document.querySelector('#play').onclick = async () => {
window.playResult = { outcome: 'pending' }
try {
await window.art.play()
window.playResult = { outcome: 'resolved' }
}
catch (error) {
window.playResult = { outcome: 'rejected', name: error.name, message: error.message }
}
}
}, mode)
await expect.poll(() => page.evaluate(() => window.initialErrorState().error)).toBe(4)
await page.locator('#play').click()
await expect.poll(() => page.evaluate(() => window.initialErrorState().playResult?.outcome)).toBe('rejected')
const failed = await page.evaluate(() => window.initialErrorState())
expect(failed.playResult.name).toBe('NotSupportedError')
expect(failed.ready).toBe(false)
expect(failed.contexts).toEqual([])
expect(failed.chunks).toEqual([])
expect(failed.subtitle).toBe('')
await testInfo.attach('asr-initial-media-failure', { contentType: 'application/json', body: JSON.stringify(failed) })
return capabilities
}
test.afterEach(async ({ page }, testInfo) => {
await page.evaluate(() => {
if (window.art && !window.art.isDestroy)
window.art.destroy()
})
const state = await page.evaluate(() => window.initialErrorState?.()).catch(error => ({ error: error.message }))
await testInfo.attach('asr-initial-error-final', { contentType: 'application/json', body: JSON.stringify(state ?? null) })
})
for (const core of ['published-5.3.0', 'published', 'candidate']) {
for (const mode of ['default', 'capture']) {
test(`ASR ${mode} + ${core}: initial unsupported media remains lazy through stop and destroy`, async ({ page }, testInfo) => {
await createFailedPlayer(page, core, mode, testInfo)
await page.evaluate(() => window.art.plugins.artplayerPluginAsr.stop())
expect(await page.evaluate(() => window.initialErrorState().contexts)).toEqual([])
await page.evaluate(() => window.art.destroy())
expect(await page.evaluate(() => window.initialErrorState())).toMatchObject({ destroyed: true, contexts: [], chunks: [] })
await expect(page.locator('.art-video-player')).toHaveCount(0)
})
test(`ASR ${mode} + ${core}: initial unsupported media recovers on the first valid source`, async ({ page }, testInfo) => {
const capabilities = await createFailedPlayer(page, core, mode, testInfo)
if (capabilities.context === 'undefined') {
expect(process.platform).toBe('win32')
expect(testInfo.project.name).toBe('webkit')
expect(capabilities.worklet).toBe('undefined')
test.skip(true, 'Initial failure checked; WebKit lacks WebAudio for recovery, not Safari acceptance')
}
await page.evaluate(() => {
window.art.url = '/test/audio-tone.m4a?asr-initial-error=recovery'
})
await expect.poll(() => page.evaluate(() => window.art.isReady && window.art.video.readyState >= 2)).toBe(true)
await page.locator('#play').click()
await expect.poll(() => page.evaluate(() => window.initialErrorState().chunks.filter(chunk => chunk.peak > 0 && chunk.source.includes('asr-initial-error=recovery')).length)).toBeGreaterThanOrEqual(3)
expect(await page.evaluate(() => window.initialErrorState().contexts)).toEqual(['running'])
await expect(page.locator('.art-layer-asr')).toHaveText('Recovered audio')
await testInfo.attach('asr-first-valid-source', { contentType: 'application/json', body: JSON.stringify(await page.evaluate(() => window.initialErrorState())) })
await page.evaluate(() => window.art.destroy())
await expect.poll(() => page.evaluate(() => window.initialErrorState().contexts)).toEqual(['closed'])
})
}
}