mirror of
https://github.com/zhw2590582/ArtPlayer.git
synced 2026-10-08 10:56:15 -08:00
test(asr): [PKG-ASR-05] verify video-only input and installed capture recovery
This commit is contained in:
1 parent
65368809ee
commit
3c89b48514
9 files changed
+12560
-4
No files matched your search
@@ -154,7 +154,7 @@ From the repository root:
|
||||
Windows Playwright WebKit 26.6 lacks AudioContext and AudioWorkletNode. Its native
|
||||
ASR cases remain explicit skips and do not establish Safari/device compatibility.
|
||||
Native direct/fallback routing, volume/mute, CORS and audio-track combinations
|
||||
have scoped browser evidence. Unsupported media, external-owner integration,
|
||||
have scoped browser evidence. Other unsupported media,
|
||||
Safari and physical devices remain PKG-ASR-05 acceptance work. CORS restrictions
|
||||
and main-video-only capture are explicit integration boundaries, not a claim
|
||||
that every media source can be recognized. Do not use the demo's external
|
||||
@@ -162,10 +162,27 @@ recognizer service as a prerequisite for local audio verification.
|
||||
|
||||
## Shared installed browser validation
|
||||
|
||||
Six browser files select verified installed ASR bytes through browser-candidate.js. The Audio Track combination independently verifies both installed plugins. Published ASR controls retain their frozen archives; attachments distinguish selected published, installed and source inputs. Tests cover native local PCM, direct/capture routing, CORS, volume/mute and ownership. Deliberately forced Firefox binding rejection remains labeled; no recognition service or physical speaker output is claimed.
|
||||
Seven browser files select verified installed ASR bytes through browser-candidate.js. The Audio Track combination independently verifies both installed plugins. Published ASR controls retain their frozen archives; attachments distinguish selected published, installed and source inputs. Tests cover native local PCM, direct/capture routing, CORS, video-only input recovery, volume/mute and ownership. Deliberately forced Firefox binding rejection remains labeled; no recognition service or physical speaker output is claimed.
|
||||
|
||||
Use `yarn test:package --browser`, then `yarn test:browser:installed`. The common
|
||||
roster in scripts/browser-validation/scope.ts drives both preparation and required
|
||||
inputs. Explicit artifact overrides are rejected when an installed map is present;
|
||||
missing or stale inputs fail without source fallback. Source mode retains deliberate
|
||||
artifact selection. See ../../refactor/changes/2026-09-15-CI-01-asr-cast-installed.md.
|
||||
|
||||
## Video-only input
|
||||
|
||||
`test/browser/asr-no-audio.spec.js` uses the generated H.264 video with no audio
|
||||
track, verifies changing decoded pixels and the media clock, then changes the same
|
||||
player to the local AAC tone. The default direct graph can yield silence or no
|
||||
chunks for the video-only source and remains available for the later audio source.
|
||||
Capture-only mode naturally fails `createMediaStreamSource` with no audio track;
|
||||
the failed graph closes its context and stops its captured video track without
|
||||
taking over or pausing native playback. A later playable source reacquires capture.
|
||||
Both paths verify nonzero recovered PCM and final context/track cleanup.
|
||||
|
||||
The test observes real browser nodes/streams and does not inject a binding failure.
|
||||
It does not prove speaker output, malformed-media recovery, Safari or device
|
||||
support. Run `yarn test:browser asr-no-audio.spec.js --workers=1` for source mode;
|
||||
the same file is included in the installed CI roster. WebAudio capability skips
|
||||
remain visible and cannot be counted as audio processing passes.
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,64 @@
|
||||
# PKG-ASR-05 无音轨视频与恢复检查点
|
||||
|
||||
来源 HEAD:`65368809ee70803e4dae7cb145a17310a386b4e4`。任务继续 doing,
|
||||
本次补齐明确列出的无音轨场景,不关闭 ASR 的设备、其他不支持媒体或发布要求。
|
||||
生产源码、类型、依赖、根锁和工作区分发字节均未修改。
|
||||
|
||||
## 验证边界
|
||||
|
||||
新增 `test/browser/asr-no-audio.spec.js`,使用同一旧核心/候选核心宿主,分别调用
|
||||
默认 direct 和显式 `audioInput: { type: 'capture' }`。原生 AudioContext 方法仅
|
||||
记录输入及返回资源,仍调用浏览器实现;不注入异常或伪造 stream/PCM。
|
||||
|
||||
`ffprobe` 确认既有 pattern.mp4 只有一条 H.264 视频流,无音轨;后续 audio-tone.m4a
|
||||
只有一条 48 kHz 单声道 AAC。两份文件未重新生成,完整字节摘要及 probe 输出
|
||||
保存在[机器证据](../baselines/asr-no-audio-validation.json)。输入变化需要重新验收。
|
||||
|
||||
每个可处理音频的场景均验证:
|
||||
|
||||
- 无音轨视频的媒体时钟和 Canvas 读取的解码画面持续变化,视频未被暂停或报错,
|
||||
音量/静音不变,ASR 不产生非零 PCM。
|
||||
- 默认路径保留原生 element source 和 Context。Chromium 输出全零块;Firefox
|
||||
此阶段没有回调。二者都是记录到的实际行为,没有强制统一为某种 callback 数量。
|
||||
- 显式捕获得到仅含一条视频轨的原生 stream,原生 stream source 初始化失败;
|
||||
Context 关闭、捕获轨停止,零直接绑定,原视频继续播放。初始化错误保留在证据中。
|
||||
- 同一播放器切换到 AAC 后出现至少三块非零 PCM。默认路径复用一个 Context;
|
||||
捕获模式新建 Context,旧 Context 已关闭。销毁后全部 Context/捕获轨关闭,DOM 清理。
|
||||
|
||||
这不是物理扬声器输出、跨域授权绕过或完整设备能力证明。此前显式跨域失败与同源
|
||||
恢复已有 ASR-09 证据;本次没有重跑它来制造新进度,ASR-CORS-01 的默认限制仍保留。
|
||||
|
||||
## 产物与 CI
|
||||
|
||||
`yarn test:package --include=artplayer-plugin-asr` 在 `run-NH0n0v` 快照重建、打包、
|
||||
仓库外安装并冻结复装 core/chapter/ASR,检查文件摘要。通用 36 项 runtime、
|
||||
5 组旧类型和 8 组精确类型仍只覆盖 core/chapter,不冒充 ASR 全包消费者验收。
|
||||
|
||||
新增测试通过 `browserCandidate` 核验安装文件与源码/构建输入。main 和 legacy
|
||||
各用同一安装目录内对应真实文件,legacy 映射也逐项符合 tarball 摘要。没有用
|
||||
源码回退代替安装产物。公开版本尚未升级为下一 major;这只是内部验收候选。
|
||||
|
||||
`scripts/browser-validation/scope.ts` 纳入该文件,ASR 安装清单从六个变为七个。
|
||||
已有清单测试保护该用例不能遗漏;没有修改 CI 并发、超时、重试或其他必需命令。
|
||||
远端 CI 尚未在本轮执行,局部通过不代表完整二十包矩阵通过。
|
||||
|
||||
## 结果与维护
|
||||
|
||||
Node 24.21.0、Yarn 1.22.22。源码、installed main、installed legacy 各 8 通过、
|
||||
4 WebAudio 能力跳过,合计 24 通过、12 跳过、0 失败、0 重试。实际处理来自
|
||||
Chromium 153.0.8010.12 和 Firefox 155.0;Windows WebKit 26.6 缺 AudioContext/
|
||||
AudioWorklet,跳过在报告中明确保留,不视为 Safari 验收。
|
||||
每轮另记录八条 Chromium 媒体 `ERR_ABORTED`,原始诊断保留;用例通过不表示
|
||||
全部网络请求都正常结束。
|
||||
|
||||
ASR 229 项专项及浏览器清单 6 项通过,docs-tools 类型、定向只读 lint 和严格
|
||||
工具链检查通过。初始 lint 的多变量同一行已修正;运行行为和断言未改变。
|
||||
|
||||
复跑:`yarn test:browser asr-no-audio.spec.js --workers=1`。使用安装内容时先运行
|
||||
上述 package 命令,再把 `ARTPLAYER_BROWSER_ARTIFACTS` 指向其 `browser-artifacts.json`。
|
||||
全量安装 CI 仍按 `yarn test:package --browser` 与 `yarn test:browser:installed` 执行。
|
||||
包内 [ARCHITECTURE](../../packages/artplayer-plugin-asr/ARCHITECTURE.md) 同批更新。
|
||||
|
||||
本批没有发现新的生产缺陷。回退只撤销新增测试、CI 清单项与文档/证据,不恢复任何
|
||||
旧运行时错误。后续继续其他不支持媒体、实际设备和完整分发验收;202/266 完成数
|
||||
不变。没有推送、部署或发布。
|
||||
+1
-1
@@ -597,7 +597,7 @@
|
||||
- PKG-ASR-02: [记录](changes/2026-09-13-PKG-ASR-02-audio-baseline.md) [记录](baselines/asr-audio-validation.json)
|
||||
- PKG-ASR-03: [记录](changes/2026-09-13-PKG-ASR-03-audio-ownership.md) [记录](baselines/asr-ownership-validation.json)
|
||||
- PKG-ASR-04: [记录](changes/2026-09-13-PKG-ASR-04-public-types.md) [记录](baselines/asr-types-validation.json) [记录](baselines/asr-type-diagnostics.json)
|
||||
- PKG-ASR-05: [记录](baselines/asr-volume-validation.json) [记录](baselines/asr-fallback-validation.json) [记录](baselines/asr-core.json) [记录](baselines/asr-local-core-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-local-core-checkpoint.md) [记录](baselines/asr-combinations-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-cors-combinations.md) [记录](baselines/asr-explicit-capture-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-editor-generation.md)
|
||||
- PKG-ASR-05: [记录](baselines/asr-volume-validation.json) [记录](baselines/asr-fallback-validation.json) [记录](baselines/asr-core.json) [记录](baselines/asr-local-core-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-local-core-checkpoint.md) [记录](baselines/asr-combinations-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-cors-combinations.md) [记录](baselines/asr-explicit-capture-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-editor-generation.md) [记录](changes/2026-09-15-PKG-ASR-05-no-audio.md) [记录](baselines/asr-no-audio-validation.json)
|
||||
- PKG-ADS-01: [记录](baselines/ads-release.json) [记录](baselines/ads-contract.md) [记录](baselines/ads-contract-validation.json) [记录](changes/2026-09-12-PKG-ADS-01-contract.md) [记录](scripts/ads-contract.test.mjs)
|
||||
- PKG-ADS-02: [记录](changes/2026-09-12-PKG-ADS-02-tests.md) [记录](ads-validation.md) [记录](baselines/ads-validation.json)
|
||||
- PKG-ADS-03: [记录](changes/2026-09-12-PKG-ADS-03-lifecycle.md) [记录](ads-validation.md) [记录](baselines/ads-lifecycle-validation.json)
|
||||
|
||||
@@ -1,5 +1,18 @@
|
||||
# 进度与证据
|
||||
|
||||
## PKG-ASR-05 无音轨视频与恢复
|
||||
|
||||
新增真实无音轨 H.264 → AAC 的默认/显式捕获测试:播放时钟与解码画面继续,
|
||||
无非零 PCM;显式捕获失败后关闭 Context/捕获轨且不直接绑定媒体;切源后恢复
|
||||
非零采集,销毁清理通过。源码、隔离安装 main/legacy 合计 24 通过、12 Windows
|
||||
WebKit WebAudio 能力跳过,零失败/重试。229 ASR 专项、6 清单测试、类型/lint/
|
||||
严格工具链通过。新增文件已纳入 installed CI 清单,远端未执行。
|
||||
|
||||
见[检查点](changes/2026-09-15-PKG-ASR-05-no-audio.md)和
|
||||
[证据](baselines/asr-no-audio-validation.json)。生产运行字节/类型/依赖/锁未改;
|
||||
任务05仍 doing,不关闭其他不支持媒体、真实设备和发布门槛。202 done、21 doing、
|
||||
43 todo;继续剩余实际验收,不重复已有跨域恢复证据。
|
||||
|
||||
## SITE-07 Monaco 历史 Unicode 数据检查点
|
||||
|
||||
三份原生成配方在冻结依赖下复现 RTL/Emoji 正则、imprecise token 表达式和全部
|
||||
|
||||
+3
-1
@@ -2921,7 +2921,9 @@
|
||||
"baselines/asr-combinations-validation.json",
|
||||
"changes/2026-09-13-PKG-ASR-05-cors-combinations.md",
|
||||
"baselines/asr-explicit-capture-validation.json",
|
||||
"changes/2026-09-13-PKG-ASR-05-editor-generation.md"
|
||||
"changes/2026-09-13-PKG-ASR-05-editor-generation.md",
|
||||
"changes/2026-09-15-PKG-ASR-05-no-audio.md",
|
||||
"baselines/asr-no-audio-validation.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -41,6 +41,7 @@ export const installedTests = [
|
||||
'asr-fallback.spec.js',
|
||||
'asr-cors.spec.js',
|
||||
'asr-capture-cors.spec.js',
|
||||
'asr-no-audio.spec.js',
|
||||
'asr-audio-track.spec.js',
|
||||
'chromecast-lifecycle.spec.js',
|
||||
'mediabunny-audio.spec.js',
|
||||
|
||||
@@ -20,6 +20,7 @@ test('Source invocation clears inherited artifact selection without changing cal
|
||||
assert.notEqual(source.directory, installed.directory)
|
||||
assert(installedTests.includes('ambilight-proxy.spec.js'))
|
||||
assert(installedTests.includes('canvas-lifecycle.spec.js'))
|
||||
assert(installedTests.includes('asr-no-audio.spec.js'))
|
||||
for (const file of installedTests)
|
||||
assert(fs.existsSync(path.resolve('test/browser', file)))
|
||||
})
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
import fs from 'node:fs'
|
||||
import process from 'node:process'
|
||||
import { hash } from '../../refactor/scripts/releases.mjs'
|
||||
import { browserCandidate } from '../helpers/browser-candidate.js'
|
||||
import { expect, test } from './fixtures.js'
|
||||
|
||||
let candidate
|
||||
test.beforeAll(async () => {
|
||||
candidate = await browserCandidate('artplayer-plugin-asr', process.env.ARTPLAYER_ASR_ARTIFACT)
|
||||
})
|
||||
|
||||
test.afterEach(async ({ page }, testInfo) => {
|
||||
const observation = await page.evaluate(() => window.noAudioState?.()).catch(error => ({ error: error.message }))
|
||||
await testInfo.attach('asr-no-audio-final', { contentType: 'application/json', body: JSON.stringify(observation ?? null) })
|
||||
await page.evaluate(() => {
|
||||
if (window.art && !window.art.isDestroy)
|
||||
window.art.destroy()
|
||||
})
|
||||
})
|
||||
|
||||
for (const core of ['published', 'candidate']) {
|
||||
for (const mode of ['default', 'capture']) {
|
||||
test(`ASR ${mode} + ${core}: video-only input keeps playback and recovers audio capture after switching`, async ({ page }, testInfo) => {
|
||||
const errors = []
|
||||
page.on('console', (message) => {
|
||||
if (message.type() === 'error')
|
||||
errors.push(message.text())
|
||||
})
|
||||
await page.goto(`/test/player.html?core=${core}`)
|
||||
const capabilities = await page.evaluate(() => ({ context: typeof (window.AudioContext || window.webkitAudioContext), worklet: typeof window.AudioWorkletNode, capture: typeof HTMLMediaElement.prototype.captureStream, mozCapture: typeof HTMLMediaElement.prototype.mozCaptureStream }))
|
||||
await testInfo.attach('asr-no-audio-inputs', { contentType: 'application/json', body: JSON.stringify({ core, mode, capabilities, provenance: candidate.provenance, pluginSha256: hash(candidate.code), videoSha256: hash(fs.readFileSync(new URL('./media/pattern.mp4', import.meta.url))), audioSha256: hash(fs.readFileSync(new URL('./media/audio-tone.m4a', import.meta.url))), forcedException: false, physicalAudioOutput: false }) })
|
||||
if (capabilities.context === 'undefined') {
|
||||
expect(process.platform).toBe('win32')
|
||||
expect(testInfo.project.name).toBe('webkit')
|
||||
expect(capabilities.worklet).toBe('undefined')
|
||||
test.skip(true, 'Windows WebKit has no WebAudio; not an Apple-device ASR acceptance')
|
||||
}
|
||||
expect(capabilities.worklet).toBe('function')
|
||||
await page.addScriptTag({ content: candidate.code })
|
||||
await page.evaluate((mode) => {
|
||||
const Context = window.AudioContext || window.webkitAudioContext
|
||||
const contexts = []
|
||||
const tracks = []
|
||||
const bindings = []
|
||||
const chunks = []
|
||||
window.AudioContext = class extends Context {
|
||||
constructor(options) {
|
||||
super(options)
|
||||
contexts.push(this)
|
||||
}
|
||||
|
||||
createMediaElementSource(element) {
|
||||
bindings.push({ kind: 'direct', source: element.currentSrc })
|
||||
return super.createMediaElementSource(element)
|
||||
}
|
||||
|
||||
createMediaStreamSource(stream) {
|
||||
bindings.push({ kind: 'stream', audio: stream.getAudioTracks().length, video: stream.getVideoTracks().length })
|
||||
tracks.push(...stream.getTracks())
|
||||
return super.createMediaStreamSource(stream)
|
||||
}
|
||||
}
|
||||
window.art = new window.Artplayer({
|
||||
container: '.player',
|
||||
url: '/test/pattern.mp4',
|
||||
volume: 0.6,
|
||||
loop: true,
|
||||
plugins: [window.artplayerPluginAsr({
|
||||
...(mode === 'capture' ? { audioInput: { type: 'capture' } } : {}),
|
||||
onAudioChunk({ pcm }) {
|
||||
let peak = 0
|
||||
for (const value of new Int16Array(pcm))
|
||||
peak = Math.max(peak, Math.abs(value))
|
||||
chunks.push({ source: window.art.video.currentSrc, peak, bytes: pcm.byteLength })
|
||||
},
|
||||
})],
|
||||
})
|
||||
document.querySelector('#play').onclick = () => window.art.play()
|
||||
window.noAudioState = () => ({
|
||||
bindings,
|
||||
contexts: contexts.map(context => context.state),
|
||||
tracks: tracks.map(track => ({ kind: track.kind, state: track.readyState })),
|
||||
chunks,
|
||||
media: { time: window.art.currentTime, source: window.art.video.currentSrc, paused: window.art.video.paused, error: window.art.video.error?.code || null, volume: window.art.volume, muted: window.art.muted },
|
||||
})
|
||||
window.noAudioPixels = () => {
|
||||
const canvas = document.createElement('canvas')
|
||||
canvas.width = 32
|
||||
canvas.height = 18
|
||||
const context = canvas.getContext('2d')
|
||||
context.drawImage(window.art.video, 0, 0, 32, 18)
|
||||
return [...context.getImageData(0, 0, 32, 18).data]
|
||||
}
|
||||
}, mode)
|
||||
await expect.poll(() => page.evaluate(() => window.art.isReady)).toBe(true)
|
||||
await page.locator('#play').click()
|
||||
await expect.poll(() => page.evaluate(() => window.art.currentTime)).toBeGreaterThan(0.5)
|
||||
const first = await page.evaluate(() => ({ pixels: window.noAudioPixels(), time: window.art.currentTime }))
|
||||
await expect.poll(() => page.evaluate(() => window.art.currentTime)).toBeGreaterThan(first.time + 0.5)
|
||||
expect(await page.evaluate(() => window.noAudioPixels())).not.toEqual(first.pixels)
|
||||
const silent = await page.evaluate(() => window.noAudioState())
|
||||
expect(silent.media).toMatchObject({ paused: false, error: null, volume: 0.6, muted: false })
|
||||
expect(silent.chunks.every(chunk => chunk.peak === 0)).toBe(true)
|
||||
if (mode === 'capture') {
|
||||
expect(silent.bindings).toEqual([{ kind: 'stream', audio: 0, video: 1 }])
|
||||
expect(silent.contexts).toEqual(['closed'])
|
||||
expect(silent.tracks).toEqual([{ kind: 'video', state: 'ended' }])
|
||||
expect(silent.chunks).toEqual([])
|
||||
expect(errors).toHaveLength(1)
|
||||
expect(errors[0]).toContain('[artplayerPluginAsr] Initialization failed:')
|
||||
}
|
||||
else {
|
||||
expect(silent.bindings).toHaveLength(1)
|
||||
expect(silent.bindings[0].kind).toBe('direct')
|
||||
expect(silent.contexts).toEqual(['running'])
|
||||
expect(silent.tracks).toEqual([])
|
||||
expect(errors).toEqual([])
|
||||
}
|
||||
await testInfo.attach('asr-no-audio-playing', { contentType: 'application/json', body: JSON.stringify({ silent, errors: [...errors], changingDecodedPixels: true }) })
|
||||
await page.evaluate(() => {
|
||||
window.art.url = '/test/audio-tone.m4a?asr-no-audio=recovery'
|
||||
})
|
||||
await expect.poll(() => page.evaluate(() => window.art.video.currentSrc.includes('asr-no-audio=recovery') && window.art.video.readyState >= 2)).toBe(true)
|
||||
await page.locator('#play').click()
|
||||
await expect.poll(() => page.evaluate(() => window.noAudioState().chunks.filter(chunk => chunk.source.includes('asr-no-audio=recovery') && chunk.peak > 0).length)).toBeGreaterThanOrEqual(3)
|
||||
const recovered = await page.evaluate(() => window.noAudioState())
|
||||
expect(recovered.contexts).toEqual(mode === 'capture' ? ['closed', 'running'] : ['running'])
|
||||
expect(recovered.bindings.filter(binding => binding.kind === 'direct')).toHaveLength(mode === 'capture' ? 0 : 1)
|
||||
expect(recovered.media).toMatchObject({ paused: false, error: null, volume: 0.6, muted: false })
|
||||
await testInfo.attach('asr-audio-recovered', { contentType: 'application/json', body: JSON.stringify(recovered) })
|
||||
await page.evaluate(() => window.art.destroy())
|
||||
await expect.poll(() => page.evaluate(() => window.noAudioState().contexts.every(state => state === 'closed'))).toBe(true)
|
||||
expect(await page.evaluate(() => window.noAudioState().tracks.every(track => track.state === 'ended'))).toBe(true)
|
||||
await expect(page.locator('.art-video-player')).toHaveCount(0)
|
||||
})
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user