mirror of
https://github.com/zhw2590582/ArtPlayer.git
synced 2026-10-29 11:11:20 -08:00
test(asr): [PKG-ASR-05] cover CORS and independent audio combinations
This commit is contained in:
1 parent
2cede66933
commit
5ab7da6e1f
12 files changed
+6360
-4
No files matched your search
@@ -112,6 +112,15 @@ From the repository root:
|
|||||||
- `yarn test:browser test/browser/asr-fallback.spec.js`: native capture/Worklet
|
- `yarn test:browser test/browser/asr-fallback.spec.js`: native capture/Worklet
|
||||||
audio with an external media owner, silent ASR output even while muted, stop,
|
audio with an external media owner, silent ASR output even while muted, stop,
|
||||||
fresh tracks after restart/source change and independent context cleanup.
|
fresh tracks after restart/source change and independent context cleanup.
|
||||||
|
- `yarn test:browser test/browser/asr-cors.spec.js`: CORS-authorized media,
|
||||||
|
opaque direct/redirected media, native zero-output or absent-chunk observations,
|
||||||
|
and restoration after switching to a same-origin source. Default routing keeps
|
||||||
|
the [Web Audio CORS restriction](https://www.w3.org/TR/webaudio-1.0/#MediaElementAudioSourceNode-security);
|
||||||
|
this test does not claim that unsupported cross-origin playback is repaired.
|
||||||
|
- `yarn test:browser test/browser/asr-audio-track.spec.js`: separate audio-track
|
||||||
|
ownership, pause/update/stop/destroy and main-video-only PCM. Both plugin orders
|
||||||
|
are exercised in distinct cases. `ARTPLAYER_AUDIO_ARTIFACT` selects a real
|
||||||
|
audio-track distribution alongside `ARTPLAYER_ASR_ARTIFACT`.
|
||||||
- `yarn typecheck`: includes the package strict/noUncheckedIndexedAccess project.
|
- `yarn typecheck`: includes the package strict/noUncheckedIndexedAccess project.
|
||||||
- `yarn test:asr-types-package`: packs and installs core/ASR outside the workspace,
|
- `yarn test:asr-types-package`: packs and installs core/ASR outside the workspace,
|
||||||
checks frozen 2.0/2.1 consumers and candidate declarations in strict compiler
|
checks frozen 2.0/2.1 consumers and candidate declarations in strict compiler
|
||||||
@@ -127,6 +136,9 @@ From the repository root:
|
|||||||
|
|
||||||
Windows Playwright WebKit 26.6 lacks AudioContext and AudioWorkletNode. Its native
|
Windows Playwright WebKit 26.6 lacks AudioContext and AudioWorkletNode. Its native
|
||||||
ASR cases remain explicit skips and do not establish Safari/device compatibility.
|
ASR cases remain explicit skips and do not establish Safari/device compatibility.
|
||||||
Real fallback track replacement, native volume/mute, CORS, unsupported media and
|
Native direct/fallback routing, volume/mute, CORS and audio-track combinations
|
||||||
physical devices remain PKG-ASR-05 acceptance work. Do not use the demo's external
|
have scoped browser evidence. Unsupported media, external-owner integration,
|
||||||
|
Safari and physical devices remain PKG-ASR-05 acceptance work. CORS restrictions
|
||||||
|
and main-video-only capture are explicit integration boundaries, not a claim
|
||||||
|
that every media source can be recognized. Do not use the demo's external
|
||||||
recognizer service as a prerequisite for local audio verification.
|
recognizer service as a prerequisite for local audio verification.
|
||||||
@@ -37,6 +37,21 @@ The root declaration retains its historical NodeNext ESM namespace shape; use
|
|||||||
`/runtime` for a directly callable default import in that mode. This is an opt-in
|
`/runtime` for a directly callable default import in that mode. This is an opt-in
|
||||||
type entry, and does not require changes to existing JavaScript consumers.
|
type entry, and does not require changes to existing JavaScript consumers.
|
||||||
|
|
||||||
|
## Media access and separate audio
|
||||||
|
|
||||||
|
Cross-origin media needs both `moreVideoAttr: { crossOrigin: 'anonymous' }` on
|
||||||
|
ArtPlayer and an appropriate `Access-Control-Allow-Origin` response from the media
|
||||||
|
server. Set the attribute before loading the source. Without CORS access, the
|
||||||
|
default Web Audio route becomes silent even when the video timeline advances;
|
||||||
|
Chromium supplies zero PCM and Firefox may supply no chunks. A same-origin URL
|
||||||
|
that redirects to an unapproved origin has the same restriction. ASR does not
|
||||||
|
change request credentials or bypass browser media security.
|
||||||
|
|
||||||
|
ASR captures `art.video`. When used with `artplayer-plugin-audio-track`, the
|
||||||
|
independent `audio` element is not mixed into recognition. Muting the main video
|
||||||
|
silences its default ASR input even if a consumer separately unmutes that audio
|
||||||
|
element. Both plugins keep their existing volume and playback APIs.
|
||||||
|
|
||||||
## Maintenance
|
## Maintenance
|
||||||
|
|
||||||
See [ARCHITECTURE.md](ARCHITECTURE.md) for the current audio flow, public behavior,
|
See [ARCHITECTURE.md](ARCHITECTURE.md) for the current audio flow, public behavior,
|
||||||
|
|||||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,41 @@
|
|||||||
|
# PKG-ASR-05 检查点:CORS 与 audio-track 组合
|
||||||
|
|
||||||
|
此项仍为 doing 检查点。新增真实原生浏览器用例,补充跨源与独立音轨的明确边界;
|
||||||
|
没有改变生产源码、默认音频路由、公开类型或版本。
|
||||||
|
|
||||||
|
## CORS
|
||||||
|
|
||||||
|
浏览器测试服务器增加专用允许跨源音频入口和同源跳转到跨源的入口;其余媒体
|
||||||
|
路由保持原响应头。实际响应 URL、状态和允许头随每个用例记录,不使用 request
|
||||||
|
拦截伪造 PCM 或 CORS 权限。
|
||||||
|
|
||||||
|
真实 2.1.0 与候选在 5.4.0/current 核心分别观察:anonymous 属性与允许头同时存在
|
||||||
|
时得到非零 PCM 和原生输出;未设置跨源属性、以及同源 URL 跳转到无授权资源时
|
||||||
|
输出为零,但媒体时间仍前进。Chromium 收到零样本,Firefox 没有分块回调。候选
|
||||||
|
随后切到同源资源,在原有一个 AudioContext 中恢复非零 PCM 与播放输出。
|
||||||
|
|
||||||
|
首次测试错误地期待 Firefox 也返回零样本分块,8 项失败;保留 probe 报告后按
|
||||||
|
原生行为明确区分引擎,不把它改成任意零值或空列表都通过的宽松断言。
|
||||||
|
这符合 [Web Audio 跨源规则](https://www.w3.org/TR/webaudio-1.0/#MediaElementAudioSourceNode-security)。
|
||||||
|
默认绑定会让本来可播放的未授权跨源音频变静音,登记 ASR-CORS-01;不能把安全
|
||||||
|
规则观察通过写成该播放问题已修复。后续显式捕获模式可隔离播放归属,但不能
|
||||||
|
绕过 CORS,也不能仅凭 URL 推测重定向、MSE、blob 或凭据的实际授权结果。
|
||||||
|
|
||||||
|
## 独立 audio-track
|
||||||
|
|
||||||
|
子代理新增专用用例,主代理审查并运行。两个包均从真实源码/指定正式产物加载,
|
||||||
|
本地音频由原生 media 与 Worklet 处理。检查两个插件的独立元素归属、暂停恢复、
|
||||||
|
音轨 update 不改主视频、ASR stop 保留播放、再次播放重采集、destroy 与晚到调用。
|
||||||
|
另一顺序用例由测试自有 AudioContext 测量独立音轨:主视频静音而独立音轨有信号
|
||||||
|
时,ASR PCM 确实为零;unmute 后恢复。两个插件均不擅自关闭外部 probe Context。
|
||||||
|
|
||||||
|
这纠正了“audio-track 会自动 mute 主视频”的未证实推测:源码和真实初始状态均
|
||||||
|
显示两个元素不自动静音。默认 ASR 采主视频,没有混入独立 Audio 的承诺;README
|
||||||
|
和架构文档已说明配置与边界。
|
||||||
|
|
||||||
|
## 验证与未完成项
|
||||||
|
|
||||||
|
当前证据和正式产物摘要见 [验证记录](../baselines/asr-combinations-validation.json)。
|
||||||
|
Windows WebKit 缺少 WebAudio 的用例明确跳过,不等同 Safari 或物理音频输出。
|
||||||
|
外部 owner 的显式集成、其他不支持的媒体与设备仍在 ASR-05;ASR-06 发行工作、
|
||||||
|
识别服务 SDK-11 和整仓发布验收尚未完成。没有推送或 npm 发布。
|
||||||
+1
-1
@@ -532,7 +532,7 @@
|
|||||||
- PKG-ASR-02: [记录](changes/2026-09-13-PKG-ASR-02-audio-baseline.md) [记录](baselines/asr-audio-validation.json)
|
- PKG-ASR-02: [记录](changes/2026-09-13-PKG-ASR-02-audio-baseline.md) [记录](baselines/asr-audio-validation.json)
|
||||||
- PKG-ASR-03: [记录](changes/2026-09-13-PKG-ASR-03-audio-ownership.md) [记录](baselines/asr-ownership-validation.json)
|
- PKG-ASR-03: [记录](changes/2026-09-13-PKG-ASR-03-audio-ownership.md) [记录](baselines/asr-ownership-validation.json)
|
||||||
- PKG-ASR-04: [记录](changes/2026-09-13-PKG-ASR-04-public-types.md) [记录](baselines/asr-types-validation.json) [记录](baselines/asr-type-diagnostics.json)
|
- PKG-ASR-04: [记录](changes/2026-09-13-PKG-ASR-04-public-types.md) [记录](baselines/asr-types-validation.json) [记录](baselines/asr-type-diagnostics.json)
|
||||||
- PKG-ASR-05: [记录](baselines/asr-volume-validation.json) [记录](baselines/asr-fallback-validation.json) [记录](baselines/asr-core.json) [记录](baselines/asr-local-core-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-local-core-checkpoint.md)
|
- PKG-ASR-05: [记录](baselines/asr-volume-validation.json) [记录](baselines/asr-fallback-validation.json) [记录](baselines/asr-core.json) [记录](baselines/asr-local-core-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-local-core-checkpoint.md) [记录](baselines/asr-combinations-validation.json) [记录](changes/2026-09-13-PKG-ASR-05-cors-combinations.md)
|
||||||
- PKG-ADS-01: [记录](baselines/ads-release.json) [记录](baselines/ads-contract.md) [记录](baselines/ads-contract-validation.json) [记录](changes/2026-09-12-PKG-ADS-01-contract.md) [记录](scripts/ads-contract.test.mjs)
|
- PKG-ADS-01: [记录](baselines/ads-release.json) [记录](baselines/ads-contract.md) [记录](baselines/ads-contract-validation.json) [记录](changes/2026-09-12-PKG-ADS-01-contract.md) [记录](scripts/ads-contract.test.mjs)
|
||||||
- PKG-ADS-02: [记录](changes/2026-09-12-PKG-ADS-02-tests.md) [记录](ads-validation.md) [记录](baselines/ads-validation.json)
|
- PKG-ADS-02: [记录](changes/2026-09-12-PKG-ADS-02-tests.md) [记录](ads-validation.md) [记录](baselines/ads-validation.json)
|
||||||
- PKG-ADS-03: [记录](changes/2026-09-12-PKG-ADS-03-lifecycle.md) [记录](ads-validation.md) [记录](baselines/ads-lifecycle-validation.json)
|
- PKG-ADS-03: [记录](changes/2026-09-12-PKG-ADS-03-lifecycle.md) [记录](ads-validation.md) [记录](baselines/ads-lifecycle-validation.json)
|
||||||
|
|||||||
@@ -1,5 +1,16 @@
|
|||||||
# 进度与证据
|
# 进度与证据
|
||||||
|
|
||||||
|
## PKG-ASR-05 CORS 与独立音轨检查点
|
||||||
|
|
||||||
|
新增跨源允许/不允许/重定向及同源恢复、audio-track组合生命周期与独立音频归属
|
||||||
|
验证;source/main/legacy共72过、36项Windows WebKit能力跳过,无失败或重试。
|
||||||
|
初次Firefox无回调与Chromium零样本差异已单独记录。默认无CORS媒体会被绑定后
|
||||||
|
静音,ASR-CORS-01保持open;不能把边界观察当作修复。见[变更](changes/2026-09-13-PKG-ASR-05-cors-combinations.md)
|
||||||
|
与[证据](baselines/asr-combinations-validation.json)。README和架构已说明配置。
|
||||||
|
227项:129 done、17 doing、81 todo;209风险。ASR-05继续,显式外部归属、设备
|
||||||
|
和完整发行仍待完成。生产代码和产物未改变;没有推送或发布。
|
||||||
|
|
||||||
|
|
||||||
## PKG-ASR-05 本地编辑器与相邻核心检查点
|
## PKG-ASR-05 本地编辑器与相邻核心检查点
|
||||||
|
|
||||||
真实 npm 5.3.0 核心扩大对照:main 14过/7能力跳过,legacy 6过/3能力跳过。
|
真实 npm 5.3.0 核心扩大对照:main 14过/7能力跳过,legacy 6过/3能力跳过。
|
||||||
|
|||||||
@@ -212,3 +212,4 @@
|
|||||||
| ASR-VOLUME-01 | resolved / 已复现 | ASR applies media volume twice and squares playback amplitude | PKG-ASR-07 |
|
| ASR-VOLUME-01 | resolved / 已复现 | ASR applies media volume twice and squares playback amplitude | PKG-ASR-07 |
|
||||||
| ASR-FALLBACK-01 | resolved / 已复现 | Captured fallback produces a second audible route even when the video is muted | PKG-ASR-08 |
|
| ASR-FALLBACK-01 | resolved / 已复现 | Captured fallback produces a second audible route even when the video is muted | PKG-ASR-08 |
|
||||||
| ASR-FOREIGN-01 | open / 已复现 | Firefox permits parallel native media-element owners instead of selecting capture fallback | PKG-ASR-05 |
|
| ASR-FOREIGN-01 | open / 已复现 | Firefox permits parallel native media-element owners instead of selecting capture fallback | PKG-ASR-05 |
|
||||||
|
| ASR-CORS-01 | open / 已复现 | Default WebAudio binding silences CORS-cross-origin media that otherwise advances normally | PKG-ASR-05 |
|
||||||
@@ -4672,6 +4672,21 @@
|
|||||||
],
|
],
|
||||||
"compatibleResolution": "Establish an explicit compatible ownership contract for external audio graphs without silently changing default direct-path PCM/mute semantics.",
|
"compatibleResolution": "Establish an explicit compatible ownership contract for external audio graphs without silently changing default direct-path PCM/mute semantics.",
|
||||||
"closureCriteria": "Natural Firefox external-owner integration has no duplicate output or has a documented, validated explicit ownership mode; controlled binding rejection alone cannot close this finding."
|
"closureCriteria": "Natural Firefox external-owner integration has no duplicate output or has a documented, validated explicit ownership mode; controlled binding rejection alone cannot close this finding."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "ASR-CORS-01",
|
||||||
|
"title": "Default WebAudio binding silences CORS-cross-origin media that otherwise advances normally",
|
||||||
|
"confirmation": "reproduced",
|
||||||
|
"status": "open",
|
||||||
|
"owners": [
|
||||||
|
"PKG-ASR-05"
|
||||||
|
],
|
||||||
|
"evidence": [
|
||||||
|
"test/browser/asr-cors.spec.js",
|
||||||
|
"refactor/baselines/asr-combinations-validation.json"
|
||||||
|
],
|
||||||
|
"compatibleResolution": "Document required CORS setup and provide an explicitly chosen capture path that never takes over media playback; do not bypass browser security or silently change default PCM semantics.",
|
||||||
|
"closureCriteria": "Authorized capture and same-origin recovery remain valid; an explicit integration avoids direct ownership for restricted input, fails or stays silent without rerouting playback, and is documented with native evidence."
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
+3
-1
@@ -2714,7 +2714,9 @@
|
|||||||
"baselines/asr-fallback-validation.json",
|
"baselines/asr-fallback-validation.json",
|
||||||
"baselines/asr-core.json",
|
"baselines/asr-core.json",
|
||||||
"baselines/asr-local-core-validation.json",
|
"baselines/asr-local-core-validation.json",
|
||||||
"changes/2026-09-13-PKG-ASR-05-local-core-checkpoint.md"
|
"changes/2026-09-13-PKG-ASR-05-local-core-checkpoint.md",
|
||||||
|
"baselines/asr-combinations-validation.json",
|
||||||
|
"changes/2026-09-13-PKG-ASR-05-cors-combinations.md"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -0,0 +1,190 @@
|
|||||||
|
import fs from 'node:fs'
|
||||||
|
import process from 'node:process'
|
||||||
|
import { hash } from '../../refactor/scripts/releases.mjs'
|
||||||
|
import { compilePackage } from '../helpers/load.js'
|
||||||
|
import { expect, test } from './fixtures.js'
|
||||||
|
|
||||||
|
let asrCode
|
||||||
|
let audioCode
|
||||||
|
let inputs
|
||||||
|
test.beforeAll(async () => {
|
||||||
|
expect(process.env.ARTPLAYER_BROWSER_ARTIFACTS, 'These source/explicit artifact checks do not represent isolated installed packages').toBeFalsy()
|
||||||
|
asrCode = process.env.ARTPLAYER_ASR_ARTIFACT ? fs.readFileSync(process.env.ARTPLAYER_ASR_ARTIFACT, 'utf8') : await compilePackage('artplayer-plugin-asr', 'umd')
|
||||||
|
audioCode = process.env.ARTPLAYER_AUDIO_ARTIFACT ? fs.readFileSync(process.env.ARTPLAYER_AUDIO_ARTIFACT, 'utf8') : await compilePackage('artplayer-plugin-audio-track', 'umd')
|
||||||
|
const media = fs.readFileSync(new URL('./media/audio-tone.m4a', import.meta.url))
|
||||||
|
const manifest = JSON.parse(fs.readFileSync(new URL('./media/audio-tone.json', import.meta.url)))
|
||||||
|
expect(hash(media)).toBe(manifest.sha256)
|
||||||
|
inputs = {
|
||||||
|
asr: { input: process.env.ARTPLAYER_ASR_ARTIFACT || 'workspace source', sha256: hash(asrCode) },
|
||||||
|
audioTrack: { input: process.env.ARTPLAYER_AUDIO_ARTIFACT || 'workspace source', sha256: hash(audioCode) },
|
||||||
|
media: { file: 'test/browser/media/audio-tone.m4a', sha256: hash(media), bytes: media.length },
|
||||||
|
recognitionService: false,
|
||||||
|
scope: 'Native media/WebAudio composition; not physical speaker, device, or isolated package acceptance',
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
async function openCombination(page, core, audioFirst, testInfo) {
|
||||||
|
await testInfo.attach('asr-audio-track-inputs', { contentType: 'application/json', body: JSON.stringify({ ...inputs, core, audioFirst }) })
|
||||||
|
await page.goto(`/test/player.html?core=${core}`)
|
||||||
|
const capabilities = await page.evaluate(() => ({ audioContext: typeof (window.AudioContext || window.webkitAudioContext), audioWorklet: typeof window.AudioWorkletNode }))
|
||||||
|
await testInfo.attach('native-audio-capabilities', { contentType: 'application/json', body: JSON.stringify(capabilities) })
|
||||||
|
if (capabilities.audioContext === 'undefined') {
|
||||||
|
expect(process.platform).toBe('win32')
|
||||||
|
expect(testInfo.project.name).toBe('webkit')
|
||||||
|
expect(capabilities.audioWorklet).toBe('undefined')
|
||||||
|
test.skip(true, 'Windows WebKit lacks native AudioContext/AudioWorklet; Safari and physical-device composition remain unverified')
|
||||||
|
}
|
||||||
|
expect(capabilities.audioWorklet).toBe('function')
|
||||||
|
await page.addScriptTag({ content: audioCode })
|
||||||
|
await page.addScriptTag({ content: asrCode })
|
||||||
|
await page.evaluate((audioFirst) => {
|
||||||
|
window.NativeAudioContext = window.AudioContext || window.webkitAudioContext
|
||||||
|
window.asrContexts = []
|
||||||
|
window.sourceElements = []
|
||||||
|
window.AudioContext = class extends window.NativeAudioContext {
|
||||||
|
constructor(options) {
|
||||||
|
super(options)
|
||||||
|
window.asrContexts.push(this)
|
||||||
|
}
|
||||||
|
|
||||||
|
createMediaElementSource(element) {
|
||||||
|
const source = super.createMediaElementSource(element)
|
||||||
|
window.sourceElements.push(element)
|
||||||
|
return source
|
||||||
|
}
|
||||||
|
}
|
||||||
|
window.asrChunks = []
|
||||||
|
const asr = window.artplayerPluginAsr({
|
||||||
|
onAudioChunk({ pcm, wav }) {
|
||||||
|
let peak = 0
|
||||||
|
for (const sample of new Int16Array(pcm))
|
||||||
|
peak = Math.max(peak, Math.abs(sample))
|
||||||
|
window.asrChunks.push({ peak, pcmBytes: pcm.byteLength, wavBytes: wav.byteLength, sampleRate: new DataView(wav).getUint32(24, true), source: window.art.video.currentSrc })
|
||||||
|
return 'Local main-media transcript.'
|
||||||
|
},
|
||||||
|
})
|
||||||
|
const audio = window.artplayerPluginAudioTrack({ url: '/test/audio-tone.m4a?track=first', sync: 0.1 })
|
||||||
|
window.art = new window.Artplayer({
|
||||||
|
container: '.player',
|
||||||
|
url: '/test/audio-tone.m4a?main=first',
|
||||||
|
volume: 0.4,
|
||||||
|
muted: false,
|
||||||
|
plugins: audioFirst ? [audio, asr] : [asr, audio],
|
||||||
|
})
|
||||||
|
window.audioPlugin = window.art.plugins.artplayerPluginAudioTrack
|
||||||
|
window.asrPlugin = window.art.plugins.artplayerPluginAsr
|
||||||
|
document.querySelector('#play').onclick = () => window.art.play()
|
||||||
|
document.querySelector('#pause').onclick = () => window.art.pause()
|
||||||
|
}, audioFirst)
|
||||||
|
await expect.poll(() => page.evaluate(() => window.art.isReady && window.audioPlugin.audio.readyState >= 2)).toBe(true)
|
||||||
|
expect(await page.evaluate(() => ({ mainMuted: window.art.video.muted, audioMuted: window.audioPlugin.audio.muted, separateElements: window.art.video !== window.audioPlugin.audio }))).toEqual({ mainMuted: false, audioMuted: false, separateElements: true })
|
||||||
|
await page.locator('#play').click()
|
||||||
|
await expect.poll(() => page.evaluate(() => window.asrChunks.filter(chunk => chunk.peak > 0).length)).toBeGreaterThanOrEqual(3)
|
||||||
|
await expect.poll(() => page.evaluate(() => window.audioPlugin.audio.currentTime)).toBeGreaterThan(0.2)
|
||||||
|
expect(await page.evaluate(() => window.sourceElements.length === 1 && window.sourceElements[0] === window.art.video)).toBe(true)
|
||||||
|
expect(await page.evaluate(() => window.asrChunks.every(chunk => chunk.pcmBytes === 3200 && chunk.wavBytes === 3244 && chunk.sampleRate === 16000 && chunk.source.includes('main=first')))).toBe(true)
|
||||||
|
}
|
||||||
|
|
||||||
|
test.afterEach(async ({ page }, testInfo) => {
|
||||||
|
const evidence = await page.evaluate(() => ({
|
||||||
|
chunks: window.asrChunks,
|
||||||
|
contexts: window.asrContexts?.map(context => context.state),
|
||||||
|
audio: window.audioPlugin && { source: window.audioPlugin.audio.currentSrc, attribute: window.audioPlugin.audio.getAttribute('src'), time: window.audioPlugin.audio.currentTime, paused: window.audioPlugin.audio.paused, muted: window.audioPlugin.audio.muted, error: window.audioPlugin.audio.error?.code },
|
||||||
|
})).catch(error => ({ error: error.message }))
|
||||||
|
await testInfo.attach('asr-audio-track-state', { contentType: 'application/json', body: JSON.stringify(evidence) })
|
||||||
|
await page.evaluate(async () => {
|
||||||
|
if (window.art && !window.art.isDestroy)
|
||||||
|
window.art.destroy()
|
||||||
|
if (window.probeContext && window.probeContext.state !== 'closed')
|
||||||
|
await window.probeContext.close()
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
for (const core of ['published', 'candidate']) {
|
||||||
|
test(`${core} core + ASR/audio-track: pause, update, ASR stop and destroy preserve separate ownership`, async ({ page }, testInfo) => {
|
||||||
|
await openCombination(page, core, true, testInfo)
|
||||||
|
await expect(page.locator('.art-layer-asr')).toContainText('Local main-media transcript.')
|
||||||
|
await page.locator('#pause').click()
|
||||||
|
await expect.poll(() => page.evaluate(() => window.audioPlugin.audio.paused && window.art.video.paused)).toBe(true)
|
||||||
|
const pausedChunks = await page.evaluate(() => window.asrChunks.length)
|
||||||
|
await page.waitForTimeout(350)
|
||||||
|
expect(await page.evaluate(() => window.asrChunks.length)).toBe(pausedChunks)
|
||||||
|
await page.locator('#play').click()
|
||||||
|
await expect.poll(() => page.evaluate(() => window.asrChunks.length)).toBeGreaterThan(pausedChunks + 2)
|
||||||
|
await expect.poll(() => page.evaluate(() => window.audioPlugin.audio.paused)).toBe(false)
|
||||||
|
expect(await page.evaluate(() => window.asrContexts.length)).toBe(1)
|
||||||
|
|
||||||
|
const beforeUpdate = await page.evaluate(() => {
|
||||||
|
window.originalAudio = window.audioPlugin.audio
|
||||||
|
const count = window.asrChunks.length
|
||||||
|
window.audioPlugin.update({ url: '/test/audio-tone.m4a?track=second' })
|
||||||
|
return count
|
||||||
|
})
|
||||||
|
await expect.poll(() => page.evaluate(() => window.audioPlugin.audio.currentSrc)).toContain('track=second')
|
||||||
|
await expect.poll(() => page.evaluate(() => !window.audioPlugin.audio.paused && window.audioPlugin.audio.currentTime > 0.1)).toBe(true)
|
||||||
|
await expect.poll(() => page.evaluate(() => window.asrChunks.length)).toBeGreaterThan(beforeUpdate + 2)
|
||||||
|
expect(await page.evaluate(() => window.originalAudio === window.audioPlugin.audio && window.art.video.currentSrc.includes('main=first'))).toBe(true)
|
||||||
|
expect(await page.evaluate(() => window.asrContexts.length)).toBe(1)
|
||||||
|
|
||||||
|
await page.evaluate(() => window.asrPlugin.stop())
|
||||||
|
const stopped = await page.evaluate(() => ({ chunks: window.asrChunks.length, time: window.audioPlugin.audio.currentTime }))
|
||||||
|
await expect.poll(() => page.evaluate(() => window.audioPlugin.audio.currentTime)).toBeGreaterThan(stopped.time + 0.4)
|
||||||
|
expect(await page.evaluate(() => window.asrChunks.length)).toBe(stopped.chunks)
|
||||||
|
expect(await page.evaluate(() => window.art.video.paused || window.audioPlugin.audio.paused)).toBe(false)
|
||||||
|
await page.locator('#pause').click()
|
||||||
|
await page.locator('#play').click()
|
||||||
|
await expect.poll(() => page.evaluate(() => window.asrChunks.length)).toBeGreaterThan(stopped.chunks + 2)
|
||||||
|
await page.evaluate(() => window.art.destroy())
|
||||||
|
await expect.poll(() => page.evaluate(() => window.asrContexts.every(context => context.state === 'closed'))).toBe(true)
|
||||||
|
await expect.poll(() => page.evaluate(() => window.audioPlugin.audio.networkState)).toBe(0)
|
||||||
|
const destroyedCount = await page.evaluate(() => window.asrChunks.length)
|
||||||
|
await page.evaluate(() => {
|
||||||
|
window.audioPlugin.update({ url: '/test/audio-tone.m4a?track=revived' })
|
||||||
|
window.asrPlugin.append('Late transcript must not revive the player')
|
||||||
|
})
|
||||||
|
expect(await page.evaluate(() => ({ src: window.audioPlugin.audio.getAttribute('src'), paused: window.audioPlugin.audio.paused, readyState: window.audioPlugin.audio.readyState }))).toEqual({ src: null, paused: true, readyState: 0 })
|
||||||
|
await expect(page.locator('.art-video-player')).toHaveCount(0)
|
||||||
|
await page.waitForTimeout(250)
|
||||||
|
expect(await page.evaluate(() => window.asrChunks.length)).toBe(destroyedCount)
|
||||||
|
})
|
||||||
|
|
||||||
|
test(`${core} core + ASR/audio-track: capture follows main media, not the independent audible track`, async ({ page }, testInfo) => {
|
||||||
|
await openCombination(page, core, false, testInfo)
|
||||||
|
await page.evaluate(async () => {
|
||||||
|
// Real native probe of the independently exposed Audio element, with its own owner.
|
||||||
|
window.probeContext = new window.NativeAudioContext()
|
||||||
|
const source = window.probeContext.createMediaElementSource(window.audioPlugin.audio)
|
||||||
|
window.trackAnalyser = window.probeContext.createAnalyser()
|
||||||
|
source.connect(window.trackAnalyser)
|
||||||
|
window.trackAnalyser.connect(window.probeContext.destination)
|
||||||
|
await window.probeContext.resume()
|
||||||
|
window.art.muted = true
|
||||||
|
})
|
||||||
|
await expect.poll(() => page.evaluate(() => window.art.video.muted && window.audioPlugin.audio.muted)).toBe(true)
|
||||||
|
await page.evaluate(() => {
|
||||||
|
// Consumers may override the public external element; the plugin never mutes main media itself.
|
||||||
|
window.audioPlugin.audio.muted = false
|
||||||
|
})
|
||||||
|
const peak = () => page.evaluate(() => {
|
||||||
|
const samples = new Float32Array(window.trackAnalyser.fftSize)
|
||||||
|
window.trackAnalyser.getFloatTimeDomainData(samples)
|
||||||
|
return Math.max(...samples.map(Math.abs))
|
||||||
|
})
|
||||||
|
await expect.poll(peak).toBeGreaterThan(0.001)
|
||||||
|
await expect.poll(() => page.evaluate(() => window.asrChunks.slice(-5).length === 5 && window.asrChunks.slice(-5).every(chunk => chunk.peak === 0))).toBe(true)
|
||||||
|
expect(await page.evaluate(() => window.audioPlugin.audio.paused)).toBe(false)
|
||||||
|
expect(await page.evaluate(() => window.sourceElements.length === 1 && window.sourceElements[0] === window.art.video)).toBe(true)
|
||||||
|
await testInfo.attach('separate-audio-routing', { contentType: 'application/json', body: JSON.stringify({ independentTrackPeak: await peak(), silentMainChunks: await page.evaluate(() => window.asrChunks.slice(-5)) }) })
|
||||||
|
const beforeUnmute = await page.evaluate(() => {
|
||||||
|
window.art.muted = false
|
||||||
|
return window.asrChunks.length
|
||||||
|
})
|
||||||
|
await expect.poll(() => page.evaluate(start => window.asrChunks.slice(start).filter(chunk => chunk.peak > 0).length, beforeUnmute)).toBeGreaterThanOrEqual(3)
|
||||||
|
await page.evaluate(() => window.art.destroy())
|
||||||
|
await expect.poll(() => page.evaluate(() => window.asrContexts.every(context => context.state === 'closed'))).toBe(true)
|
||||||
|
// The test owns the probe; neither plugin may close a context created by another owner.
|
||||||
|
expect(await page.evaluate(() => window.probeContext.state)).toBe('running')
|
||||||
|
await page.evaluate(() => window.probeContext.close())
|
||||||
|
await expect.poll(() => page.evaluate(() => window.audioPlugin.audio.networkState)).toBe(0)
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -0,0 +1,136 @@
|
|||||||
|
import fs from 'node:fs'
|
||||||
|
import process from 'node:process'
|
||||||
|
import { verifyAsrContract } from '../../refactor/scripts/asr-contract.mjs'
|
||||||
|
import { hash, readMember } from '../../refactor/scripts/releases.mjs'
|
||||||
|
import { compilePackage } from '../helpers/load.js'
|
||||||
|
import { expect, test } from './fixtures.js'
|
||||||
|
|
||||||
|
let contract
|
||||||
|
let candidateCode
|
||||||
|
test.beforeAll(async () => {
|
||||||
|
contract = await verifyAsrContract()
|
||||||
|
candidateCode = process.env.ARTPLAYER_ASR_ARTIFACT ? fs.readFileSync(process.env.ARTPLAYER_ASR_ARTIFACT) : await compilePackage('artplayer-plugin-asr', 'umd')
|
||||||
|
})
|
||||||
|
|
||||||
|
for (const version of ['2.1.0', 'candidate']) {
|
||||||
|
for (const core of ['published', 'candidate']) {
|
||||||
|
for (const scenario of ['anonymous-cors', 'opaque', 'opaque-redirect']) {
|
||||||
|
test(`ASR ${version} + ${core}: ${scenario} native audio access and source recovery`, async ({ page }, testInfo) => {
|
||||||
|
await page.goto(`/test/player.html?core=${core}`)
|
||||||
|
const capabilities = await page.evaluate(() => ({ context: typeof window.AudioContext, prefixed: typeof window.webkitAudioContext, worklet: typeof window.AudioWorkletNode }))
|
||||||
|
await testInfo.attach('asr-cors-capabilities', { contentType: 'application/json', body: JSON.stringify(capabilities) })
|
||||||
|
if (capabilities.context === 'undefined' && capabilities.prefixed === 'undefined') {
|
||||||
|
expect(process.platform).toBe('win32')
|
||||||
|
expect(testInfo.project.name).toBe('webkit')
|
||||||
|
expect(capabilities.worklet).toBe('undefined')
|
||||||
|
test.skip(true, 'Windows WebKit lacks native WebAudio; this is not Safari CORS acceptance')
|
||||||
|
}
|
||||||
|
const code = version === 'candidate' ? candidateCode : readMember(contract.archives.get(version), 'package/dist/artplayer-plugin-asr.js')
|
||||||
|
await testInfo.attach('asr-cors-inputs', { contentType: 'application/json', body: JSON.stringify({ version, core, scenario, pluginSha256: hash(code), mediaSha256: hash(fs.readFileSync(new URL('./media/audio-tone.m4a', import.meta.url))), physicalAudioOutput: false, securityBypass: false }) })
|
||||||
|
const responses = []
|
||||||
|
page.on('response', (response) => {
|
||||||
|
if (/\/test\/(?:asr-cors\.m4a|audio-tone\.m4a|asr-opaque-redirect)(?:\?|$)/u.test(response.url()))
|
||||||
|
responses.push({ url: response.url(), status: response.status(), allowOrigin: response.headers()['access-control-allow-origin'] || null })
|
||||||
|
})
|
||||||
|
await page.addScriptTag({ content: code.toString() })
|
||||||
|
await page.evaluate((scenario) => {
|
||||||
|
const NativeContext = window.AudioContext || window.webkitAudioContext
|
||||||
|
window.asrContexts = []
|
||||||
|
window.asrChunks = []
|
||||||
|
window.asrMeters = []
|
||||||
|
window.AudioContext = class extends NativeContext {
|
||||||
|
constructor(options) {
|
||||||
|
super(options)
|
||||||
|
window.asrContexts.push(this)
|
||||||
|
}
|
||||||
|
|
||||||
|
createGain() {
|
||||||
|
const gain = super.createGain()
|
||||||
|
const analyser = super.createAnalyser()
|
||||||
|
const connect = gain.connect.bind(gain)
|
||||||
|
const disconnect = gain.disconnect.bind(gain)
|
||||||
|
gain.connect = (destination, ...args) => {
|
||||||
|
if (destination === this.destination) {
|
||||||
|
connect(analyser)
|
||||||
|
analyser.connect(destination)
|
||||||
|
return destination
|
||||||
|
}
|
||||||
|
return connect(destination, ...args)
|
||||||
|
}
|
||||||
|
gain.disconnect = (...args) => {
|
||||||
|
analyser.disconnect()
|
||||||
|
return disconnect(...args)
|
||||||
|
}
|
||||||
|
window.asrMeters.push(analyser)
|
||||||
|
return gain
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const alternate = location.origin.replace('127.0.0.1', 'localhost')
|
||||||
|
const url = scenario === 'anonymous-cors' ? `${alternate}/test/asr-cors.m4a` : scenario === 'opaque' ? `${alternate}/test/audio-tone.m4a` : '/test/asr-opaque-redirect'
|
||||||
|
window.art = new window.Artplayer({
|
||||||
|
container: '.player',
|
||||||
|
url,
|
||||||
|
volume: 1,
|
||||||
|
loop: true,
|
||||||
|
moreVideoAttr: scenario === 'anonymous-cors' ? { crossOrigin: 'anonymous' } : {},
|
||||||
|
plugins: [window.artplayerPluginAsr({ onAudioChunk({ pcm }) {
|
||||||
|
const values = new Int16Array(pcm)
|
||||||
|
window.asrChunks.push({ max: values.reduce((max, value) => Math.max(max, Math.abs(value)), 0), source: window.art.video.currentSrc })
|
||||||
|
} })],
|
||||||
|
})
|
||||||
|
document.querySelector('#play').onclick = () => window.art.play()
|
||||||
|
window.asrReading = () => {
|
||||||
|
const analyser = window.asrMeters.at(-1)
|
||||||
|
const values = new Float32Array(analyser.fftSize)
|
||||||
|
analyser.getFloatTimeDomainData(values)
|
||||||
|
return { outputMax: values.reduce((max, value) => Math.max(max, Math.abs(value)), 0), chunks: window.asrChunks.slice(-3), time: window.art.currentTime, paused: window.art.video.paused, error: window.art.video.error?.code || null, crossOrigin: window.art.video.crossOrigin }
|
||||||
|
}
|
||||||
|
}, scenario)
|
||||||
|
await expect.poll(() => page.evaluate(() => window.art.isReady)).toBe(true)
|
||||||
|
await page.locator('#play').click()
|
||||||
|
// Firefox suppresses restricted input entirely; Chromium delivers zero samples.
|
||||||
|
if (testInfo.project.name === 'firefox' && scenario !== 'anonymous-cors') {
|
||||||
|
await expect.poll(() => page.evaluate(() => window.asrMeters.length)).toBe(1)
|
||||||
|
await expect.poll(() => page.evaluate(() => window.art.currentTime)).toBeGreaterThan(1)
|
||||||
|
expect(await page.evaluate(() => window.asrChunks.length)).toBe(0)
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
await expect.poll(() => page.evaluate(() => window.asrChunks.length)).toBeGreaterThanOrEqual(4)
|
||||||
|
}
|
||||||
|
const initial = await page.evaluate(() => window.asrReading())
|
||||||
|
expect(initial.error).toBeNull()
|
||||||
|
expect(initial.paused).toBe(false)
|
||||||
|
if (scenario === 'anonymous-cors') {
|
||||||
|
expect(initial.crossOrigin).toBe('anonymous')
|
||||||
|
expect(initial.outputMax).toBeGreaterThan(0.01)
|
||||||
|
expect(initial.chunks.every(chunk => chunk.max > 0)).toBe(true)
|
||||||
|
expect(responses.some(response => response.allowOrigin === '*')).toBe(true)
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
expect(initial.crossOrigin).toBeNull()
|
||||||
|
expect(initial.outputMax).toBe(0)
|
||||||
|
expect(initial.chunks.every(chunk => chunk.max === 0)).toBe(true)
|
||||||
|
expect(responses.every(response => response.allowOrigin === null)).toBe(true)
|
||||||
|
if (scenario === 'opaque-redirect')
|
||||||
|
expect(responses.some(response => response.status === 302)).toBe(true)
|
||||||
|
}
|
||||||
|
await expect.poll(() => page.evaluate(() => window.art.currentTime)).toBeGreaterThan(initial.time + 0.3)
|
||||||
|
let recovery
|
||||||
|
if (version === 'candidate') {
|
||||||
|
await page.evaluate(() => {
|
||||||
|
window.art.url = '/test/audio-tone.m4a?asr-cors=recovery'
|
||||||
|
})
|
||||||
|
await expect.poll(() => page.evaluate(() => window.art.video.currentSrc.includes('asr-cors=recovery') && window.art.video.readyState >= 2)).toBe(true)
|
||||||
|
await page.locator('#play').click()
|
||||||
|
await expect.poll(() => page.evaluate(() => window.asrChunks.filter(chunk => chunk.source.includes('asr-cors=recovery') && chunk.max > 0).length)).toBeGreaterThanOrEqual(3)
|
||||||
|
recovery = await page.evaluate(() => window.asrReading())
|
||||||
|
expect(recovery.outputMax).toBeGreaterThan(0.01)
|
||||||
|
expect(await page.evaluate(() => window.asrContexts.length)).toBe(1)
|
||||||
|
}
|
||||||
|
await testInfo.attach('asr-cors-observation', { contentType: 'application/json', body: JSON.stringify({ initial, recovery, responses }) })
|
||||||
|
await page.evaluate(() => window.art.destroy())
|
||||||
|
await expect.poll(() => page.evaluate(() => window.asrContexts.map(context => context.state))).toEqual(['closed'])
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -116,6 +116,15 @@ const server = http.createServer((req, res) => {
|
|||||||
assert(['GET', 'HEAD'].includes(req.method), 'Unsupported method')
|
assert(['GET', 'HEAD'].includes(req.method), 'Unsupported method')
|
||||||
const url = new URL(req.url, `http://127.0.0.1:${port}`)
|
const url = new URL(req.url, `http://127.0.0.1:${port}`)
|
||||||
const caseId = url.searchParams.get('case')
|
const caseId = url.searchParams.get('case')
|
||||||
|
if (url.pathname === '/test/asr-cors.m4a') {
|
||||||
|
res.setHeader('Access-Control-Allow-Origin', '*')
|
||||||
|
send(req, res, files.get('/test/audio-tone.m4a'), url.pathname)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (url.pathname === '/test/asr-opaque-redirect') {
|
||||||
|
res.writeHead(302, { Location: `http://localhost:${port}/test/audio-tone.m4a` }).end()
|
||||||
|
return
|
||||||
|
}
|
||||||
if (url.pathname === '/test/iframe-boundary-redirect') {
|
if (url.pathname === '/test/iframe-boundary-redirect') {
|
||||||
res.writeHead(302, { Location: `http://localhost:${port}/test/iframe-boundary-child.html${url.search}` }).end()
|
res.writeHead(302, { Location: `http://localhost:${port}/test/iframe-boundary-child.html${url.search}` }).end()
|
||||||
return
|
return
|
||||||
|
|||||||
Reference in new issue
Block a user