test(performance): [ENG-13] enforce measured teardown observation minimum

This commit is contained in:
Harvey Zhao committed 2026-09-14 16:02:05 +08:00
1 parent 23a4f22b72
commit 48a7f1ecb0
11 files changed
+22120 -12

No files matched your search

File diff suppressed because it is too large. Load diff
@@ -0,0 +1,47 @@
# ENG-13 性能资源探针的最小观察时间
MOD-03在新的实际安装包上执行正式性能配对时,Firefox第一轮报告因一项candidate
资源观察只有349ms而失败。该项是第二组chapter的最后一次资源探针;原门槛要求
至少350ms,因此失败是有效的观察缺口,不能把它四舍五入成通过或覆盖原报告。
Chromium/WebKit同轮完成,仍有既有体积/计时review-required信号。
## 原因与修复范围
冻结BASE-06夹具在记录起始时钟后只执行一次原生setTimeout(350),然后记录实际
performance.now差值。名义延时不等于该次记录一定已满350ms;现有证据不能进一步
区分计时量化和原生调度因素,不能据此断言Firefox存在特定计时器实现缺陷。
修改只在`scripts/performance-fixture.mjs`的当前适配层加入waitForObservation。
它按同一个实测时钟计算剩余时间,等待后重新核对,不足时继续等待剩余部分。
使用probe已有的原始定时器,不把观察器自身加入播放器待释放定时器列表;没有
改变库的清理动作。记录仍使用实际now()-waitStarted,保持超出的实测时间。
`refactor/fixtures/performance.js`、旧JSON、所有计时阈值及350ms校验均不修改。
适配层对旧、新两个播放器使用相同处理;构造/ready/play/destroy计时循环、媒体、
热身、样本数量和交替顺序不变。变化仅限计时采样之外的销毁后资源观察窗口及
原有结果传输替换,测试逐字反向恢复冻结源来保护这个边界。
## 验证
- 原报告保留:`run-firefox-ERtLa6/report.json`,observedMs=349,校验明确失败;
原Chromium/WebKit报告也保留,不把本轮重测覆盖到它们。
- 虚拟单调时钟覆盖首次349ms后补1ms、原生超出到364ms时保留实测值、已经满足
时不再等待,以及定时器拒绝原样传播。原349ms候选报告变体仍须被校验器拒绝。
- 适配层测试证明去掉结果传输和最小观察适配后,内容与冻结文件完全一致。
- 使用同一`run-rawoRh`实际安装的core/chapter产物重新执行三引擎完整三组配对。
包的源文件、构建工具和安装字节由现有严格验证器核对;该补丁不改变包内容。
最终9项测试通过(含冻结发布性能/压缩基线),定向lint、严格工具链、计划及风险
校验通过。原生三引擎三项完整配对测试通过,共108个旧新资源窗口,54个候选窗口
均通过严格清理检查。最短实际观察为Chromium350.2000000476837ms、Firefox350ms、
WebKit350ms;没有取整或使用名义延时代替真实值。第二轮没有计时reviewSignals,
但首轮Chromium销毁计时信号仍保留,不能把样本波动当作本补丁带来的计时收益。
三引擎均因原有分发体积信号保持review-required。
具体最终结果、环境、来源指纹、原失败和修复后报告见
[机器证据](../baselines/observation-window-validation.json)。该任务只处理观察窗口
可靠性,不能把review-required变成性能发布通过;MOD-03及ENG-PERF-01/02继续。
也不表示所有插件、物理设备、远端CI或最终候选发布完成。无新依赖或API改动。
回退本任务提交即可恢复旧适配器;不要回写冻结夹具或放宽350ms断言。原失败应
继续保留。完成后独立本地提交;不push/publish。
+5
View File
@@ -50,6 +50,11 @@ Node CI 新增 Ubuntu/Windows 覆盖率 job,验证固定工具链并运行完
计时和体积触发审查不等于资源泄漏。候选资源检查严格要求销毁后无实例、DOM、proxy 监听器、RAF、剩余定时器和晚到回调,不以旧版 BASE-PERF-01 的残留作为容许值。计时阶段不装资源探针,资源探针清理自身计时器发生在记录之后。物理设备、GPU/浏览器原生内存仍不能靠这些计数证明无泄漏。
ENG-13修复了名义350ms定时器返回后实测只有349ms的观察缺口。当前页面适配器
使用同一单调时钟补足剩余时间,仍记录实际观察值;原350ms校验、冻结BASE-06
夹具和计时循环不变。旧新双方使用相同等待规则,详见
[观察窗口记录](changes/2026-09-14-ENG-13-observation-window.md)。
先运行 `yarn test:package`,将 ARTPLAYER_BROWSER_ARTIFACTS 设为生成的 browser-artifacts.json,再执行 `yarn test:performance`。单独的 Playwright 配置按单 worker 运行三种引擎,关闭 trace/video,禁止与其他浏览器或重型构建任务同时计时。测试复用冻结 BASE-06 测量体,仅通过适配器修改脚本 URL 和报告提交方式,原始 fixture 不变。加载前检查包源码/声明来源、构建脚本、依赖和分发哈希,旧产物不能代替当前源码。
scripts/performance-artifacts.mjs 管输入和压缩,performance-fixture.mjs 管冻结 fixture 适配,performance-report.mjs 管配对与资源门槛,performance-summary.mjs 输出审查表;test/performance/paired.spec.js 管真实浏览器执行,test/performance-report.test.js 覆盖不完整配对、换环境、假播放、资源残留、过期源码和改动产物等失败。共享的测量验证从原 performance.mjs 提取为函数,旧发布基线仍保留完整来源校验。
+5 -3
View File
@@ -2,9 +2,9 @@
> 由 tasks.json 生成。请修改数据后运行 `node refactor/scripts/plan.mjs --write`,不要手改本表。
基线:`40fcda6a37d0049d42e49c1e64e70d4fd9ba5f7f`。总任务 254 项,范围 22 个包及工作区/示例。
基线:`40fcda6a37d0049d42e49c1e64e70d4fd9ba5f7f`。总任务 255 项,范围 22 个包及工作区/示例。
状态:todo 53 / doing 18 / blocked 0 / done 183 / deferred 0。风险 L/M/H 表示兼容风险,不表示工期。
状态:todo 53 / doing 18 / blocked 0 / done 184 / deferred 0。风险 L/M/H 表示兼容风险,不表示工期。
前置依赖是启动条件;验收是完成条件。任务可以继续拆分,但不能复用或悄悄删除旧 ID。
@@ -434,7 +434,7 @@
| ID | 范围 / 步骤 | 前置依赖 | 交付物 | 验收条件 | 风险 | 状态 |
| --- | --- | --- | --- | --- | --- | --- |
| REVIEW-01 | workspace<br>第一轮全项目架构与兼容性复盘 | REL-02, CORE-22, SITE-04, DOC-08, MOD-01, MOD-02, MOD-03, CORE-25 | 22 包结构、类型、旧 API/路径、测试盲区和维护文档的逐包报告 | 本轮阻断项由独立修复任务关闭并复测;环境缺口转交下一轮,不宣称已发布就绪 | H | todo |
| REVIEW-01 | workspace<br>第一轮全项目架构与兼容性复盘 | REL-02, CORE-22, SITE-04, DOC-08, MOD-01, MOD-02, MOD-03, CORE-25, ENG-13 | 22 包结构、类型、旧 API/路径、测试盲区和维护文档的逐包报告 | 本轮阻断项由独立修复任务关闭并复测;环境缺口转交下一轮,不宣称已发布就绪 | H | todo |
| REVIEW-02 | workspace<br>第二轮真实浏览器与生态集成复盘 | REVIEW-01, REL-03 | Chrome 交互、自动浏览器、新旧组合、真机/SDK/性能/资源的全范围报告 | 所需环境证据齐全,前轮修复再次核对;本轮阻断项关闭,无关 mock 不替代真实验收 | H | todo |
| REVIEW-03 | workspace<br>第三轮 npm 候选内容与发布准备复盘 | REVIEW-02, REL-04, CI-04 | 实际候选 integrity、干净安装构建/消费者、入口/许可/版本/tag/回退及前轮证据汇总 | 最终候选完整检查通过,发布阻断项为零;准备可审阅的批次报告,不自动执行 publish;逐包核对 major 目标,不能用大版本豁免旧 API 兼容 | H | todo |
@@ -455,6 +455,7 @@
| ID | 范围 / 步骤 | 前置依赖 | 交付物 | 验收条件 | 风险 | 状态 |
| --- | --- | --- | --- | --- | --- | --- |
| ENG-LINT-01 | artplayer-plugin-danmuku-mask, workspace<br>修复 Mask manifest 排序并恢复全仓 lint | PKG-MASK-04 | 仅调整 manifest 字段顺序,独立验证全仓只读 lint 与元数据语义不变 | 根 lint 通过;解析后的 manifest 深度相等;不改包版本、依赖或公开入口 | L | done |
| ENG-13 | workspace<br>保证性能资源探针的最小观察时长 | ENG-08 | 性能页面按实测时钟补足既有350ms观察窗口,保留冻结基线和原校验门槛;时钟边界测试和实际安装包配对证据 | 349ms仍被拒绝;早到定时器补足观察、正常超时保留实际值;冻结计时样本逻辑未变,三浏览器实际350ms观察及资源校验通过 | M | done |
## 完成证据与阻塞
@@ -661,3 +662,4 @@
- PKG-AUTO-THUMB-11: [记录](changes/2026-09-14-PKG-AUTO-THUMB-11-handler-cleanup.md) [记录](baselines/auto-thumbnail-handler-validation.json)
- PKG-VAST-07: [记录](changes/2026-09-14-PKG-VAST-07-default-alias.md) [记录](baselines/vast-alias-validation.json)
- PKG-JASSUB-10: [记录](changes/2026-09-14-PKG-JASSUB-10-render-resources.md) [记录](baselines/jassub-render-validation.json) [记录](baselines/jassub-render-patch.json)
- ENG-13: [记录](changes/2026-09-14-ENG-13-observation-window.md) [记录](baselines/observation-window-validation.json)
+13
View File
@@ -1,5 +1,18 @@
# 进度与证据
## ENG-13 性能探针观察窗口修复完成
MOD-03新隔离包run-rawoRh通过36项运行时、5组旧类型和8组精确类型消费。正式配对
首轮Firefox有一个candidate资源窗口实际仅349ms,正确触发350ms门槛;保留原失败。
当前页面适配器按同一时钟补足剩余观察时间,保留实际读数,不改冻结夹具、样本循环
或阈值。9项时钟/证据/冻结基线回归、lint和工具链通过;同一安装包重新完成三引擎
全部三组配对,54次candidate资源探针通过,所有旧新观察都达到350ms。
见[修复记录](changes/2026-09-14-ENG-13-observation-window.md)及
[证据](baselines/observation-window-validation.json)。第二轮无计时超阈值,体积仍需
审查;不把首轮波动归因于仅修改观察窗口的修复,也不把review-required改成通过。
255项:184 done、18 doing、53 todo。MOD-03继续持久化和初始化/布局分析;本次没有
改变播放器源码、依赖、公开接口或产物,不代表设备/全插件/远端CI/发布验收完成。
## MOD-03 进度/音量重复ARIA写入优化检查点(仍doing)
核心slider保留每次读取range/文本、即时更新和外部DOM修复,仅在属性值实际改变
+1
View File
@@ -269,3 +269,4 @@
| MULTI-SUB-SWITCH-01 | open / 已复现 | Intermittent WebKit core5.3.0 caption loss after selected-track source switch | PKG-MULTI-SUB-05, PKG-MULTI-SUB-09 |
| MULTI-SUB-ASS-01 | resolved / 已复现 | Published core5.1.2 ASS converter loses required VTT line breaks | PKG-MULTI-SUB-11 |
| JASSUB-RENDER-RESOURCES-01 | resolved / 已复现 | Main-thread async rendering leaves received ImageBitmaps open when native canvas operations throw | PKG-JASSUB-10 |
| ENG-PERF-OBS-01 | resolved / 已复现 | A nominal 350 ms resource observation can record only 349 ms and fail the required minimum | ENG-13 |
+23
View File
@@ -6131,6 +6131,29 @@
"test/browser/jassub-render-failure.spec.js"
],
"resolutionRationale": "Old exact main fails all four controlled disposal cases and all three native batch-disposal cases. Candidate full suite passes 201 cases; actual main/legacy runtime groups each pass 69 cases and each has 12 native failure/recovery/playback cases. The fix changes only async bitmap release on main-thread render and retains exceptions and synchronous buffers."
},
{
"id": "ENG-PERF-OBS-01",
"title": "A nominal 350 ms resource observation can record only 349 ms and fail the required minimum",
"confirmation": "reproduced",
"status": "resolved",
"owners": [
"ENG-13"
],
"evidence": [
"refactor/baselines/observation-window-validation.json",
"refactor/changes/2026-09-14-ENG-13-observation-window.md",
"test/performance-report.test.js"
],
"compatibleResolution": "Wait for the remaining measured interval using the original probe timer, applying the same rule to both variants and retaining actual times and all frozen inputs.",
"closureCriteria": "349 ms remains rejected; simulated early callback is re-armed; native three-engine candidate resource probes meet at least 350 ms with no resource residue; frozen timing loops and existing thresholds remain unchanged.",
"workspaceState": "The old Firefox report records one 349 ms candidate window. The page adapter now enforces the measured minimum; 54 candidate and 54 published resource windows meet the observation minimum, with strict candidate cleanup passing.",
"resolutionEvidence": [
"refactor/baselines/observation-window-validation.json",
"refactor/changes/2026-09-14-ENG-13-observation-window.md",
"test/performance-report.test.js"
],
"resolutionRationale": "The fix enforces an existing observation requirement rather than rounding failed values or relaxing a gate. Exact reverse-adaptation tests preserve the frozen body, deterministic clock tests cover early/overshooting/error paths, and the same installed candidate passes the full native paired protocol in all three engines. Performance review signals remain separate."
}
]
}
+21 -1
View File
@@ -4733,7 +4733,8 @@
"MOD-01",
"MOD-02",
"MOD-03",
"CORE-25"
"CORE-25",
"ENG-13"
],
"status": "todo",
"risk": "H",
@@ -5296,6 +5297,25 @@
"baselines/jassub-render-validation.json",
"baselines/jassub-render-patch.json"
]
},
{
"id": "ENG-13",
"phase": "2 工程基础",
"title": "保证性能资源探针的最小观察时长",
"scope": [
"workspace"
],
"dependsOn": [
"ENG-08"
],
"status": "done",
"risk": "M",
"deliverable": "性能页面按实测时钟补足既有350ms观察窗口,保留冻结基线和原校验门槛;时钟边界测试和实际安装包配对证据",
"acceptance": "349ms仍被拒绝;早到定时器补足观察、正常超时保留实际值;冻结计时样本逻辑未变,三浏览器实际350ms观察及资源校验通过",
"evidence": [
"changes/2026-09-14-ENG-13-observation-window.md",
"baselines/observation-window-validation.json"
]
}
]
}
+13 -3
View File
@@ -2,7 +2,7 @@ import assert from 'node:assert/strict'
import fs from 'node:fs'
import { fileURLToPath } from 'node:url'
// Reuse the frozen BASE-06 measurement body; only adapt transport and bundle URLs.
// Keep BASE-06 timing samples frozen; adapt delivery and enforce its actual observation minimum.
const directory = fileURLToPath(new URL('../refactor/fixtures/', import.meta.url))
function replaceOnce(source, before, after) {
assert.equal(source.split(before).length, 2, `Performance fixture adapter drift: ${before}`)
@@ -10,8 +10,18 @@ function replaceOnce(source, before, after) {
}
export function performanceScript() {
const source = fs.readFileSync(`${directory}/performance.js`, 'utf8')
return replaceOnce(source, 'const response = await fetch(\'/reports/performance\', { method: \'POST\', headers: { \'Content-Type\': \'application/json\' }, body: JSON.stringify(report) })', 'window.artplayerPerformanceReport = report; const response = { ok: true }')
let source = fs.readFileSync(`${directory}/performance.js`, 'utf8')
source = replaceOnce(source, 'await probe.wait(350)', 'await waitForObservation(probe.wait, now, waitStarted, 350)')
source = replaceOnce(source, 'const response = await fetch(\'/reports/performance\', { method: \'POST\', headers: { \'Content-Type\': \'application/json\' }, body: JSON.stringify(report) })', 'window.artplayerPerformanceReport = report; const response = { ok: true }')
return `${waitForObservation.toString()}\n${source}`
}
export async function waitForObservation(wait, now, started, minimumMs) {
let remaining = minimumMs - (now() - started)
while (remaining > 0) {
await wait(remaining)
remaining = minimumMs - (now() - started)
}
}
export function performanceHtml(variant) {
+38 -3
View File
@@ -7,7 +7,7 @@ import { test } from 'node:test'
import { fileURLToPath } from 'node:url'
import { hash } from '../refactor/scripts/releases.mjs'
import { verifyPerformanceArtifacts } from '../scripts/performance-artifacts.mjs'
import { performanceHtml, performanceScript } from '../scripts/performance-fixture.mjs'
import { performanceHtml, performanceScript, waitForObservation } from '../scripts/performance-fixture.mjs'
import { validatePairedPerformance } from '../scripts/performance-report.mjs'
const baseline = JSON.parse(fs.readFileSync(new URL('../refactor/baselines/performance.json', import.meta.url), 'utf8')).runs[0]
@@ -44,6 +44,7 @@ test('paired performance requires comparable cohorts, real playback and strict c
report => report.runs[1].measurements.resources[0].after.timers.push({ kind: 'interval' }),
report => report.runs[1].measurements.resources[0].after.lateCallbacks.push({ kind: 'timeout' }),
report => report.runs[1].measurements.resources[0].after.lateResizeEvents++,
report => report.runs[1].measurements.resources[0].after.observedMs = 349,
]) {
const report = fixture()
mutate(report)
@@ -60,17 +61,51 @@ test('paired performance requires comparable cohorts, real playback and strict c
assert(comparison.reviewRequired && comparison.groups.every(group => group.reviewSignals.some(signal => signal.metric === 'constructorMs')))
})
test('performance adapter changes only delivery while retaining the frozen measurement procedure', () => {
test('performance adapter preserves frozen measurements except delivery and enforcing the observation minimum', () => {
const frozen = fs.readFileSync(new URL('../refactor/fixtures/performance.js', import.meta.url), 'utf8')
const adapted = performanceScript()
const before = 'const response = await fetch(\'/reports/performance\', { method: \'POST\', headers: { \'Content-Type\': \'application/json\' }, body: JSON.stringify(report) })'
const after = 'window.artplayerPerformanceReport = report; const response = { ok: true }'
assert.equal(adapted.replace(after, before), frozen)
const prefix = `${waitForObservation.toString()}\n`
assert(adapted.startsWith(prefix))
assert.equal(adapted.slice(prefix.length).replace(after, before).replace('await waitForObservation(probe.wait, now, waitStarted, 350)', 'await probe.wait(350)'), frozen)
assert.match(performanceHtml('candidate'), /src="\/candidate\/artplayer.js"/)
assert.match(performanceHtml('published'), /src="\/published\/artplayer-plugin-chapter.js"/)
assert.throws(() => performanceHtml('../uncompiled'))
})
test('resource observation re-arms after an early timer without rounding up evidence', async () => {
let clock = 1000
const delays = []
await waitForObservation(async (delay) => {
delays.push(delay)
clock += delays.length === 1 ? 349 : 1
}, () => clock, 1000, 350)
assert.deepEqual(delays, [350, 1])
assert.equal(clock - 1000, 350)
})
test('resource observation retains overshoot and does not wait again after the minimum', async () => {
let clock = 1000
const delays = []
const wait = async (delay) => {
delays.push(delay)
clock += 364
}
await waitForObservation(wait, () => clock, 1000, 350)
assert.deepEqual(delays, [350])
assert.equal(clock - 1000, 364)
await waitForObservation(wait, () => clock, 1000, 350)
assert.deepEqual(delays, [350])
})
test('resource observation propagates timer failures rather than certifying an incomplete window', async () => {
const failure = new Error('timer unavailable')
await assert.rejects(waitForObservation(async () => {
throw failure
}, () => 0, 0, 350), error => error === failure)
})
test('performance refuses stale sources, build tools and altered installed bundles', () => {
const cache = fileURLToPath(new URL('../refactor/.cache/performance/', import.meta.url))
fs.mkdirSync(cache, { recursive: true })
+12 -2
View File
@@ -9,8 +9,9 @@ worker, no trace/video recording and the same media and viewport within a pair.
Each engine runs three alternating published/candidate pairs. Each side retains
one warm-up and five timed samples for both core and core + chapter, followed by
separate instrumented resource probes. `scripts/performance-fixture.mjs` adapts
only the frozen BASE-06 fixture's script URLs and report transport. Its original
files remain unchanged and their existing baseline tests still run.
the frozen BASE-06 fixture's script URLs, report transport and enforcement of its
minimum observation window. The frozen files and all timed samples remain
unchanged, and their existing baseline tests still run.
The candidate must come from an isolated package installation. Artifact hashes,
source snapshots, package inputs and build tools are checked before measurement;
@@ -28,3 +29,12 @@ A successful test proves the measurement contract and candidate resource cleanup
approval. CORE-22 and release reviews must resolve the corresponding risks before
publishing. These counters do not measure native heap/GPU memory, real mobile
hardware or screen-reader behavior. See `refactor/coverage-performance.md`.
## Observation window
The current performance fixture adapter enforces the existing 350 ms teardown
observation using the measured clock. It re-arms the probe's original timer if
the nominal delay returns before that minimum, without rounding up reported
time or relaxing validation. This applies equally to both variants and leaves
the frozen BASE-06 fixture and all timed samples unchanged. See
`test/performance-report.test.js` and `refactor/changes/2026-09-14-ENG-13-observation-window.md`.