ArtPlayer/packages/artplayer-plugin-asr/ARCHITECTURE.md

# ASR maintenance

The player captures first-channel audio and hands PCM/WAV to a caller-provided
recognizer. There is no network request, microphone permission or embedded model.

| File | Responsibility |
| --- | --- |
| src/index.ts | Snapshot options, register player events, return the compatible facade and install the stylesheet |
| src/types.ts | Internal options, result and browser capability types; public declarations remain in types/ |
| src/subtitles.ts | Punctuation/HTML rendering and the owned auto-hide timer |
| src/capture.ts | Initialization deduplication, active generation, FIFO consumption and one pending recognizer call |
| src/audio-graph.ts | AudioContext, element/stream source, Worklet loading, playback routing and recorder ownership |
| src/sample-queue.ts | Copied sample blocks with a linked queue and head offset; preserves partial blocks and tails |
| src/encoding.ts | Pure PCM16LE/WAV encoding, including historical clamping and the 44-byte mono WAV header |
| src/worklet.ts | Inline processor code; first-channel messages and the existing recorder-processor registration name |
| src/style.less | Existing art-layer-asr layout, art-asr-line content and player CSS variables |

Dependencies flow from the facade into subtitles/capture, then from capture into
the graph, queue and pure encoder. Audio modules do not receive the whole player.
The Worklet is loaded from a temporary Blob before binding the media element, so
a failed module load can close its context without taking over video playback.

## States and ownership

Play shares one pending initialization. Successful setup attaches one recorder
and starts one interval. Each tick takes exactly one complete block; an incomplete
block stays queued. Only one callback is pending within the active generation.
Rejections are logged and later ticks continue. Pause, source restart, stop and
destroy invalidate old recognizer completions and discard obsolete queued audio.
Each recorder also checks its own identity, rejecting messages from retired ports.

Pause detaches the recorder and cancels the interval. The direct media source
continues through its playback gain to the destination. A source switch reuses the
direct source and replaces the recorder; a captured-stream fallback is closed and
reacquired because captured tracks can end on source changes.

The playback gain stays at unity. Native media-element source output already
reflects the video's volume and mute; applying the same volume again would turn
50% volume into 25% output amplitude. PCM also follows the main video's volume
and becomes silent when that element is muted. Native analyser tests cover the
linear output ratio, mute, stop and resume on Chromium/Firefox; they do not measure
physical speakers or establish Safari/device support.

`stop()` returns `Promise<void>` and releases ASR work. An already displayed
subtitle retains its original auto-hide deadline; obsolete callbacks cannot add timers.
It retains playback subscriptions and allows explicit append or a later play.
For a directly bound video, the player keeps its original source/context and audio
route until `art.destroy()`. Closing that context at every stop caused Chromium
to lose audio on restart; a media element cannot simply bind a replacement source.
For a captured fallback, stop closes its disposable context and stops its tracks.
Before direct binding, failed/cancelled initialization closes its context and
revokes the Worklet URL. Destroy closes all contexts, disconnects nodes, stops
captured tracks, cancels subtitle timers and removes subscriptions.

This lifetime follows the [Web Audio close semantics](https://www.w3.org/TR/webaudio-1.0/#dom-audiocontext-close)
and [media-element source behavior](https://www.w3.org/TR/webaudio-1.0/#MediaElementAudioSourceNode).
Captured-stream source changes follow [Media Capture from DOM Elements](https://www.w3.org/TR/mediacapture-fromelement/#html-media-element-media-capture-extensions).

Queued audio is limited to the larger of 60 seconds or two configured chunks. A caller that
cannot keep up gets an explicit console error and capture pauses, leaving normal
playback routed. This avoids an unbounded backlog; it does not claim lossless
recognition under overload. A later play can restart capture. The caller owns
cancellation of its own network requests; obsolete completions cannot repaint.

## Compatibility and verification

The factory still snapshots options and registers synchronously, returning
`name`, `stop`, `hide`, `append`. There is no public start method. Append preserves
HTML, punctuation splitting, replacement, ignored non-string values and the
historical `length=0` behavior. The CJS function has a non-enumerable self `default`
alias for consumers of the 2.0 namespace, while preserving the callable 2.1 form.

Root and `/legacy` declarations intentionally retain the historical factory,
void stop and void/Promise<void> callback. Their NodeNext ESM default import remains
the old namespace type: changing it into a callable default would break legal
namespace substitution. Historical direct-call errors are frozen separately from
legal namespace consumers. Do not silently rewrite these declarations to match
runtime behavior or require a self-default property on old replacement factories.

The additive `/runtime` entry references the same main/ESM implementation, with
`RuntimeOption`, `RuntimeResult` and `RuntimeFactory`: callback text/null/no result
and asynchronous stop are accurate. Its `.d.mts` default export and `.d.cts`/
`.d.ts` export assignment support ESM, raw CommonJS and old Node10 resolution.
Pure type namespaces make all four named runtime types available in every mode.
The root also exports those types for convenient annotations. No runtime code or
version changes are needed for this entry.

From the repository root:

- `yarn test:asr`: frozen public/audio observations, queue/encoding parity and
  candidate lifecycle regressions. Historical defects remain separate tests.
- `yarn test:browser test/browser/asr-audio.spec.js`: local AAC, actual Worklet,
  published/current core, pause/resume, source switch, stop/restart and destroy.
  Set `ARTPLAYER_ASR_ARTIFACT` to an actual main/legacy file to test that artifact.
- `yarn test:browser test/browser/asr-playback.spec.js`: frozen 2.1 output
  attenuation versus candidate linear volume, mute and stop/resume. A native
  unity analyser measures the gain-to-destination route with local audio.
- `yarn typecheck`: includes the package strict/noUncheckedIndexedAccess project.
- `yarn test:asr-types-package`: packs and installs core/ASR outside the workspace,
  checks frozen 2.0/2.1 consumers and candidate declarations in strict compiler
  modes, verifies offline frozen reinstall and file hashes, exercises actual
  import/require exports, and rejects invalid uses. Historical declaration errors
  are explicit expectations and never counted as candidate success.
- `yarn dev artplayer-plugin-asr` and `yarn build artplayer-plugin-asr`: normal
  repository development and three-format production builds.

Windows Playwright WebKit 26.6 lacks AudioContext and AudioWorkletNode. Its native
ASR cases remain explicit skips and do not establish Safari/device compatibility.
Real fallback track replacement, native volume/mute, CORS, unsupported media and
physical devices remain PKG-ASR-05 acceptance work. Do not use the demo's external
recognizer service as a prerequisite for local audio verification.