mirror of
https://github.com/zhw2590582/ArtPlayer.git
synced 2026-10-27 01:21:20 -08:00
docs(site): [SITE-04] document PiP and ASR contracts in both languages
This commit is contained in:
1 parent
3138812610
commit
94adb77fd6
97 files changed
+4318
-1817
No files matched your search
+254
@@ -4663,6 +4663,161 @@ const installDefaultLight = ambilight.default({ opacity: 0.5 });
|
||||
The root declarations export the named types `Option`, `Result`, `Callable`, `Factory` and `RuntimeFactory`. Version 1.0.0 used `export =` and required option fields, unlike 1.1.0. Earlier `import = require` callers can move to `/runtime`; reads of optional fields need a fallback. Node10 TypeScript default imports of `/runtime` require `esModuleInterop`; `import = require` works without it.
|
||||
|
||||
|
||||
===== packages/artplayer-vitepress/docs/en/plugin/asr.md =====
|
||||
|
||||
# Audio Capture and Recognition Subtitles
|
||||
|
||||
[简体中文](../../plugin/asr.md)
|
||||
|
||||
Capture audio from the player's video, pass PCM/WAV chunks to your recognition callback, and display the returned subtitles. The plugin includes no recognition model or network service and does not access the microphone. This page describes the unreleased refactor branch; the new type entry and capture option are not claims about the live release.
|
||||
|
||||
## Installation and local example
|
||||
|
||||
```sh
|
||||
yarn add artplayer artplayer-plugin-asr
|
||||
```
|
||||
|
||||
```js
|
||||
import Artplayer from 'artplayer';
|
||||
import artplayerPluginAsr from 'artplayer-plugin-asr';
|
||||
```
|
||||
|
||||
For script usage, load ArtPlayer before `dist/artplayer-plugin-asr.js`. The global is `artplayerPluginAsr`. This preserves the original `asr.local` example: it captures the site's sample video and displays statistics. Subtitles are simulated, audio is not uploaded, and the example does not measure recognition accuracy. Integrate your own recognition service inside the callback.
|
||||
|
||||
<div className="run-code" data-libs="./uncompiled/artplayer-plugin-asr/index.js">▶ Run Code</div>
|
||||
|
||||
```js
|
||||
/* global Artplayer, artplayerPluginAsr */
|
||||
// Local audio capture demo. The subtitles below are simulated, not recognized speech.
|
||||
// No audio is uploaded; only the sample media is loaded from this local site.
|
||||
const statistics = document.createElement('div')
|
||||
statistics.textContent = 'Local ASR demo: press play. No recognition service is used.'
|
||||
let chunks = 0
|
||||
let pcmBytes = 0
|
||||
let wavBytes = 0
|
||||
|
||||
const art = new Artplayer({
|
||||
container: '.artplayer-app',
|
||||
url: '/assets/sample/steve-jobs.mp4',
|
||||
autoSize: true,
|
||||
fullscreen: true,
|
||||
fullscreenWeb: true,
|
||||
layers: [{
|
||||
name: 'asr-local-statistics',
|
||||
html: statistics,
|
||||
style: {
|
||||
position: 'absolute',
|
||||
top: '12px',
|
||||
left: '12px',
|
||||
right: '12px',
|
||||
padding: '8px 12px',
|
||||
background: 'rgba(0, 0, 0, 0.65)',
|
||||
color: '#fff',
|
||||
fontSize: '12px',
|
||||
whiteSpace: 'pre-line',
|
||||
pointerEvents: 'none',
|
||||
},
|
||||
}],
|
||||
controls: [{
|
||||
name: 'asr-local-stop',
|
||||
position: 'right',
|
||||
html: 'Stop ASR',
|
||||
tooltip: 'Stop capture; pause and play to restart',
|
||||
async click() {
|
||||
await art.plugins.artplayerPluginAsr.stop()
|
||||
if (!art.isDestroy)
|
||||
statistics.textContent = 'Local capture stopped. Pause and play to restart. Nothing was uploaded.'
|
||||
},
|
||||
}],
|
||||
plugins: [artplayerPluginAsr({
|
||||
length: 2,
|
||||
interval: 250,
|
||||
sampleRate: 16000,
|
||||
autoHideTimeout: 5000,
|
||||
onAudioChunk({ pcm, wav }) {
|
||||
if (art.isDestroy)
|
||||
return
|
||||
chunks++
|
||||
pcmBytes += pcm.byteLength
|
||||
wavBytes += wav.byteLength
|
||||
const sampleRate = new DataView(wav).getUint32(24, true)
|
||||
const samples = new DataView(pcm)
|
||||
let peak = 0
|
||||
for (let offset = 0; offset < pcm.byteLength; offset += 2)
|
||||
peak = Math.max(peak, Math.abs(samples.getInt16(offset, true)))
|
||||
const duration = (pcmBytes / 2 / sampleRate).toFixed(2)
|
||||
statistics.textContent = [
|
||||
'Local capture only - simulated subtitles, no speech recognition',
|
||||
`Chunks: ${chunks} | ${sampleRate} Hz mono PCM16 | ${duration} seconds captured`,
|
||||
`PCM: ${pcmBytes} bytes | WAV: ${wavBytes} bytes | Current peak: ${peak}`,
|
||||
].join('\n')
|
||||
return `Simulated local subtitle: audio chunk ${chunks} received.`
|
||||
},
|
||||
})],
|
||||
})
|
||||
```
|
||||
|
||||
## Options and audio chunks
|
||||
|
||||
| Field | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `length` | `3` | Number of final nonempty punctuation-separated segments displayed from the current subtitle, not seconds of audio |
|
||||
| `interval` | `100` | Consumption timer interval and target audio duration per chunk, in milliseconds |
|
||||
| `sampleRate` | `16000` | Requested sample rate in Hz; requires environment support |
|
||||
| `autoHideTimeout` | `10000` | Delay before hiding an accepted subtitle, in milliseconds |
|
||||
| `onAudioChunk` | Function returning `null` | Receives `{ pcm, wav }`; a returned string displays subtitles; Promises are supported |
|
||||
| `audioInput` | Unset | Direct Web Audio by default; `{ type: 'capture' }` explicitly selects a captured stream |
|
||||
|
||||
Both `pcm` and `wav` are `ArrayBuffer` values. Audio comes from the first channel. PCM is signed 16-bit little-endian; WAV adds a 44-byte mono header. A chunk contains `Math.floor(sampleRate * interval / 1000)` samples, requiring at least one sample and a positive, finite interval. Audio availability, browser scheduling and recognition latency affect callback frequency; exact timing is not guaranteed.
|
||||
|
||||
At most one recognition callback is pending within the active capture generation. Partial chunks stay queued. Rejected callbacks log an error and allow later processing. A backlog exceeding the larger of about one minute of audio or two chunks pauses capture and logs an error to bound memory. Pause, source changes, stop and destroy invalidate old results. They do not cancel network requests already sent by your callback; manage those resources in your application.
|
||||
|
||||
## Subtitles and lifecycle
|
||||
|
||||
Access the result through `art.plugins.artplayerPluginAsr`; its `name` is always `artplayerPluginAsr`:
|
||||
|
||||
| Method | Behavior |
|
||||
| --- | --- |
|
||||
| `append(text)` | Returns `undefined` synchronously; displays the final `length` segments of this text and resets the hide timer, replacing earlier subtitles rather than accumulating history |
|
||||
| `hide()` | Returns `undefined` synchronously; hides subtitles without stopping capture or clearing their text |
|
||||
| `stop()` | Actually returns `Promise<void>`; stops the current capture and discards old results; a later play event can restart it |
|
||||
|
||||
Both `append` and returned strings preserve historical HTML rendering and are not automatically escaped. Supply trusted subtitles; escape or sanitize external recognition text in your application. `null`, `undefined` and other non-string callback results do not update subtitles. `stop()` does not immediately hide existing subtitles; their original auto-hide deadline remains active.
|
||||
|
||||
Play starts capture; pause stops recording and discards obsolete work. Source changes rebuild the applicable capture state, and media errors use a nonterminal stop. The default direct connection retains the video's audio output route until `art.destroy()`, allowing playback after stop. Destroy releases subscriptions, timers, recording resources and audio contexts. There is no separate public plugin `start()` or `destroy()`.
|
||||
|
||||
## Audio routing and cross-origin media
|
||||
|
||||
The default route reads `art.video`, not the independent Audio Track plugin's audio. The video's own volume and mute affect captured data. Cross-origin media requires `moreVideoAttr: { crossOrigin: 'anonymous' }` before loading and appropriate server CORS headers. Playable media is not necessarily readable by Web Audio: without access, the clock can advance while output is silent, samples are zero or no chunks arrive. Cross-origin redirects are also restricted.
|
||||
|
||||
If your application already owns a Web Audio graph, explicitly choose `audioInput: { type: 'capture' }`. It uses `captureStream/mozCaptureStream`, does not take over the existing output, and never falls back to a direct connection on failure. It releases only its own contexts and captured tracks. Pause retains this graph; stop or source changes release it, and later playback obtains a fresh stream. Unsupported capability or initialization failure is logged and resources are cleaned up.
|
||||
|
||||
Captured data can remain nonzero despite video volume or mute and does not include an external effects graph or independent audio mix. CORS still applies: capture may be rejected, or a track may exist without readable audio. Desktop results do not establish Safari, mobile, every proxy or physical speaker behavior.
|
||||
|
||||
## TypeScript entries
|
||||
|
||||
The root preserves historical `AsrPluginOption`, `AsrPluginInstance` and `AudioChunk` types: callbacks return `void | Promise<void>`, and stop returns void. To type returned recognition strings, await stop or select captured input, use `/runtime`, which points to the same implementation:
|
||||
|
||||
```ts
|
||||
import asr from 'artplayer-plugin-asr/runtime';
|
||||
import type { RuntimeResult } from 'artplayer-plugin-asr/runtime';
|
||||
|
||||
const installAsr = asr({
|
||||
audioInput: { type: 'capture' },
|
||||
onAudioChunk({ pcm, wav }) {
|
||||
console.log(pcm.byteLength, wav.byteLength);
|
||||
return null; // Replace with your recognizer; null displays no subtitle.
|
||||
},
|
||||
});
|
||||
|
||||
async function stopAsr(plugin: RuntimeResult): Promise<void> {
|
||||
await plugin.stop();
|
||||
}
|
||||
```
|
||||
|
||||
Precise types are `RuntimeOption`, `RuntimeResult` and `RuntimeFactory`, with `AudioChunk` also exported. Some NodeNext ESM consumers retain the old root namespace shape; choose `/runtime` for a callable default import. CommonJS runtime supports the function itself and its `.default` self-alias; `/runtime` also supports TypeScript `import = require`.
|
||||
|
||||
|
||||
===== packages/artplayer-vitepress/docs/en/plugin/audio-track.md =====
|
||||
|
||||
# Audio Track
|
||||
@@ -5389,6 +5544,105 @@ const plugin = dashControl<Level, Track>({
|
||||
The refactor tests fixed dash.js 4.5.2 and 5.2.1 with local adaptive media and old/new core combinations. Those are tested points, not a new blanket support range. The plugin includes a targeted 4.5.2 paused-seek recovery for stale empty-buffer metrics while preserving the caller's SDK settings and media time. Windows Playwright WebKit lacks the MSE path used by these tests; it is not Safari/device playback acceptance. Test your own MPDs, DRM and target devices before adopting the unpublished refactor.
|
||||
|
||||
|
||||
===== packages/artplayer-vitepress/docs/en/plugin/document-pip.md =====
|
||||
|
||||
# Document Picture-in-Picture
|
||||
|
||||
[简体中文](../../plugin/document-pip.md)
|
||||
|
||||
Move the whole player, including its controls, into a browser Document Picture-in-Picture window. Closing the window restores the same player node to its original position. This page describes the unreleased refactor branch; the online example and unpinned npm/CDN packages are not the current candidate.
|
||||
|
||||
## Installation and example
|
||||
|
||||
```sh
|
||||
yarn add artplayer artplayer-plugin-document-pip
|
||||
```
|
||||
|
||||
```js
|
||||
import Artplayer from 'artplayer';
|
||||
import artplayerPluginDocumentPip from 'artplayer-plugin-document-pip';
|
||||
```
|
||||
|
||||
For script usage, load ArtPlayer before `dist/artplayer-plugin-document-pip.js`. The global is `artplayerPluginDocumentPip`. The following code preserves the original [online example](https://artplayer.org/?libs=./uncompiled/artplayer-plugin-document-pip/index.js&example=document.pip).
|
||||
|
||||
<div className="run-code" data-libs="./uncompiled/artplayer-plugin-document-pip/index.js">▶ Run Code</div>
|
||||
|
||||
```js
|
||||
// npm i artplayer-plugin-document-pip
|
||||
// import artplayerPluginDocumentPip from 'artplayer-plugin-document-pip';
|
||||
|
||||
const art = new Artplayer({
|
||||
container: '.artplayer-app',
|
||||
url: '/assets/sample/video.mp4',
|
||||
plugins: [
|
||||
artplayerPluginDocumentPip({
|
||||
width: 480,
|
||||
height: 270,
|
||||
fallbackToVideoPiP: true,
|
||||
placeholder: `Playing in Document Picture-in-Picture`,
|
||||
}),
|
||||
],
|
||||
})
|
||||
|
||||
art.on('document-pip', (state) => {
|
||||
console.log('Document Picture-in-Picture', state)
|
||||
})
|
||||
```
|
||||
|
||||
## Options
|
||||
|
||||
| Field | Type | Default | Meaning |
|
||||
| --- | --- | --- | --- |
|
||||
| `width` | `number` | `480` | Requested window width; the browser determines the actual size |
|
||||
| `height` | `number` | `270` | Requested window height |
|
||||
| `placeholder` | `string` | `'Playing in Document Picture-in-Picture'` | Placeholder text at the original player position |
|
||||
| `fallbackToVideoPiP` | `boolean` | `true` | Try setting `art.pip = true` when the Document PiP API is absent |
|
||||
|
||||
The plugin registers as `artplayerPluginDocumentPip` and adds a PiP control button. It detects `documentPictureInPicture.requestWindow`; a positive result does not guarantee that permissions, the calling context or user activation will allow opening. Call `open()` or `toggle()` directly from a user click handler, before awaiting unrelated work.
|
||||
|
||||
Video PiP fallback also depends on the browser and media. It does not move the whole player, set the plugin's `isActive` flag or emit a Document PiP activation event. `close()` manages the Document PiP window; use `art.pip = false` to exit video PiP. Disabling fallback does not add Document PiP support to the browser.
|
||||
|
||||
## State, methods and events
|
||||
|
||||
Access the result through `art.plugins.artplayerPluginDocumentPip`:
|
||||
|
||||
| Member | Runtime behavior |
|
||||
| --- | --- |
|
||||
| `name` | Always `artplayerPluginDocumentPip` |
|
||||
| `isSupported` | Readonly getter; Document PiP API capability snapshot taken at plugin creation |
|
||||
| `isActive` | Readonly getter; whether a Document PiP session is held, not video PiP state |
|
||||
| `open()` | Returns `Promise<void>`; requests a window and moves the player, coalesces pending requests and does nothing when already open |
|
||||
| `close()` | Returns `Promise<void>`; cancels a pending request or restores the node and closes the window |
|
||||
| `toggle()` | Returns `undefined` synchronously; opens or closes the active/pending window |
|
||||
|
||||
Successful activation and normal closure emit the player's `document-pip` event with `true` and `false`, respectively. The plugin updates the existing `artplayer-document-pip` class, rebinds document events and schedules resize. This event does not signal successful media loading or playback.
|
||||
|
||||
Normal window request/restoration failures display a notice and console warning; awaiting `open()` alone is not proof that a window opened. An error thrown by the video PiP fallback setter can still reject its Promise. A window arriving after cancellation is closed without adopting the player. Destroying the player releases the window, control, subscriptions and timers, suppressing further plugin state events. There is no separate public plugin `destroy()`.
|
||||
|
||||
Styles are copied from the player's document on a best-effort basis. Inaccessible cross-origin styles and externally managed DOM need validation in your environment. Canvas, other proxies, keyboard focus and continuous playback also need their own checks; the capability flag cannot establish those results.
|
||||
|
||||
## Compatible TypeScript views
|
||||
|
||||
The root and `/legacy` declarations preserve the old required options object, writable `Result` flags and void actions. JavaScript permits omitted options, the runtime flags are readonly getters, and `open/close` return Promises. The old shape preserves consumer and replacement-function compatibility; it does not make the runtime getters writable.
|
||||
|
||||
This package has no `/runtime` subpath. For accurate types, explicitly view the real factory as `RuntimeFactory`:
|
||||
|
||||
```ts
|
||||
import documentPip from 'artplayer-plugin-document-pip';
|
||||
import type { AsyncResult, RuntimeFactory } from 'artplayer-plugin-document-pip';
|
||||
|
||||
const runtimeFactory = documentPip as RuntimeFactory;
|
||||
const installPip = runtimeFactory();
|
||||
const installDefaultPip = runtimeFactory.default({ width: 480 });
|
||||
|
||||
async function closePip(pip: AsyncResult): Promise<void> {
|
||||
await pip.close();
|
||||
}
|
||||
```
|
||||
|
||||
Named types are `Option`, `Result`, `AsyncResult`, `Factory` and `RuntimeFactory`. Apply the precise view only to the unmodified implementation, not a void-returning mock or replaced method. CommonJS runtime supports both `require(package)(options)` and `.default(options)`. Historical `import = require` types use `.default`; use the precise factory view when direct calling is needed.
|
||||
|
||||
|
||||
===== packages/artplayer-vitepress/docs/en/plugin/hls-control.md =====
|
||||
|
||||
# HLS Control
|
||||
|
||||
Reference in new issue
Block a user