Update package.json

Remove options section from README for clarity

Add 'waiting' event emission during seeking in mediabunnyEngine

This change emits a 'waiting' event when the seeking process begins,
improving the event handling and allowing for better synchronization
during media playback.

fix(audioEngine): adjust audio context suspension timing for improved playback responsiveness

refactor(mediabunnyVideoShim): remove redundant width and height getters; enhance poster setter and add setAttribute method

feat: initialize artplayer-proxy-mediabunny with essential files and structure

feat: implement audio and video engines for mediabunny integration

- Added audioEngine.js to handle audio playback using MediaBunny.
- Created mediabunnyEngine.js to manage both audio and video playback.
- Implemented videoEngine.js for rendering video on a canvas.
- Introduced canvasProxy.js to create a canvas element and manage its methods.
- Developed mediabunnyVideoShim.js to bridge ArtPlayer and MediaBunny functionalities.
- Added videoEvents.js for custom event handling related to video playback.
- Included a utility function clamp.js for value clamping.

feat: add playback rate control to audio and video engines

- Introduced playback rate management in the audio engine, allowing dynamic adjustment of playback speed.
- Updated the mediabunny engine to expose the new setPlaybackRate method.
- Enhanced the mediabunny video shim to support playback rate adjustments, including getter and setter for playbackRate.
- Added event emission for rate changes to ensure proper handling in the application.

Enhance audio engine and video shim functionality

- Added a check to ensure the audio context is initialized before playing audio in the audio engine.
- Introduced a new 'playing' getter in the video shim to accurately reflect the playback state, indicating when the video is neither paused nor ended.

Refactor audio context initialization to handle errors and remove async/await

- Changed `ensureAudioContext` from async to synchronous to simplify error handling.
- Added a try-catch block to handle potential errors when creating the AudioContext with a specified sample rate.
- Updated calls to `ensureAudioContext` to remove unnecessary await statements.

Enhance Mediabunny Engine and Video Shim with State Management and Error Handling

- Added readyState, networkState, and error properties to the Mediabunny Engine for better state management.
- Implemented error handling during media loading, emitting appropriate events on error.
- Updated the Video Engine to emit a 'timeupdate' event for better synchronization.
- Extended the Mediabunny Video Shim to expose new properties (readyState, networkState, error) and methods for improved functionality.
- Added seekable property to the Video Shim to provide seekable range based on media duration.
- Implemented additional properties and methods in the Video Shim for autoplay, loop, controls, currentSrc, poster, preload, defaultMuted, and defaultPlaybackRate.
- Enhanced requestVideoFrameCallback and cancelVideoFrameCallback methods for better frame handling.

refactor: streamline audio and video engine implementations

- Updated audioEngine.js to improve readability and maintainability by using consistent formatting and simplifying function structures.
- Refactored mediabunnyEngine.js to enhance clarity and ensure consistent coding style.
- Enhanced videoEngine.js with improved async handling and clearer function definitions.
- Modified canvasProxy.js to maintain consistent formatting and improve readability.
- Updated mediabunnyVideoShim.js to streamline event handling and improve clarity.
- Refactored videoEvents.js for better readability and consistency in event listener management.
- Simplified clamp.js for improved readability.

feat: enhance audio and video loading with metadata callbacks

- Updated audioEngine to accept an optional onMetadata callback in the load function, which is triggered upon loading completion.
- Modified mediabunnyEngine to handle video and audio loading concurrently, emitting loadedmetadata and durationchange events only when both are ready.
- Enhanced videoEngine to include an onMetadata callback in the load function, ensuring metadata events are emitted appropriately.
- Implemented a throttling mechanism for the timeupdate event in videoEngine to reduce event emissions and improve performance.

feat: add seeking state and played property to mediabunny engine and video shim

- Introduced a `_seeking` state in the mediabunny engine to track seek operations.
- Emitted 'seeking' and 'seeked' events during seek operations.
- Added a `seeking` getter to expose the seeking state.
- Implemented a `played` property in the video shim to provide playback range information.
- Added `width` and `height` getters to the video shim for better video dimension access.
- Enhanced cross-origin handling in the video shim.

feat(audioEngine): enhance audio playback management with starvation detection

- Added an event emitter to notify when the audio context is waiting.
- Implemented a starvation check that suspends the audio context if playback is delayed.
- Introduced a mechanism to track the latest scheduled end time of audio buffers.
- Updated the handling of audio buffer playback to ensure proper state transitions.

refactor: remove unnecessary comments and streamline code in audio, video, and mediabunny engines

feat: add resize functionality to canvas proxy and update type definitions

- Implemented a resize function in canvasProxy.js to adjust canvas size based on player dimensions and video metadata.
- Updated TypeScript definitions for artplayer-proxy-mediabunny to include option and result types.
- Modified build-ts script to include all plugin type definitions.
- Added TypeScript definitions for artplayer-proxy-canvas and artplayer-proxy-mediabunny.

refactor: remove unused properties from Artplayer instance and make option parameter optional in artplayerProxyCanvas type definition

Enhance Mediabunny and Video Engine functionality

- Added event listener for 'ended' event in mediabunnyEngine to handle playback state.
- Modified seek function to check if playback should resume after seeking.
- Reset canvas dimensions to zero in videoEngine when video track is not available or cannot be decoded.
- Ensured canvas is cleared when video track is invalid to prevent rendering issues.

Emit 'waiting' event during loadstart in mediabunnyEngine

Refactor event emission in mediabunnyEngine and add sleep utility function

- Changed the event emission for 'loadstart' to be asynchronous using setTimeout.
- Introduced a new utility function 'sleep' to facilitate delayed execution in promises.

feat: enhance mediabunny proxy with new options and source handling

- Added options for event delay, load timeout, timeupdate interval, AV sync tolerance, poster, and source in README.
- Implemented source normalization in audio and video engines to support URL, Blob, and ReadableStream.
- Updated mediabunny engine to handle new options and improve load handling with timeout and error management.
- Enhanced video engine to draw a poster when no video source is available and manage playback rate.
- Modified shim to accept source option and integrate with the new engine options.
- Updated TypeScript definitions to reflect new options and source types.

feat: add dropLateFrames option to video and mediabunny engines

- Introduced dropLateFrames option in mediabunnyEngine and videoEngine to control frame dropping behavior.
- Updated videoEngine logic to handle frame rendering based on the dropLateFrames setting.
- Added type definition for dropLateFrames in TypeScript declaration file.

fix: delay loadstart event emission to ensure proper event handling

The 'loadstart' event emission is now wrapped in a setTimeout to ensure it is triggered after the current call stack is cleared. This change aims to improve the timing of event emissions and prevent potential issues with event handling in the mediabunny engine.

feat: add preflight range check and drop late frames option

- Updated README.md to include new options: dropLateFrames and preflightRange.
- Enhanced mediabunnyEngine to support new options with default values.
- Implemented preflight function in videoEngine to check server support for Range requests.
- Adjusted videoEngine to utilize preflight check before loading media.
- Updated TypeScript definitions to include preflightRange option.

Remove aspectRatio option from Artplayer configuration

Update README and mediabunny example URL for consistency

Refactor code structure for improved readability and maintainability

Update dependencies and fix play/seek state management in mediabunnyEngine
This commit is contained in:
Harvey Zhao committed 2025-12-13 13:49:32 +08:00
1 parent 2f42f85f90
commit 529ade25d5
132 files changed
+29076 -2258

No files matched your search

@@ -0,0 +1,316 @@
import {
ALL_FORMATS,
AudioBufferSink,
BlobSource,
Input,
ReadableStreamSource,
UrlSource,
} from 'mediabunny'
export default function createAudioEngine(events) {
let input = null
let audioSink = null
let audioContext = null
let gainNode = null
let audioIterator = null
let audioContextStartTime = 0
let playbackTimeAtStart = 0
let latestScheduledEndTime = 0
let _duration = Number.NaN
let _paused = true
let _volume = 0.7
let _muted = false
let _playbackRate = 1
let asyncId = 0
const queuedNodes = new Set()
function getCurrentTime() {
if (_paused)
return playbackTimeAtStart
return (
(audioContext.currentTime - audioContextStartTime) * _playbackRate
+ playbackTimeAtStart
)
}
function updateGain() {
if (!gainNode)
return
const v = _muted ? 0 : _volume
gainNode.gain.value = v * v
}
function ensureAudioContext(sampleRate) {
if (audioContext)
return
const AudioContext
= window.AudioContext || window.webkitAudioContext
try {
audioContext = new AudioContext({ sampleRate })
}
catch {
audioContext = new AudioContext()
}
gainNode = audioContext.createGain()
gainNode.connect(audioContext.destination)
updateGain()
}
function stopQueuedNodes() {
for (const node of queuedNodes) {
node.stop()
}
queuedNodes.clear()
}
async function stopIterator() {
await audioIterator?.return()
audioIterator = null
}
function normalizeSource(src) {
if (typeof src === 'string')
return new UrlSource(src)
if (src instanceof Blob)
return new BlobSource(src)
if (typeof ReadableStream !== 'undefined' && src instanceof ReadableStream)
return new ReadableStreamSource(src)
return src
}
async function load(src, onMetadata) {
asyncId++
const id = asyncId
await stopIterator()
stopQueuedNodes()
_paused = true
playbackTimeAtStart = 0
audioContextStartTime = 0
const source = normalizeSource(src)
if (!source)
return
input = new Input({
source,
formats: ALL_FORMATS,
})
_duration = await input.computeDuration()
if (id !== asyncId)
return
const audioTrack = await input.getPrimaryAudioTrack()
if (!audioTrack) {
audioSink = null
ensureAudioContext()
if (onMetadata)
onMetadata()
return
}
if (audioTrack.codec === null || !(await audioTrack.canDecode())) {
audioSink = null
ensureAudioContext()
if (onMetadata)
onMetadata()
return
}
ensureAudioContext(audioTrack.sampleRate)
audioSink = new AudioBufferSink(audioTrack)
if (onMetadata)
onMetadata()
}
async function runIterator(localId) {
if (!audioSink)
return
await stopIterator()
audioIterator = audioSink.buffers(getCurrentTime())
while (true) {
if (localId !== asyncId)
return
if (_paused)
return
const nextPromise = audioIterator.next()
const checkStarvation = setInterval(() => {
if (localId !== asyncId || _paused) {
clearInterval(checkStarvation)
return
}
if (audioContext.state === 'running'
&& audioContext.currentTime >= latestScheduledEndTime - 0.2) {
audioContext.suspend()
events.emit('waiting')
}
}, 50)
let result
try {
result = await nextPromise
}
catch (e) {
console.error(e)
break
}
finally {
clearInterval(checkStarvation)
}
if (localId !== asyncId)
return
if (_paused)
return
if (audioContext.state === 'suspended') {
await audioContext.resume()
events.emit('canplay')
events.emit('playing')
}
if (result.done)
break
const { buffer, timestamp } = result.value
const node = audioContext.createBufferSource()
node.buffer = buffer
node.connect(gainNode)
node.playbackRate.value = _playbackRate
const startAt
= audioContextStartTime
+ (timestamp - playbackTimeAtStart) / _playbackRate
const duration = buffer.duration
const endAt = startAt + duration / _playbackRate
if (endAt > latestScheduledEndTime) {
latestScheduledEndTime = endAt
}
if (startAt >= audioContext.currentTime) {
node.start(startAt)
}
else {
node.start(
audioContext.currentTime,
(audioContext.currentTime - startAt) * _playbackRate,
)
}
queuedNodes.add(node)
node.onended = () => queuedNodes.delete(node)
}
}
async function play() {
if (!_paused)
return
if (!audioContext) {
ensureAudioContext()
}
if (audioContext.state === 'suspended') {
await audioContext.resume()
}
audioContextStartTime = audioContext.currentTime
latestScheduledEndTime = audioContextStartTime
_paused = false
asyncId++
runIterator(asyncId)
}
function pause() {
if (_paused)
return
playbackTimeAtStart = getCurrentTime()
_paused = true
stopIterator()
stopQueuedNodes()
}
async function seek(time) {
playbackTimeAtStart = Math.max(0, time)
audioContextStartTime = audioContext.currentTime
latestScheduledEndTime = audioContextStartTime
asyncId++
if (!_paused) {
runIterator(asyncId)
}
}
function destroy() {
asyncId++
pause()
audioContext?.close()
audioContext = null
input = null
audioSink = null
}
return {
load,
play,
pause,
seek,
destroy,
setVolume(v, muted) {
_volume = v
_muted = muted
updateGain()
},
setPlaybackRate(rate) {
if (rate === _playbackRate)
return
if (!_paused) {
playbackTimeAtStart = getCurrentTime()
audioContextStartTime = audioContext.currentTime
}
_playbackRate = rate
if (!_paused) {
asyncId++
runIterator(asyncId)
}
},
get currentTime() {
return getCurrentTime()
},
get duration() {
return _duration
},
get paused() {
return _paused
},
}
}
@@ -0,0 +1,220 @@
import createAudioEngine from './audioEngine.js'
import createVideoEngine from './videoEngine.js'
export default function createMediabunnyEngine({
canvas,
ctx,
events,
option = {},
}) {
const defaults = {
timeupdateInterval: 250,
avSyncTolerance: 0.12,
loadTimeout: 12000,
dropLateFrames: false,
poster: '',
}
const audio = createAudioEngine(events)
const video = createVideoEngine({
canvas,
ctx,
events,
timeupdateInterval: option.timeupdateInterval ?? defaults.timeupdateInterval,
avSyncTolerance: option.avSyncTolerance ?? defaults.avSyncTolerance,
dropLateFrames: option.dropLateFrames ?? defaults.dropLateFrames,
poster: option.poster ?? defaults.poster,
preflightRange: option.preflightRange ?? false,
})
let _paused = true
let _ended = false
let _readyState = 0
let _networkState = 0
let _error = null
let _seeking = false
let loadSeq = 0
events.addEventListener?.('ended', () => {
_ended = true
_paused = true
})
async function load(src) {
const id = ++loadSeq
pause()
_ended = false
_error = null
_networkState = 2
_readyState = 0
setTimeout(() => events.emit('waiting'), 0)
setTimeout(() => events.emit('loadstart'), 0)
const loadTimeout = Number.isFinite(option.loadTimeout)
? option.loadTimeout
: defaults.loadTimeout
let timer = null
const timeoutPromise
= loadTimeout > 0
? new Promise((_, reject) => {
timer = setTimeout(() => reject(new Error('Load timeout')), loadTimeout)
})
: null
const loadTask = (async () => {
try {
let videoMetadataLoaded = false
let audioMetadataLoaded = false
const checkMetadata = () => {
if (videoMetadataLoaded && audioMetadataLoaded) {
_readyState = 1
events.emit('loadedmetadata')
events.emit('durationchange')
}
}
const p1 = video.load(src, () => {
if (id !== loadSeq)
return
videoMetadataLoaded = true
checkMetadata()
})
const p2 = audio.load(src, () => {
if (id !== loadSeq)
return
audioMetadataLoaded = true
checkMetadata()
})
await Promise.all([p1, p2])
if (id !== loadSeq)
return
_readyState = 4
_networkState = 1
events.emit('loadeddata')
events.emit('canplay')
events.emit('canplaythrough')
}
catch (err) {
if (id !== loadSeq)
return
_error = { code: 4, message: err.message }
_networkState = 3
events.emit('error')
console.error(err)
}
finally {
if (timer)
clearTimeout(timer)
}
})()
if (timeoutPromise) {
try {
await Promise.race([loadTask, timeoutPromise])
}
catch (err) {
if (id !== loadSeq)
return
loadSeq++
_error = { code: 4, message: err.message }
_networkState = 3
events.emit('error')
}
}
else {
await loadTask
}
}
async function play() {
if (!_paused)
return
_ended = false
_paused = false
await audio.play()
video.start(audio)
events.emit('play')
events.emit('playing')
}
function pause() {
if (_paused)
return
_paused = true
audio.pause()
video.stop()
events.emit('pause')
}
async function seek(time) {
const shouldResume = !_paused
_ended = false
_seeking = true
events.emit('seeking')
events.emit('waiting')
pause()
await audio.seek(time)
await video.seek(time)
_seeking = false
events.emit('seeked')
if (shouldResume && !_ended) {
await play()
}
}
function destroy() {
pause()
audio.destroy()
video.destroy()
}
return {
load,
play,
pause,
seek,
destroy,
setVolume: audio.setVolume,
setPlaybackRate(rate) {
audio.setPlaybackRate(rate)
video.setPlaybackRate(rate)
},
get currentTime() {
return audio.currentTime
},
get duration() {
return audio.duration || video.duration
},
get paused() {
return _paused
},
get seeking() {
return _seeking
},
get ended() {
return _ended
},
get readyState() {
return _readyState
},
get networkState() {
return _networkState
},
get error() {
return _error
},
get videoWidth() {
return video.width
},
get videoHeight() {
return video.height
},
}
}
@@ -0,0 +1,308 @@
import {
ALL_FORMATS,
BlobSource,
CanvasSink,
Input,
ReadableStreamSource,
UrlSource,
} from 'mediabunny'
export default function createVideoEngine({
canvas,
ctx,
events,
timeupdateInterval = 250,
avSyncTolerance = 0.12,
dropLateFrames = false,
poster = '',
preflightRange = false,
}) {
let input = null
let videoSink = null
let videoIterator = null
let nextFrame = null
let rafId = 0
let asyncId = 0
let _width = 0
let _height = 0
let _duration = Number.NaN
let audioClock = null
let lastTimeUpdate = 0
let stalled = false
let playbackRate = 1
let posterDrawn = false
function normalizeSource(src) {
if (typeof src === 'string')
return new UrlSource(src)
if (src instanceof Blob)
return new BlobSource(src)
if (typeof ReadableStream !== 'undefined' && src instanceof ReadableStream)
return new ReadableStreamSource(src)
return src
}
async function preflight(url) {
if (!preflightRange || typeof url !== 'string')
return true
try {
const res = await fetch(url, { method: 'HEAD' })
const acceptRanges = res.headers.get('accept-ranges')
if (!acceptRanges || acceptRanges === 'none') {
events.emit('error', new Event('RangeNotSupported'))
return false
}
return true
}
catch (e) {
console.warn('preflight failed', e)
return true
}
}
function drawPoster() {
if (!poster || posterDrawn)
return
const img = new Image()
img.onload = () => {
ctx.clearRect(0, 0, canvas.width, canvas.height)
canvas.width = img.naturalWidth || canvas.width
canvas.height = img.naturalHeight || canvas.height
ctx.drawImage(img, 0, 0, canvas.width, canvas.height)
posterDrawn = true
}
img.src = poster
}
async function stopIterator() {
await videoIterator?.return()
videoIterator = null
}
function clear() {
ctx.clearRect(0, 0, canvas.width, canvas.height)
}
async function load(src, onMetadata) {
asyncId++
const id = asyncId
await stopIterator()
clear()
posterDrawn = false
if (!(await preflight(src)))
return
const source = normalizeSource(src)
if (!source) {
drawPoster()
return
}
input = new Input({
source,
formats: ALL_FORMATS,
})
_duration = await input.computeDuration()
if (id !== asyncId)
return
const videoTrack = await input.getPrimaryVideoTrack()
if (!videoTrack) {
videoSink = null
_width = 0
_height = 0
canvas.width = 0
canvas.height = 0
drawPoster()
if (onMetadata)
onMetadata()
return
}
if (videoTrack.codec === null || !(await videoTrack.canDecode())) {
videoSink = null
_width = 0
_height = 0
canvas.width = 0
canvas.height = 0
clear()
drawPoster()
if (onMetadata)
onMetadata()
return
}
const transparent = await videoTrack.canBeTransparent()
videoSink = new CanvasSink(videoTrack, {
poolSize: 2,
fit: 'contain',
alpha: transparent,
})
_width = videoTrack.displayWidth
_height = videoTrack.displayHeight
canvas.width = _width
canvas.height = _height
if (onMetadata)
onMetadata()
await resetIterator(0)
}
async function resetIterator(time) {
await stopIterator()
if (!videoSink)
return
videoIterator = videoSink.canvases(time)
const first = (await videoIterator.next()).value ?? null
const second = (await videoIterator.next()).value ?? null
nextFrame = second
if (first) {
ctx.drawImage(first.canvas, 0, 0)
events.emit('loadeddata')
}
else {
drawPoster()
}
}
async function updateNextFrame(localId) {
if (!videoIterator)
return
while (true) {
const frame = (await videoIterator.next()).value ?? null
if (!frame)
return
if (localId !== asyncId)
return
const t = audioClock.currentTime
if (dropLateFrames) {
const tolerance = Math.max(0.06, avSyncTolerance / Math.max(1, playbackRate))
if (frame.timestamp < t - tolerance) {
continue
}
if (frame.timestamp <= t + tolerance) {
ctx.clearRect(0, 0, canvas.width, canvas.height)
ctx.drawImage(frame.canvas, 0, 0)
}
else {
nextFrame = frame
return
}
}
else {
if (frame.timestamp <= t) {
ctx.clearRect(0, 0, canvas.width, canvas.height)
ctx.drawImage(frame.canvas, 0, 0)
}
else {
nextFrame = frame
return
}
}
}
}
function render() {
if (!audioClock)
return
const t = audioClock.currentTime
const now = Date.now()
if (now - lastTimeUpdate >= timeupdateInterval) {
events.emit('timeupdate')
lastTimeUpdate = now
}
if (Number.isFinite(_duration) && t >= _duration) {
stop()
events.emit('ended')
return
}
if (nextFrame && nextFrame.timestamp <= t) {
ctx.clearRect(0, 0, canvas.width, canvas.height)
ctx.drawImage(nextFrame.canvas, 0, 0)
nextFrame = null
updateNextFrame(asyncId)
if (stalled) {
events.emit('canplay')
events.emit('playing')
stalled = false
}
}
else if (!nextFrame) {
updateNextFrame(asyncId)
if (!nextFrame && Number.isFinite(_duration) && t < _duration && !stalled) {
stalled = true
events.emit('waiting')
}
}
rafId = requestAnimationFrame(render)
}
function start(audioEngine) {
audioClock = audioEngine
asyncId++
stalled = false
updateNextFrame(asyncId)
rafId = requestAnimationFrame(render)
}
function stop() {
cancelAnimationFrame(rafId)
}
async function seek(time) {
asyncId++
await resetIterator(time)
}
function setPlaybackRate(rate) {
playbackRate = Math.max(0.1, Number(rate) || 1)
}
function destroy() {
asyncId++
stop()
stopIterator()
posterDrawn = false
input = null
videoSink = null
}
return {
load,
start,
stop,
seek,
setPlaybackRate,
destroy,
get width() {
return _width
},
get height() {
return _height
},
get duration() {
return _duration
},
}
}