ArtPlayer/packages/artplayer-plugin-auto-thumbnail/ARCHITECTURE.md

Auto-thumbnail maintenance

This package registers an asynchronous ArtPlayer plugin and generates progressive JPEG thumbnail sheets using a separate media element. Its public result remains { name: 'artplayerPluginAutoThumbnail' }; the registration Promise does not wait for video processing.

Modules and ownership

• src/index.ts owns ArtPlayer subscriptions and the public factory. Options are read at each video:loadedmetadata event, including subsequent changes to the original options object. restart cancels the old decoder immediately. destroy closes the session and removes subscriptions. Partial registration is rolled back. Before accessing host subscriptions, a truthy public isDestroy makes a delayed registrar resolve with its unchanged name result and no resources. The internal host flag is optional, preserving structural hosts without that property. Core plugin-manager rejection policy is unchanged; this protects direct retained registrars such as callbacks delivered after an asynchronous module load.
• src/options.ts preserves truthy defaults and computes the ten-column layout. Coercible numeric JS inputs and fractional frame counts retain their previous behavior; raw option values are forwarded to the thumbnail configuration. Invalid/nonfinite dimensions fail before canvas allocation. Canvas dimensions are checked against integer/IDL bounds, not a universal memory budget.
• src/session.ts owns one active extraction job and the generated object URLs. Guards prevent obsolete callbacks from publishing or scheduling another frame. Cleanup runs all registered actions even if one fails. Installing a replacement before cleaning up its predecessor makes synchronous host reentry observable.
• src/extraction.ts owns the private video and canvas, metadata/seek/error handlers, and the frame/encode sequence. Each valid draw is encoded before the next seek. The final decoder is paused, cleared and reset; the final sheet URL remains owned by the session until another usable sheet replaces it or destroy. Metadata loading has a separate 30-second deadline, started before assigning the URL and canceled when metadata arrives or the job ends. A queued deadline becomes inert after that phase; it cannot cancel frame reading or encoding. A stalled load reports through the existing warning callback and disposes its decoder without replacing the previous usable sheet. Timer-registration reentry cannot restart loading after destruction. This does not time out registration, change the public result, retry the URL, or impose a total extraction deadline. The private canvas is also job-owned: cancellation, failure and normal completion reset both dimensions to zero, even if an encoding callback still retains it. Width and height resets are independent cleanup actions, so one throwing setter cannot prevent the other reset or decoder cleanup. Allocation checks job identity after each dimension write; synchronous destruction cannot reallocate the sheet. Previously encoded JPEG Blobs/URLs retain their independent bytes. This controls canvas dimensions, not the browser's precise GPU/encoder reclamation time.
• src/encoding.ts owns one pending JPEG operation, its30-second deadline and once-only completion. It clears that deadline before publishing or starting the next sample, and ignores callbacks from an earlier operation. One job cleanup handles all encodes rather than adding a cleanup closure for every frame. Timeout disposes the job and retains the last usable preview; the browser's toBlob work itself cannot be canceled. Late Blob delivery cannot publish after timeout, replacement or destruction. Synchronous callbacks and timer-registration reentry also keep timer ownership intact.
• src/video.ts creates and owns the hidden media element. It is attached to the document root because detached media loses drawable pixels on Windows WebKit. visibility:hidden preserves its rendered box; display:none and a 1px box do not. Metadata locks the box to intrinsic dimensions, overriding ordinary global video sizing rules. It is silent, excluded from focus/accessibility, and never played by this implementation. Cleanup also removes the element if insertion, cancellation, pause or decoder reset fails.
• src/frames.ts owns one pending sample, its seek/data handlers, deadline and native presentation callback. When request/cancel frame callbacks are both available it waits for loaded data, then requires both seek completion and frame presentation before drawing. Callback identities are invalidated on retry and replacement; late/duplicate delivery cannot complete a newer sample. One job cleanup handles every frame without accumulating per-frame cleanup closures. It invalidates the pending sample before independently clearing each handler, deadline and frame callback. A throwing event-property setter cannot prevent the remaining releases. The first cleanup error reaches the existing warning path; even a handler that cannot be removed becomes inert after cancellation.
• src/types.ts defines internal configuration, sheet data, the minimal host and guarded extraction-job callbacks. guard retains argument/result types and the inactive undefined result. It shares the opt-in runtime-api.d.ts option type; the entry's Promise result is checked against that public result. No public declarations are generated from this file.

The original types/artplayer-plugin-auto-thumbnail.d.ts remains byte-equivalent to npm 1.1.0 after newline normalization. types/runtime-api.d.ts owns the precise option/result/factory contracts; runtime.d.mts, runtime.d.cts, and runtime.d.ts adapt module resolution without adding a separate runtime implementation. src/index.ts assigns a writable .default reference to the same factory, preserving both 1.0.1 default calls and 1.1.0 direct calls. Factory retains the plain asynchronous callable type; RuntimeFactory includes the recursive alias. TypeScript infers the recursive property from the function assignment; no type assertion is needed. The implementation fixture checks both the plain Factory and the full RuntimeFactory against the actual exported value. The root editor declaration is generated semantically by yarn build:ts; its historical synchronous callable shape is retained, without mixing default and export-assignment syntax. See README for the optional accurate import.

Only the entry module receives ArtPlayer. Extraction receives a guarded job and a configuration snapshot. These internal modules do not add public player fields, plugin methods, or a dependency on a particular core implementation.

Compatibility and intentional fixes

The factory/global name, Promise registration, result name, default width 160, count 100, scale 1, ten columns, aspect-ratio height, and time formula duration * index / number are retained. An explicit URL takes precedence over art.option.url; the fallback is read lazily only when no explicit URL is supplied. Duration and sheet dimensions are captured at metadata time; later duration changes do not move the remaining samples within the current sheet. The old height option remains ignored at runtime.

A seeked callback is ignored while another seek is pending. Before drawing, extraction checks that the media time is finite and within 50ms of the requested sample; a mismatch retries that same target at most three times, then fails with cleanup and retains any previous preview. This prevents observed stale seek events from advancing the sheet, but does not prove frame presentation accuracy: the current time can match even when the drawable first frame is stale. Each pending data/seek/presentation wait has a 30-second deadline. Expiry cancels its callback and decoder, reports through the existing warning path, and retains the last usable preview. This is a per-sample readiness deadline, not a complete network, background-tab or aggregate resource policy. PKG-AUTO-THUMB-10 adds a separate30-second encoding deadline starting just before toBlob. This is an intentional bound on formerly unbounded pending encoding; it does not shorten the frame wait or change the public frame-time formula. Very slow encodes can now warn and retain the last preview instead of retaining the decoder indefinitely. Initial metadata/network acquisition has its own 30-second deadline, as described above, and also ends on media errors or session cancellation. None of these deadlines guarantees browser-level GPU/encoder reclamation or exact background-tab wall-clock scheduling.

The previous empty first encode is removed. Encoding is serial, duplicate native callbacks are consumed once, and source replacement/destruction invalidates old metadata, seek and Blob callbacks. Failure does not revoke an earlier usable preview. Only generated URLs are revoked; a foreign public thumbnail URL is not owned by this package. A failed host setter releases the newly created URL.

Extraction failures are observed with console.warn('ArtPlayer auto-thumbnail failed:', error) after cleanup. The already-resolved registration Promise cannot represent later media failures. Native callbacks do not escape with an unhandled exception. A registration failure still rejects its original Promise with the original error.

Verification and remaining work

Use the root Yarn toolchain:

yarn test:auto-thumbnail
yarn test:auto-thumbnail-types
yarn test:auto-thumbnail-types-package
yarn exec tsc -p packages/artplayer-plugin-auto-thumbnail/tsconfig.json --noEmit
yarn test:browser test/browser/auto-thumbnail-lifecycle.spec.js
yarn test:browser test/browser/auto-thumbnail-pixels.spec.js
yarn build artplayer-plugin-auto-thumbnail

test/auto-thumbnail-encoding.test.js controls withheld/late callbacks and timer delivery; it does not reproduce a spontaneous native encoder hang. The browser lifecycle case performs actual JPEG encoding, deliberately withholds its callback, then invokes the captured deadline to check real decoder/canvas cleanup. It does not claim a measured30-second browser stall. Existing complete/restart/destroy cases retain actual encoded Blob and image-decode checks.

ARTPLAYER_AUTO_THUMBNAIL_BASELINE=1 selects the frozen workspace for candidate regression tests. ARTPLAYER_AUTO_THUMBNAIL_ARTIFACT selects an actual bundle. test/auto-thumbnail-exports.test.js checks the factory identity, alias descriptor, registration and extraction cleanup through the alias for CommonJS and globals. The isolated package runner also installs actual 1.0.1 exports and confirms the historical missing 1.0.0 main/legacy files without substituting source-only code. Historical contract/failure tests remain separate and continue to reproduce old defects; their passing status does not mean those defects should remain.

The public type entry originated in PKG-AUTO-THUMB-08; task09 restored the historical factory alias. Task04 has verified the complete strict source and installed public type/module surfaces using those changes and resource fixes 07/10/11/12. Task03 remains unfinished; task05 depends directly on both 03 and 04, so type completion cannot waive its pixel gate. Native lifecycle validation does not prove correct pixels. The attached renderer now has native pixel checks for all five cells when native frame callbacks are available, including the unique first frame, real black content, changing colors, spatial detail and presentation timestamps. The callback-less fallback still only has acceptance for cells 2-4 (zero-based); its cells 0-1 are diagnostic only. Initial transparent/stale draws and Windows WebKit frame-time offsets remain unresolved under AUTO-THUMB-PIXEL-01. Merely waiting for seeked, adding two animation frames, or seeking away and back did not consistently repair the first frame. Do not close this risk based on the later-cell assertions or replace it with a nonblack test; legitimate black frames have opaque pixels. Frame timing, additional reentry/cleanup boundaries and resource budgets still need the remaining task03 work.

yarn probe:auto-thumbnail-rendering captures an isolated diagnostic matrix from the current built ESM and the frozen timeline media. It compares intrinsic-size hidden/transparent/clipped/visible elements with current readiness, a readyState gate, and waiting for the first loadeddata event. The captured Windows WebKit profiles all miss the unique first frame; the Chromium/Firefox controls retain it. Seek records distinguish an already-ready element from a delivered loadeddata event. A zero getVideoPlaybackQuality().totalVideoFrames is also observed in the correct Firefox control, so it is not a portable frame-readiness gate. The command reports observations, not a passing acceptance suite, and never edits production bundles. See refactor's rendering-readiness record before repeating these options.

yarn probe:auto-thumbnail-rendering --first-seek separately compares initial loadeddata drawing without a seek, one/two animation-frame waits, delayed post-seek drawing and a completed forward/back seek. Native setter instrumentation records actual seek targets; the warm variant must reach one second before seeking zero. These variants still fail the unique-first-frame check on the recorded Windows WebKit host. They are diagnostic page-owned callbacks, not production fixes or cancellation code. See refactor's first-seek checkpoint before repeating them.

yarn probe:auto-thumbnail-native removes ArtPlayer and this plugin entirely. It serves the frozen timeline with the owned dev server, plus a diagnostic H.264 transcode with B frames disabled, and samples native video/canvas in hidden paused, visible paused and hidden play-then-immediately-pause modes. It verifies both files' first purple pixel with FFmpeg before browser capture. FFmpeg/FFprobe must be installed (optional FFMPEG/FFPROBE executable paths); exact version, arguments, media hashes, browser versions, events and all31 samples are recorded in an ignored cache directory. It does not replace the acceptance fixture or add an npm dependency.

The Windows18-case capture still misses the first purple frame in all6 WebKit cases without any plugin code; Chromium/Firefox controls recover it. The visible original video can remain at currentTime0 while canvas exposes the later red frame. This narrows investigation to the native surface on that tested host, but does not certify all Safari versions or excuse plugin pixel gates. Exploratory rate0/slow-rate, fastSeek, tiny first-seek offsets and pre-load/metadata layout or paint waits did not fix it either. The next useful pixel evidence is the same native/candidate case on another supported backend/device or a decoded-frame path with explicit timestamps, not another arbitrary wait. See the native-decoder checkpoint in refactor; AUTO-THUMB-PIXEL-01 and task03 remain open.

The frozen eight-second timeline has one unique purple first frame followed by red, black, blue and yellow sections. Its command and fingerprint are in refactor/baselines/auto-thumbnail-timeline-media.json. Reproduce into a new directory with ARTPLAYER_FFMPEG and node scripts/generate-auto-thumbnail-fixture.mjs <new-directory>; never silently replace the committed fixture.

The current implementation has eight strict TypeScript modules, including the frame reader and encoder split after task03's initial checkpoint. Numeric annotations describe the nominal options; no runtime casts or normalization were added, so historical coercible JS inputs retain the same validation/arithmetic and raw values. Missing options retain the existing behavior. Tasks08/09 supplied the compatible public declarations, implementation assignment fixtures and historical CommonJS alias. Task04's migration acceptance is recorded in refactor/changes/2026-09-15-PKG-AUTO-THUMB-04-migration.md; unresolved runtime behavior remains with task03. Task05 requires both and verifies old/final cores and actual devices; task06 covers installed package entries and the real demo/editor. The initial migration left the version unchanged; release preparation now sets the next major in package.json. Use refactor/baselines/auto-thumbnail-contract.md and auto-thumbnail-failures.md as the historical evidence map; do not regenerate those baselines from this source.

The internal type checkpoint and its core-host assignment fixture are recorded in refactor/changes/2026-09-13-PKG-AUTO-THUMB-03-internal-types.md. A core host can hold partial user thumbnail options; extraction publishes a complete sheet. Do not require the host's existing getter to have every generated-sheet field.

Shared installed browser scope

test/auto-thumbnail-registration.test.js covers destroyed hosts, unavailable destroyed-host members and successful extraction with false/absent flags. The auto-thumbnail-registration.spec.js browser regression installs a retained registrar after actual published/candidate core destruction, records subscriptions, and sends a late metadata event to prove no native video is allocated. It is in both the source collection and the installed roster; configuration alone does not prove a fresh installed run. Main/legacy regression evidence is tracked by PKG-AUTO-THUMB-13. This fix does not resolve AUTO-THUMB-PIXEL-01.

autoThumbnailCandidate reads verified installed bytes through browser-candidate.js. Native video/JPEG lifecycle and pixel files retain their stub player host and exact presentation-callback/fallback limits. The historical file still loads its frozen old artifacts. Installed mode rejects frozen-workspace and explicit-artifact overrides. These are native extraction checks, not full core integration or first-frame acceptance on every engine.

Use yarn test:package --browser and the shared scope rules in ../../scripts/browser-validation/README.md. Missing, stale or changed installation inputs fail without source fallback. Attachments distinguish selected installed, source and published inputs. Source mode retains existing explicit artifact use. The generic Node/type consumer remains core/chapter-only. Module forms, physical devices, editor demos and release readiness retain their separate package gates. See ../../refactor/changes/2026-09-15-CI-01-adaptive-installed.md for actual results.