refactor(mask): [PKG-MASK-03] isolate model runs and release stale work

packages/artplayer-plugin-danmuku-mask/ARCHITECTURE.md

@@ -0,0 +1,133 @@
+# Danmuku Mask maintenance map
+
+The compatibility baseline is the actual npm 1.1.0 package and the earlier
+1.0.0 export shape. Sources and historical failure probes are recorded in
+`refactor/baselines/danmuku-mask-contract.md`, `danmuku-mask-release.json` and
+`danmuku-mask-failures-validation.json`, relative to the repository root.
+
+## Modules and ownership
+
+| Module | Responsibility |
+| --- | --- |
+| `src/index.js` | Preserve synchronous registration, capture core template nodes before option getters, expose named start/stop closures |
+| `src/config.js` | Snapshot the original option defaults without changing OR/undefined semantics |
+| `src/sdk.js` | Existing TF backend selection, MediaPipe adapter configuration, mask calls and model disposal outlet |
+| `src/controller.js` | One active run, initialization/inference serialization, cancellation, RAF ownership, ready/destroy subscriptions |
+| `src/output.js` | Private canvas/context, unchanged binary mask colors and threshold conversion, guarded maskImage commit, canvas release |
+
+These modules remain JavaScript for PKG-MASK-03. Full owned-source TypeScript and
+public declaration work belongs to PKG-MASK-04; this task does not claim it is
+finished. Public declarations are unchanged.
+
+Mask captures the core's video and `.art-danmuku` layer. The core template
+provides that layer before plugins register; Danmuku does not create it.
+Mask does not access the Danmuku queue, individual item nodes, Worker protocol,
+facade/Owner methods or `artplayerPluginDanmuku:*` events. Danmuku07's fixed CSS
+mask test establishes root-layer coexistence, not segmentation/model acceptance.
+The existing demo registers Danmuku then Mask; that order alone is not a model
+or template readiness guarantee.
+
+## Start, cancellation and resource lifecycle
+
+Registration returns `{ name, start, stop }` synchronously. The public named
+`startSegmentation` closure returns `Promise<void>`; `stopSegmentation` returns
+undefined. Extracted calls keep their captured controller. Concurrent/repeated
+starts share the current initialization and do not add a second inference or
+RAF chain. Successful start still does not wait for a completed first mask.
+
+Each run owns its model, canvas/context, cancellation resolver and at most one
+RAF or inference. A run may commit output only while it remains current, running
+and attached to an undestroyed player. Check this after backend/model loading,
+segmentPeople, toBinaryMask and drawMask. An old continuation must never update
+the layer or reschedule its own loop after stop, destroy or replacement.
+
+Stop marks the run inactive, resolves pending public start, cancels its exact
+RAF (including id zero), and writes the historical `maskImage = 'none'`.
+It leaves the other setup styles intact. Destroy additionally prevents restart
+and detaches the exact ready/destroy callbacks. Registration rollback detaches
+subscriptions without changing preexisting host styles. Ready observes startup
+rejections locally; explicit start still rejects backend or canvas failures.
+
+Stopping now releases the model and canvas instead of retaining them indefinitely.
+A subsequent start initializes a new model. This is an intentional resource
+correction; the extra restart initialization cost needs native validation.
+An in-progress SDK operation cannot be aborted through the current SDK API.
+Its late result is ignored, then its private canvas bitmap is reset to zero
+dimensions and its accessible model is disposed. Once frame work has settled,
+a pending SDK disposal must not keep the independent canvas bitmap allocated.
+Restart waits for that outstanding work and
+visible disposal result instead of overlapping two model runs. Repeated stop
+calls share disposal and cannot bypass that wait.
+
+If an SDK operation never settles, a restart can remain pending. Cancellation
+still settles its public start when stopped again. The plugin cannot prove
+termination of unabortable native/WASM work or recover resources hidden inside
+a rejected SDK initialization. Do not claim otherwise from Promise cancellation.
+
+`releaseSegmenter` calls the supported segmenter.dispose API and observes any
+returned Promise. The installed body-segmentation 1.0.2 MediaPipe implementation
+calls its underlying solution.close without returning that result. Consequently
+awaiting dispose does not prove the native close or GPU release has completed.
+Actual resource release, model requests, cross-instance TF state and devices
+remain PKG-MASK-05 evidence requirements. Do not depend on private SDK fields to
+make a disposal test appear stronger than its real API.
+
+## Behavior and compatibility boundaries
+
+- Keep registrar-time option snapshots and the historical OR defaults. Zero
+  modelSelection/opacity/threshold/blur still take their defaults; explicit
+  smoothSegmentation=false is retained. Template references are captured before
+  getters in the option snapshot run.
+- Keep runtime=mediapipe and modelType=general. The current adapter maps general
+  to modelSelection=0 and ignores several additionally passed plugin options;
+  connecting those options would be a behavior change, not a cleanup.
+- tf.setBackend('webgl') fulfilled false does not mean CPU fallback. Only rejection
+  tries CPU. TF backend selection does not prove MediaPipe's inference backend.
+- Model creation failure still logs `Error initializing segmenter:` and resolves
+  start, but no longer creates an idle endless RAF loop. Explicit start retries.
+  CPU/backend failure still rejects explicit start; automatic ready logs the
+  observed rejection through `Failed to start danmuku mask:`.
+- No video/layer or no 2D context fails startup before retaining an accessible
+  model or loop. Missing dimensions waits for readable frames. These are failure
+  handling corrections; normal video inference/output is unchanged.
+- Active inference/read failures retain `Error in segmentBody:` and retry on the
+  next frame without silently clearing an earlier mask. Late failures after
+  cancellation are observed without restarting work or overwriting user state.
+- Preserve white foreground/black background binary colors, drawMask arguments,
+  canvas PNG data URLs and strict RGB >250 transparency (250 stays opaque).
+  The canvas remains private; it is never inserted into Danmuku's node pool.
+- This task introduces no public events, type augmentation, SDK dependency
+  updates, default CDN pinning or new model-selection semantics.
+
+## Editing and validation
+
+From the repository root with pinned Node and Yarn:
+
+```sh
+node --test test/danmuku-mask-lifecycle.test.js
+yarn test:danmuku-mask
+node --test test/danmuku-mask-failures.test.js refactor/scripts/danmuku-mask-contract.test.mjs
+node node_modules/eslint/bin/eslint.js packages/artplayer-plugin-danmuku-mask/src test/helpers/danmuku-mask-candidate.js test/danmuku-mask-lifecycle.test.js
+yarn build artplayer-plugin-danmuku-mask
+yarn dev artplayer-plugin-danmuku-mask
+```
+
+The candidate helper bundles current owned modules and substitutes the SDK
+imports with controlled implementations. Its fake RAF/video/canvas test
+cancellation and output ownership; they do not prove real model quality, canvas
+pixels, CSS alignment, CORS, browser scheduling, GPU memory or WASM cleanup.
+Historical probes remain frozen and assert old failures, not candidate success.
+
+Future changes to controller scheduling must cover stop/destroy during backend,
+model, inference, binary-mask and draw waits; repeated/concurrent starts; restart
+behind disposal; synchronous SDK reentry; RAF zero; late rejection; and listener
+rollback. Change the output module only with pixel-boundary and late-commit tests.
+Two registrations in the same controlled SDK/RAF environment also verify that
+stopping or destroying one leaves the other model, mask and scheduled frame owned
+by its original player. Native shared TensorFlow backend state remains unverified.
+
+Native acceptance still needs actual local model loading, new/old cores, real
+Danmuku composition, pause/seek/source/layout changes, multiple players, failure
+recovery and post-destroy resources. The default unversioned solutionPath can
+load assets independently of Yarn's SDK resolution. PKG-MASK-04/05/06 remain
+separate source/type, real combination and distribution/release stages.

packages/artplayer-plugin-danmuku-mask/dist/artplayer-plugin-danmuku-mask.js

@@ -4,4 +4,4 @@
  * (c) 2017-2026 Harvey Zhao
  * Released under the MIT License.
  */
-!function(e,t){"object"===typeof exports&&"undefined"!==typeof module?module.exports=t():"function"==typeof define&&define.amd?(t.artplayerPluginDanmukuMask=t(),define(function(){return t.artplayerPluginDanmukuMask})):(e="undefined"!==typeof globalThis?globalThis:e||self).artplayerPluginDanmukuMask=t()}(this,function(){"use strict";function e(e,t){for(var n=0;n<t.length;n++){const r=t[n];if("string"!==typeof r&&!Array.isArray(r))for(const t in r)if("default"!==t&&!(t in e)){const n=Object.getOwnPropertyDescriptor(r,t);n&&Object.defineProperty(e,t,n.get?n:{enumerable:!0,get:()=>r[t]})}}return Object.freeze(Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}))}class t{constructor(e,t){this.backend=e,this.dataMover=t,this.data=new WeakMap,this.dataIdsCount=0}get(e){return this.data.has(e)||this.dataMover.moveData(this.backend,e),this.data.get(e)}set(e,t){this.dataIdsCount++,this.data.set(e,t)}has(e){return this.data.has(e)}delete(e){return this.dataIdsCount--,this.data.delete(e)}numDataIds(){return this.dataIdsCount}}class n{refCount(e){return r("refCount")}incRef(e){return r("incRef")}timerAvailable(){return!0}time(e){return r("time")}read(e){return r("read")}readSync(e){return r("readSync")}readToGPU(e,t){return r("readToGPU")}numDataIds(){return r("numDataIds")}disposeData(e,t){return r("disposeData")}write(e,t,n){return r("write")}move(e,t,n,a,s){return r("move")}createTensorFromGPUData(e,t,n){return r("createTensorFromGPUData")}memory(){return r("memory")}floatPrecision(){return r("floatPrecision")}epsilon(){return 32===this.floatPrecision()?1e-7:1e-4}dispose(){return r("dispose")}}function r(e){throw new Error(`'${e}' not yet implemented or not found in the registry. This kernel may not be supported by the tfjs backend you have chosen`)}function a(e,t,n){return Math.max(e,Math.min(t,n))}function s(e){return e%2===0?e:e+1}function o(e,t,n){const r=e[t];e[t]=e[n],e[n]=r}function i(e,t){if(!e)throw new Error("string"===typeof t?t:t())}function u(e,t,n=""){i(d(e,t),()=>n+` Shapes ${e} and ${t} must match`)}function l(e){i(null!=e,()=>"The input to the tensor constructor must be a non-null value.")}function c(e){if(0===e.length)return 1;let t=e[0];for(let n=1;n<e.length;n++)t*=e[n];return t}function d(e,t){if(e===t)return!0;if(null==e||null==t)return!1;if(e.length!==t.length)return!1;for(let n=0;n<e.length;n++)if(e[n]!==t[n])return!1;return!0}function p(e){return e%1===0}function h(e){const t=Math.ceil(Math.sqrt(e));return[t,Math.ceil(e/t)]}function f(e,t){return t<=e.length?e:e+" ".repeat(t-e.length)}function m(e,t=e=>0,n,r){return new Promise((a,s)=>{let o=0;const i=()=>{if(e())return void a();o++;const u=t(o);null!=n&&o>=n?s():null!=r?r(i,u):setTimeout(i,u)};i()})}function g(e,t){let n=1,r=-1;for(let s=0;s<e.length;++s)if(e[s]>=0)n*=e[s];else if(-1===e[s]){if(-1!==r)throw Error(`Shapes can only have 1 implicit size. Found -1 at dim ${r} and dim ${s}`);r=s}else if(e[s]<0)throw Error(`Shapes can not be < 0. Found ${e[s]} at dim ${s}`);if(-1===r){if(t>0&&t!==n)throw Error(`Size(${t}) must match the product of shape ${e}`);return e}if(0===n)throw Error(`Cannot infer the missing size in [${e}] when there are 0 elements`);if(t%n!==0)throw Error(`The implicit shape can't be a fractional number. Got ${t} / ${n}`);const a=e.slice();return a[r]=t/n,a}function y(e,t){const n=t.length;return i((e=null==e?t.map((e,t)=>t):[].concat(e)).every(e=>e>=-n&&e<n),()=>`All values in axis param must be in range [-${n}, ${n}) but got axis ${e}`),i(e.every(e=>p(e)),()=>`All values in axis param must be integers but got axis ${e}`),e.map(e=>e<0?n+e:e)}function b(e,t){const n=[],r=[],a=null!=t&&Array.isArray(t)&&0===t.length,s=null==t||a?null:y(t,e).sort();let o=0;for(let i=0;i<e.length;++i){if(null!=s){if(s[o]===i&&1!==e[i])throw new Error(`Can't squeeze axis ${i} since its dim '${e[i]}' is not 1`);(null==s[o]||s[o]>i)&&1===e[i]&&(n.push(e[i]),r.push(i)),s[o]<=i&&o++}1!==e[i]&&(n.push(e[i]),r.push(i))}return{newShape:n,keptDims:r}}function x(e,t){return v(e,t)}function v(e,t){let n=null;if(null==e||"float32"===e)n=new Float32Array(t);else if("int32"===e)n=new Int32Array(t);else if("bool"===e)n=new Uint8Array(t);else{if("string"!==e)throw new Error(`Unknown data type ${e}`);n=new Array(t)}return n}function w(e,t){return"complex64"!==t&&(("float32"!==t||"complex64"===e)&&(("int32"!==t||"float32"===e||"complex64"===e)&&("bool"!==t||"bool"!==e)))}function k(e){if("float32"===e||"int32"===e)return 4;if("complex64"===e)return 8;if("bool"===e)return 1;throw new Error(`Unknown dtype ${e}`)}function I(e){return"string"===typeof e||e instanceof String}function N(e){return Array.isArray(e)?N(e[0]):e instanceof Float32Array?"float32":e instanceof Int32Array||e instanceof Uint8Array||e instanceof Uint8ClampedArray?"int32":"number"===typeof e?"float32":I(e)?"string":function(e){return"boolean"===typeof e}(e)?"bool":"float32"}function S(e){return!!(e&&e.constructor&&e.call&&e.apply)}function T(e,t){for(let n=t;n<e;++n)if(e%n===0)return nLine truncated
+!function(e,t){"object"===typeof exports&&"undefined"!==typeof module?module.exports=t():"function"==typeof define&&define.amd?(e.artplayerPluginDanmukuMask=t(),define(function(){return e.artplayerPluginDanmukuMask})):(e="undefined"!==typeof globalThis?globalThis:e||self).artplayerPluginDanmukuMask=t()}(this,function(){"use strict";function e(e,t){for(var n=0;n<t.length;n++){const r=t[n];if("string"!==typeof r&&!Array.isArray(r))for(const t in r)if("default"!==t&&!(t in e)){const n=Object.getOwnPropertyDescriptor(r,t);n&&Object.defineProperty(e,t,n.get?n:{enumerable:!0,get:()=>r[t]})}}return Object.freeze(Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}))}class t{constructor(e,t){this.backend=e,this.dataMover=t,this.data=new WeakMap,this.dataIdsCount=0}get(e){return this.data.has(e)||this.dataMover.moveData(this.backend,e),this.data.get(e)}set(e,t){this.dataIdsCount++,this.data.set(e,t)}has(e){return this.data.has(e)}delete(e){return this.dataIdsCount--,this.data.delete(e)}numDataIds(){return this.dataIdsCount}}class n{refCount(e){return r("refCount")}incRef(e){return r("incRef")}timerAvailable(){return!0}time(e){return r("time")}read(e){return r("read")}readSync(e){return r("readSync")}readToGPU(e,t){return r("readToGPU")}numDataIds(){return r("numDataIds")}disposeData(e,t){return r("disposeData")}write(e,t,n){return r("write")}move(e,t,n,a,s){return r("move")}createTensorFromGPUData(e,t,n){return r("createTensorFromGPUData")}memory(){return r("memory")}floatPrecision(){return r("floatPrecision")}epsilon(){return 32===this.floatPrecision()?1e-7:1e-4}dispose(){return r("dispose")}}function r(e){throw new Error(`'${e}' not yet implemented or not found in the registry. This kernel may not be supported by the tfjs backend you have chosen`)}function a(e,t,n){return Math.max(e,Math.min(t,n))}function s(e){return e%2===0?e:e+1}function o(e,t,n){const r=e[t];e[t]=e[n],e[n]=r}function i(e,t){if(!e)throw new Error("string"===typeof t?t:t())}function u(e,t,n=""){i(d(e,t),()=>n+` Shapes ${e} and ${t} must match`)}function l(e){i(null!=e,()=>"The input to the tensor constructor must be a non-null value.")}function c(e){if(0===e.length)return 1;let t=e[0];for(let n=1;n<e.length;n++)t*=e[n];return t}function d(e,t){if(e===t)return!0;if(null==e||null==t)return!1;if(e.length!==t.length)return!1;for(let n=0;n<e.length;n++)if(e[n]!==t[n])return!1;return!0}function p(e){return e%1===0}function h(e){const t=Math.ceil(Math.sqrt(e));return[t,Math.ceil(e/t)]}function f(e,t){return t<=e.length?e:e+" ".repeat(t-e.length)}function m(e,t=e=>0,n,r){return new Promise((a,s)=>{let o=0;const i=()=>{if(e())return void a();o++;const u=t(o);null!=n&&o>=n?s():null!=r?r(i,u):setTimeout(i,u)};i()})}function g(e,t){let n=1,r=-1;for(let s=0;s<e.length;++s)if(e[s]>=0)n*=e[s];else if(-1===e[s]){if(-1!==r)throw Error(`Shapes can only have 1 implicit size. Found -1 at dim ${r} and dim ${s}`);r=s}else if(e[s]<0)throw Error(`Shapes can not be < 0. Found ${e[s]} at dim ${s}`);if(-1===r){if(t>0&&t!==n)throw Error(`Size(${t}) must match the product of shape ${e}`);return e}if(0===n)throw Error(`Cannot infer the missing size in [${e}] when there are 0 elements`);if(t%n!==0)throw Error(`The implicit shape can't be a fractional number. Got ${t} / ${n}`);const a=e.slice();return a[r]=t/n,a}function y(e,t){const n=t.length;return i((e=null==e?t.map((e,t)=>t):[].concat(e)).every(e=>e>=-n&&e<n),()=>`All values in axis param must be in range [-${n}, ${n}) but got axis ${e}`),i(e.every(e=>p(e)),()=>`All values in axis param must be integers but got axis ${e}`),e.map(e=>e<0?n+e:e)}function b(e,t){const n=[],r=[],a=null!=t&&Array.isArray(t)&&0===t.length,s=null==t||a?null:y(t,e).sort();let o=0;for(let i=0;i<e.length;++i){if(null!=s){if(s[o]===i&&1!==e[i])throw new Error(`Can't squeeze axis ${i} since its dim '${e[i]}' is not 1`);(null==s[o]||s[o]>i)&&1===e[i]&&(n.push(e[i]),r.push(i)),s[o]<=i&&o++}1!==e[i]&&(n.push(e[i]),r.push(i))}return{newShape:n,keptDims:r}}function x(e,t){return v(e,t)}function v(e,t){let n=null;if(null==e||"float32"===e)n=new Float32Array(t);else if("int32"===e)n=new Int32Array(t);else if("bool"===e)n=new Uint8Array(t);else{if("string"!==e)throw new Error(`Unknown data type ${e}`);n=new Array(t)}return n}function w(e,t){return"complex64"!==t&&(("float32"!==t||"complex64"===e)&&(("int32"!==t||"float32"===e||"complex64"===e)&&("bool"!==t||"bool"!==e)))}function k(e){if("float32"===e||"int32"===e)return 4;if("complex64"===e)return 8;if("bool"===e)return 1;throw new Error(`Unknown dtype ${e}`)}function I(e){return"string"===typeof e||e instanceof String}function N(e){return Array.isArray(e)?N(e[0]):e instanceof Float32Array?"float32":e instanceof Int32Array||e instanceof Uint8Array||e instanceof Uint8ClampedArray?"int32":"number"===typeof e?"float32":I(e)?"string":function(e){return"boolean"===typeof e}(e)?"bool":"float32"}function S(e){return!!(e&&e.constructor&&e.call&&e.apply)}function T(e,t){for(let n=t;n<e;++n)if(e%n===0)return nLine truncated

packages/artplayer-plugin-danmuku-mask/dist/artplayer-plugin-danmuku-mask.legacy.js

@@ -4,4 +4,4 @@
  * (c) 2017-2026 Harvey Zhao
  * Released under the MIT License.
  */
-!function(e,t){"object"===typeof exports&&"undefined"!==typeof module?module.exports=t():"function"==typeof define&&define.amd?(t.artplayerPluginDanmukuMask=t(),define(function(){return t.artplayerPluginDanmukuMask})):(e="undefined"!==typeof globalThis?globalThis:e||self).artplayerPluginDanmukuMask=t()}(this,function(){"use strict";var e=(e,t,n)=>new Promise((r,a)=>{var s=e=>{try{i(n.next(e))}catch(t){a(t)}},o=e=>{try{i(n.throw(e))}catch(t){a(t)}},i=e=>e.done?r(e.value):Promise.resolve(e.value).then(s,o);i((n=n.apply(e,t)).next())});function t(e,t){for(var n=0;n<t.length;n++){const r=t[n];if("string"!==typeof r&&!Array.isArray(r))for(const t in r)if("default"!==t&&!(t in e)){const n=Object.getOwnPropertyDescriptor(r,t);n&&Object.defineProperty(e,t,n.get?n:{enumerable:!0,get:()=>r[t]})}}return Object.freeze(Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}))}class n{constructor(e,t){this.backend=e,this.dataMover=t,this.data=new WeakMap,this.dataIdsCount=0}get(e){return this.data.has(e)||this.dataMover.moveData(this.backend,e),this.data.get(e)}set(e,t){this.dataIdsCount++,this.data.set(e,t)}has(e){return this.data.has(e)}delete(e){return this.dataIdsCount--,this.data.delete(e)}numDataIds(){return this.dataIdsCount}}class r{refCount(e){return a("refCount")}incRef(e){return a("incRef")}timerAvailable(){return!0}time(e){return a("time")}read(e){return a("read")}readSync(e){return a("readSync")}readToGPU(e,t){return a("readToGPU")}numDataIds(){return a("numDataIds")}disposeData(e,t){return a("disposeData")}write(e,t,n){return a("write")}move(e,t,n,r,s){return a("move")}createTensorFromGPUData(e,t,n){return a("createTensorFromGPUData")}memory(){return a("memory")}floatPrecision(){return a("floatPrecision")}epsilon(){return 32===this.floatPrecision()?1e-7:1e-4}dispose(){return a("dispose")}}function a(e){throw new Error(`'${e}' not yet implemented or not found in the registry. This kernel may not be supported by the tfjs backend you have chosen`)}function s(e,t,n){return Math.max(e,Math.min(t,n))}function o(e){return e%2===0?e:e+1}function i(e,t,n){const r=e[t];e[t]=e[n],e[n]=r}function u(e,t){if(!e)throw new Error("string"===typeof t?t:t())}function l(e,t,n=""){u(p(e,t),()=>n+` Shapes ${e} and ${t} must match`)}function c(e){u(null!=e,()=>"The input to the tensor constructor must be a non-null value.")}function d(e){if(0===e.length)return 1;let t=e[0];for(let n=1;n<e.length;n++)t*=e[n];return t}function p(e,t){if(e===t)return!0;if(null==e||null==t)return!1;if(e.length!==t.length)return!1;for(let n=0;n<e.length;n++)if(e[n]!==t[n])return!1;return!0}function h(e){return e%1===0}function f(e){const t=Math.ceil(Math.sqrt(e));return[t,Math.ceil(e/t)]}function m(e,t){return t<=e.length?e:e+" ".repeat(t-e.length)}function g(e,t=e=>0,n,r){return new Promise((a,s)=>{let o=0;const i=()=>{if(e())return void a();o++;const u=t(o);null!=n&&o>=n?s():null!=r?r(i,u):setTimeout(i,u)};i()})}function y(e,t){let n=1,r=-1;for(let s=0;s<e.length;++s)if(e[s]>=0)n*=e[s];else if(-1===e[s]){if(-1!==r)throw Error(`Shapes can only have 1 implicit size. Found -1 at dim ${r} and dim ${s}`);r=s}else if(e[s]<0)throw Error(`Shapes can not be < 0. Found ${e[s]} at dim ${s}`);if(-1===r){if(t>0&&t!==n)throw Error(`Size(${t}) must match the product of shape ${e}`);return e}if(0===n)throw Error(`Cannot infer the missing size in [${e}] when there are 0 elements`);if(t%n!==0)throw Error(`The implicit shape can't be a fractional number. Got ${t} / ${n}`);const a=e.slice();return a[r]=t/n,a}function b(e,t){const n=t.length;return u((e=null==e?t.map((e,t)=>t):[].concat(e)).every(e=>e>=-n&&e<n),()=>`All values in axis param must be in range [-${n}, ${n}) but got axis ${e}`),u(e.every(e=>h(e)),()=>`All values in axis param must be integers but got axis ${e}`),e.map(e=>e<0?n+e:e)}function x(e,t){const n=[],r=[],a=null!=t&&Array.isArray(t)&&0===t.length,s=null==t||a?null:b(t,e).sort();let o=0;for(let i=0;i<e.length;++i){if(null!=s){if(s[o]===i&&1!==e[i])throw new Error(`Can't squeeze axis ${i} since its dim '${e[i]}' is not 1`);(null==s[o]||s[o]>i)&&1===e[i]&&(n.push(e[i]),r.push(i)),s[o]<=i&&o++}1!==e[i]&&(n.push(e[i]),r.push(i))}return{newShape:n,keptDims:r}}function v(e,t){return w(e,t)}function w(e,t){let n=null;if(null==e||"float32"===e)n=new Float32Array(t);else if("int32"===e)n=new Int32Array(t);else if("bool"===e)n=new Uint8Array(t);else{if("string"!==e)throw new Error(`Unknown data type ${e}`);n=new Array(t)}return n}function k(e,t){return"complex64"!==t&&(("float32"!==t||"complex64"===e)&&(("int32"!==t||"float32"===e||"complex64"===e)&&("bool"!==t||"bool"!==e)))}function I(e){if("float32"===e||"int32"===e)return 4;if("complex64"===e)return 8;if("bool"===e)return 1;throw new Error(`Unknown dtype ${e}`)}function N(e){return"string"===typeof e||e instanceof String}function S(e){return Array.isArray(e)?S(e[0]):e instanceof Float32Array?"float32":e instanceof Int32Array||e instanceof Uint8Array||e instanceof Uint8ClampedArray?"int32":"number"===tLine truncated
+!function(e,t){"object"===typeof exports&&"undefined"!==typeof module?module.exports=t():"function"==typeof define&&define.amd?(e.artplayerPluginDanmukuMask=t(),define(function(){return e.artplayerPluginDanmukuMask})):(e="undefined"!==typeof globalThis?globalThis:e||self).artplayerPluginDanmukuMask=t()}(this,function(){"use strict";var e=(e,t,n)=>new Promise((r,a)=>{var s=e=>{try{i(n.next(e))}catch(t){a(t)}},o=e=>{try{i(n.throw(e))}catch(t){a(t)}},i=e=>e.done?r(e.value):Promise.resolve(e.value).then(s,o);i((n=n.apply(e,t)).next())});function t(e,t){for(var n=0;n<t.length;n++){const r=t[n];if("string"!==typeof r&&!Array.isArray(r))for(const t in r)if("default"!==t&&!(t in e)){const n=Object.getOwnPropertyDescriptor(r,t);n&&Object.defineProperty(e,t,n.get?n:{enumerable:!0,get:()=>r[t]})}}return Object.freeze(Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}))}class n{constructor(e,t){this.backend=e,this.dataMover=t,this.data=new WeakMap,this.dataIdsCount=0}get(e){return this.data.has(e)||this.dataMover.moveData(this.backend,e),this.data.get(e)}set(e,t){this.dataIdsCount++,this.data.set(e,t)}has(e){return this.data.has(e)}delete(e){return this.dataIdsCount--,this.data.delete(e)}numDataIds(){return this.dataIdsCount}}class r{refCount(e){return a("refCount")}incRef(e){return a("incRef")}timerAvailable(){return!0}time(e){return a("time")}read(e){return a("read")}readSync(e){return a("readSync")}readToGPU(e,t){return a("readToGPU")}numDataIds(){return a("numDataIds")}disposeData(e,t){return a("disposeData")}write(e,t,n){return a("write")}move(e,t,n,r,s){return a("move")}createTensorFromGPUData(e,t,n){return a("createTensorFromGPUData")}memory(){return a("memory")}floatPrecision(){return a("floatPrecision")}epsilon(){return 32===this.floatPrecision()?1e-7:1e-4}dispose(){return a("dispose")}}function a(e){throw new Error(`'${e}' not yet implemented or not found in the registry. This kernel may not be supported by the tfjs backend you have chosen`)}function s(e,t,n){return Math.max(e,Math.min(t,n))}function o(e){return e%2===0?e:e+1}function i(e,t,n){const r=e[t];e[t]=e[n],e[n]=r}function u(e,t){if(!e)throw new Error("string"===typeof t?t:t())}function l(e,t,n=""){u(p(e,t),()=>n+` Shapes ${e} and ${t} must match`)}function c(e){u(null!=e,()=>"The input to the tensor constructor must be a non-null value.")}function d(e){if(0===e.length)return 1;let t=e[0];for(let n=1;n<e.length;n++)t*=e[n];return t}function p(e,t){if(e===t)return!0;if(null==e||null==t)return!1;if(e.length!==t.length)return!1;for(let n=0;n<e.length;n++)if(e[n]!==t[n])return!1;return!0}function h(e){return e%1===0}function f(e){const t=Math.ceil(Math.sqrt(e));return[t,Math.ceil(e/t)]}function m(e,t){return t<=e.length?e:e+" ".repeat(t-e.length)}function g(e,t=e=>0,n,r){return new Promise((a,s)=>{let o=0;const i=()=>{if(e())return void a();o++;const u=t(o);null!=n&&o>=n?s():null!=r?r(i,u):setTimeout(i,u)};i()})}function y(e,t){let n=1,r=-1;for(let s=0;s<e.length;++s)if(e[s]>=0)n*=e[s];else if(-1===e[s]){if(-1!==r)throw Error(`Shapes can only have 1 implicit size. Found -1 at dim ${r} and dim ${s}`);r=s}else if(e[s]<0)throw Error(`Shapes can not be < 0. Found ${e[s]} at dim ${s}`);if(-1===r){if(t>0&&t!==n)throw Error(`Size(${t}) must match the product of shape ${e}`);return e}if(0===n)throw Error(`Cannot infer the missing size in [${e}] when there are 0 elements`);if(t%n!==0)throw Error(`The implicit shape can't be a fractional number. Got ${t} / ${n}`);const a=e.slice();return a[r]=t/n,a}function b(e,t){const n=t.length;return u((e=null==e?t.map((e,t)=>t):[].concat(e)).every(e=>e>=-n&&e<n),()=>`All values in axis param must be in range [-${n}, ${n}) but got axis ${e}`),u(e.every(e=>h(e)),()=>`All values in axis param must be integers but got axis ${e}`),e.map(e=>e<0?n+e:e)}function x(e,t){const n=[],r=[],a=null!=t&&Array.isArray(t)&&0===t.length,s=null==t||a?null:b(t,e).sort();let o=0;for(let i=0;i<e.length;++i){if(null!=s){if(s[o]===i&&1!==e[i])throw new Error(`Can't squeeze axis ${i} since its dim '${e[i]}' is not 1`);(null==s[o]||s[o]>i)&&1===e[i]&&(n.push(e[i]),r.push(i)),s[o]<=i&&o++}1!==e[i]&&(n.push(e[i]),r.push(i))}return{newShape:n,keptDims:r}}function v(e,t){return w(e,t)}function w(e,t){let n=null;if(null==e||"float32"===e)n=new Float32Array(t);else if("int32"===e)n=new Int32Array(t);else if("bool"===e)n=new Uint8Array(t);else{if("string"!==e)throw new Error(`Unknown data type ${e}`);n=new Array(t)}return n}function k(e,t){return"complex64"!==t&&(("float32"!==t||"complex64"===e)&&(("int32"!==t||"float32"===e||"complex64"===e)&&("bool"!==t||"bool"!==e)))}function I(e){if("float32"===e||"int32"===e)return 4;if("complex64"===e)return 8;if("bool"===e)return 1;throw new Error(`Unknown dtype ${e}`)}function N(e){return"string"===typeof e||e instanceof String}function S(e){return Array.isArray(e)?S(e[0]):e instanceof Float32Array?"float32":e instanceof Int32Array||e instanceof Uint8Array||e instanceof Uint8ClampedArray?"int32":"number"===tLine truncated

packages/artplayer-plugin-danmuku-mask/dist/artplayer-plugin-danmuku-mask.mjs

@@ -23,6 +23,20 @@ function _mergeNamespaces(n, m) {
   }
   return Object.freeze(Object.defineProperty(n, Symbol.toStringTag, { value: "Module" }));
 }
+function maskConfig(option) {
+  return {
+    solutionPath: option.solutionPath || "https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation",
+    modelSelection: option.modelSelection || 1,
+    smoothSegmentation: option.smoothSegmentation !== void 0 ? option.smoothSegmentation : true,
+    minDetectionConfidence: option.minDetectionConfidence || 0.5,
+    minTrackingConfidence: option.minTrackingConfidence || 0.5,
+    selfieMode: option.selfieMode || false,
+    drawContour: option.drawContour || false,
+    foregroundThreshold: option.foregroundThreshold || 0.5,
+    opacity: option.opacity || 1,
+    maskBlurAmount: option.maskBlurAmount || 3
+  };
+}
 const EPSILON_FLOAT32$1 = 1e-7;
 const EPSILON_FLOAT16$1 = 1e-4;
 class DataStorage {
@@ -49040,129 +49054,278 @@ const kernelConfigs = [
 for (const kernelConfig of kernelConfigs) {
   registerKernel(kernelConfig);
 }
-function artplayerPluginDanmukuMask(option = {}) {
-  return (art) => {
-    const {
-      template: { $video, $danmuku }
-    } = art;
-    let segmenter = null;
-    let canvas = null;
-    let ctx = null;
-    let animationFrameId = null;
-    let isInitialized = false;
-    const config = {
-      solutionPath: option.solutionPath || "https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation",
-      modelSelection: option.modelSelection || 1,
-      smoothSegmentation: option.smoothSegmentation !== void 0 ? option.smoothSegmentation : true,
-      minDetectionConfidence: option.minDetectionConfidence || 0.5,
-      minTrackingConfidence: option.minTrackingConfidence || 0.5,
-      selfieMode: option.selfieMode || false,
-      drawContour: option.drawContour || false,
-      foregroundThreshold: option.foregroundThreshold || 0.5,
-      opacity: option.opacity || 1,
-      maskBlurAmount: option.maskBlurAmount || 3
+async function loadSegmenter(config, active) {
+  try {
+    await setBackend("webgl");
+  } catch (error) {
+    if (!active())
+      return null;
+    console.warn("WebGL backend not available, falling back to CPU", error.message);
+    await setBackend("cpu");
+  }
+  if (!active())
+    return null;
+  try {
+    return await ke(Se.MediaPipeSelfieSegmentation, {
+      runtime: "mediapipe",
+      modelType: "general",
+      solutionPath: config.solutionPath,
+      modelSelection: config.modelSelection,
+      smoothSegmentation: config.smoothSegmentation,
+      minDetectionConfidence: config.minDetectionConfidence,
+      minTrackingConfidence: config.minTrackingConfidence,
+      selfieMode: config.selfieMode
+    });
+  } catch (error) {
+    if (active())
+      console.error("Error initializing segmenter:", error);
+    return null;
+  }
+}
+async function releaseSegmenter(segmenter) {
+  try {
+    await segmenter.dispose();
+  } catch (error) {
+    console.warn("Failed to dispose danmuku mask segmenter:", error);
+  }
+}
+const toBinaryMask = (...args) => Ue(...args);
+const drawMask = (...args) => Ne(...args);
+function makeWhiteTransparent(imageData) {
+  const data = imageData.data;
+  for (let i = 0; i < data.length; i += 4) {
+    if (data[i] > 250 && data[i + 1] > 250 && data[i + 2] > 250)
+      data[i + 3] = 0;
+  }
+  return imageData;
+}
+function createOutput(layer) {
+  const canvas = document.createElement("canvas");
+  try {
+    const ctx = canvas.getContext("2d");
+    if (!ctx)
+      throw new Error("Danmuku mask requires a 2D canvas context");
+    Object.assign(layer.style, {
+      maskMode: "alpha",
+      maskSize: "contain",
+      maskRepeat: "no-repeat",
+      backgroundSize: "contain",
+      backgroundRepeat: "no-repeat"
+    });
+    return { canvas, ctx };
+  } catch (error) {
+    canvas.width = 0;
+    canvas.height = 0;
+    throw error;
+  }
+}
+async function renderMask(output, video, layer, segmenter, config, active) {
+  const { canvas, ctx } = output;
+  canvas.width = video.videoWidth;
+  canvas.height = video.videoHeight;
+  const segmentation = await segmenter.segmentPeople(video);
+  if (!active() || !segmentation || !segmentation.length)
+    return;
+  const mask = await toBinaryMask(
+    segmentation,
+    { r: 255, g: 255, b: 255, a: 255 },
+    { r: 0, g: 0, b: 0, a: 255 },
+    config.drawContour,
+    config.foregroundThreshold
+  );
+  if (!active())
+    return;
+  await drawMask(canvas, video, mask, config.opacity, config.maskBlurAmount);
+  if (!active())
+    return;
+  const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
+  ctx.putImageData(makeWhiteTransparent(imageData), 0, 0);
+  const url = canvas.toDataURL();
+  if (active())
+    layer.style.maskImage = `url(${url})`;
+}
+function releaseOutput(output) {
+  if (output) {
+    output.canvas.width = 0;
+    output.canvas.height = 0;
+  }
+}
+class MaskController {
+  constructor(art, config, { $video, $danmuku }) {
+    this.art = art;
+    this.config = config;
+    this.video = $video;
+    this.layer = $danmuku;
+    this.closed = art.isDestroy;
+    this.run = null;
+    this.tail = Promise.resolve();
+    this.start = this.start.bind(this);
+    this.stop = this.stop.bind(this);
+    this.destroy = this.destroy.bind(this);
+    this.ready = () => {
+      this.start().catch((error) => console.error("Failed to start danmuku mask:", error));
     };
-    async function initTensorFlow() {
+    if (!this.closed) {
       try {
-        await setBackend("webgl");
+        art.on("destroy", this.destroy);
+        if (!this.closed)
+          art.on("ready", this.ready);
       } catch (error) {
-        console.warn("WebGL backend not available, falling back to CPU", error.message);
-        await setBackend("cpu");
-      }
-    }
-    async function initSegmenter() {
-      await initTensorFlow();
-      const model = Se.MediaPipeSelfieSegmentation;
-      const segmenterConfig = {
-        runtime: "mediapipe",
-        modelType: "general",
-        solutionPath: config.solutionPath,
-        modelSelection: config.modelSelection,
-        smoothSegmentation: config.smoothSegmentation,
-        minDetectionConfidence: config.minDetectionConfidence,
-        minTrackingConfidence: config.minTrackingConfidence,
-        selfieMode: config.selfieMode
-      };
-      try {
-        segmenter = await ke(model, segmenterConfig);
-        isInitialized = true;
-      } catch (error) {
-        console.error("Error initializing segmenter:", error);
-        isInitialized = false;
-      }
-    }
-    function setupDanmukuStyle() {
-      Object.assign($danmuku.style, {
-        maskMode: "alpha",
-        maskSize: "contain",
-        maskRepeat: "no-repeat",
-        backgroundSize: "contain",
-        backgroundRepeat: "no-repeat"
-      });
-    }
-    function createCanvas2() {
-      canvas = document.createElement("canvas");
-      ctx = canvas.getContext("2d");
-    }
-    function makeWhiteTransparent(imageData) {
-      const data = imageData.data;
-      for (let i = 0; i < data.length; i += 4) {
-        if (data[i] > 250 && data[i + 1] > 250 && data[i + 2] > 250) {
-          data[i + 3] = 0;
+        this.closed = true;
+        this.halt();
+        try {
+          art.off("ready", this.ready);
+        } catch {
         }
+        try {
+          art.off("destroy", this.destroy);
+        } catch {
+        }
+        throw error;
       }
-      return imageData;
     }
-    async function segmentBody() {
-      if (!isInitialized || $video.paused || $video.ended) {
-        animationFrameId = requestAnimationFrame(segmentBody);
-        return;
-      }
+  }
+  active(run) {
+    return run === this.run && run.running && !this.closed && !this.art.isDestroy;
+  }
+  release(run) {
+    if (run.releasing)
+      return run.releasing;
+    const segmenter = run.segmenter;
+    run.segmenter = null;
+    const output = run.output;
+    run.output = null;
+    run.releasing = (async () => {
+      await Promise.resolve();
       try {
-        canvas.width = $video.videoWidth;
-        canvas.height = $video.videoHeight;
-        const segmentation = await segmenter.segmentPeople($video);
-        if (!segmentation || segmentation.length === 0) {
-          animationFrameId = requestAnimationFrame(segmentBody);
+        releaseOutput(output);
+      } finally {
+        if (segmenter)
+          await releaseSegmenter(segmenter);
+      }
+    })();
+    return run.releasing;
+  }
+  schedule(run) {
+    if (!this.active(run) || run.frame !== null || run.busy)
+      return;
+    run.frame = requestAnimationFrame(() => {
+      run.frame = null;
+      this.tick(run);
+    });
+  }
+  tick(run) {
+    if (!this.active(run) || run.busy)
+      return;
+    const video = this.video;
+    if (video.paused || video.ended || !(video.videoWidth > 0 && video.videoHeight > 0)) {
+      this.schedule(run);
+      return;
+    }
+    run.busy = true;
+    const work = (async () => {
+      try {
+        await Promise.resolve();
+        if (this.active(run))
+          await renderMask(run.output, video, this.layer, run.segmenter, this.config, () => this.active(run));
+      } catch (error) {
+        if (this.active(run))
+          console.error("Error in segmentBody:", error);
+      } finally {
+        run.busy = false;
+        if (!this.active(run))
+          await this.release(run);
+        else this.schedule(run);
+      }
+    })();
+    this.tail = work.catch((error) => console.warn("Failed to release danmuku mask resources:", error));
+  }
+  async start() {
+    if (this.closed || this.art.isDestroy)
+      return;
+    if (this.run?.running)
+      return this.run.started;
+    let cancel;
+    const cancelled = new Promise((resolve) => cancel = resolve);
+    const run = { running: true, initializing: true, busy: false, frame: null, segmenter: null, output: null, cancel, started: null };
+    const previous = this.tail;
+    this.run = run;
+    const initialize = (async () => {
+      try {
+        await previous;
+        if (!this.active(run))
+          return;
+        if (!this.video || !this.layer?.style)
+          throw new Error("Danmuku mask requires core video and danmuku template nodes");
+        run.segmenter = await loadSegmenter(this.config, () => this.active(run));
+        if (!this.active(run))
+          return;
+        if (!run.segmenter) {
+          run.running = false;
           return;
         }
-        const foregroundColor = { r: 255, g: 255, b: 255, a: 255 };
-        const backgroundColor = { r: 0, g: 0, b: 0, a: 255 };
-        const mask = await Ue(
-          segmentation,
-          foregroundColor,
-          backgroundColor,
-          config.drawContour,
-          config.foregroundThreshold
-        );
-        await Ne(canvas, $video, mask, config.opacity, config.maskBlurAmount);
-        const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
-        ctx.putImageData(makeWhiteTransparent(imageData), 0, 0);
-        $danmuku.style.maskImage = `url(${canvas.toDataURL()})`;
+        run.output = createOutput(this.layer);
       } catch (error) {
-        console.error("Error in segmentBody:", error);
+        if (this.active(run)) {
+          run.running = false;
+          throw error;
+        }
+      } finally {
+        run.initializing = false;
+        if (!this.active(run))
+          await this.release(run);
       }
-      animationFrameId = requestAnimationFrame(segmentBody);
+      if (this.active(run))
+        this.tick(run);
+    })();
+    this.tail = initialize.catch(() => {
+    });
+    run.started = Promise.race([initialize, cancelled]);
+    return run.started;
+  }
+  halt() {
+    const run = this.run;
+    if (run) {
+      run.running = false;
+      run.cancel();
+      if (run.frame !== null) {
+        cancelAnimationFrame(run.frame);
+        run.frame = null;
+      }
+      if (!run.initializing && !run.busy)
+        this.tail = this.release(run).catch((error) => console.warn("Failed to release danmuku mask resources:", error));
     }
+  }
+  stop() {
+    this.halt();
+    if (this.layer?.style)
+      this.layer.style.maskImage = "none";
+  }
+  destroy() {
+    if (this.closed)
+      return;
+    this.closed = true;
+    try {
+      this.stop();
+    } finally {
+      try {
+        this.art.off("ready", this.ready);
+      } finally {
+        this.art.off("destroy", this.destroy);
+      }
+    }
+  }
+}
+function artplayerPluginDanmukuMask(option = {}) {
+  return (art) => {
+    const { template: { $video, $danmuku } } = art;
+    const controller = new MaskController(art, maskConfig(option), { $video, $danmuku });
     async function startSegmentation() {
-      if (!isInitialized) {
-        await initSegmenter();
-      }
-      if (!canvas) {
-        createCanvas2();
-      }
-      setupDanmukuStyle();
-      segmentBody();
+      await controller.start();
     }
     function stopSegmentation() {
-      $danmuku.style.maskImage = "none";
-      if (animationFrameId) {
-        cancelAnimationFrame(animationFrameId);
-        animationFrameId = null;
-      }
+      controller.stop();
     }
-    art.on("ready", startSegmentation);
-    art.on("destroy", stopSegmentation);
     return {
       name: "artplayerPluginDanmukuMask",
       start: startSegmentation,

packages/artplayer-plugin-danmuku-mask/src/config.js

@@ -0,0 +1,14 @@
+export default function maskConfig(option) {
+  return {
+    solutionPath: option.solutionPath || 'https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation',
+    modelSelection: option.modelSelection || 1,
+    smoothSegmentation: option.smoothSegmentation !== undefined ? option.smoothSegmentation : true,
+    minDetectionConfidence: option.minDetectionConfidence || 0.5,
+    minTrackingConfidence: option.minTrackingConfidence || 0.5,
+    selfieMode: option.selfieMode || false,
+    drawContour: option.drawContour || false,
+    foregroundThreshold: option.foregroundThreshold || 0.5,
+    opacity: option.opacity || 1,
+    maskBlurAmount: option.maskBlurAmount || 3,
+  }
+}

packages/artplayer-plugin-danmuku-mask/src/controller.js

@@ -0,0 +1,188 @@
+import { createOutput, releaseOutput, renderMask } from './output'
+import { loadSegmenter, releaseSegmenter } from './sdk'
+
+export default class MaskController {
+  constructor(art, config, { $video, $danmuku }) {
+    this.art = art
+    this.config = config
+    this.video = $video
+    this.layer = $danmuku
+    this.closed = art.isDestroy
+    this.run = null
+    this.tail = Promise.resolve()
+    this.start = this.start.bind(this)
+    this.stop = this.stop.bind(this)
+    this.destroy = this.destroy.bind(this)
+    this.ready = () => {
+      this.start().catch(error => console.error('Failed to start danmuku mask:', error))
+    }
+    if (!this.closed) {
+      try {
+        art.on('destroy', this.destroy)
+        if (!this.closed)
+          art.on('ready', this.ready)
+      }
+      catch (error) {
+        this.closed = true
+        this.halt()
+        try {
+          art.off('ready', this.ready)
+        }
+        catch {}
+        try {
+          art.off('destroy', this.destroy)
+        }
+        catch {}
+        throw error
+      }
+    }
+  }
+
+  active(run) {
+    return run === this.run && run.running && !this.closed && !this.art.isDestroy
+  }
+
+  release(run) {
+    if (run.releasing)
+      return run.releasing
+    const segmenter = run.segmenter
+    run.segmenter = null
+    const output = run.output
+    run.output = null
+    run.releasing = (async () => {
+      await Promise.resolve()
+      try {
+        // Frame work has settled; an SDK close must not retain this independent bitmap.
+        releaseOutput(output)
+      }
+      finally {
+        if (segmenter)
+          await releaseSegmenter(segmenter)
+      }
+    })()
+    return run.releasing
+  }
+
+  schedule(run) {
+    if (!this.active(run) || run.frame !== null || run.busy)
+      return
+    run.frame = requestAnimationFrame(() => {
+      run.frame = null
+      this.tick(run)
+    })
+  }
+
+  tick(run) {
+    if (!this.active(run) || run.busy)
+      return
+    const video = this.video
+    if (video.paused || video.ended || !(video.videoWidth > 0 && video.videoHeight > 0)) {
+      this.schedule(run)
+      return
+    }
+    run.busy = true
+    const work = (async () => {
+      try {
+        // Reserve the work slot before SDK code can synchronously reenter start/stop.
+        await Promise.resolve()
+        if (this.active(run))
+          await renderMask(run.output, video, this.layer, run.segmenter, this.config, () => this.active(run))
+      }
+      catch (error) {
+        if (this.active(run))
+          console.error('Error in segmentBody:', error)
+      }
+      finally {
+        run.busy = false
+        if (!this.active(run))
+          await this.release(run)
+        else this.schedule(run)
+      }
+    })()
+    // A restart waits for uncancellable SDK work and disposal; it never overlaps it.
+    this.tail = work.catch(error => console.warn('Failed to release danmuku mask resources:', error))
+  }
+
+  async start() {
+    if (this.closed || this.art.isDestroy)
+      return
+    if (this.run?.running)
+      return this.run.started
+    let cancel
+    const cancelled = new Promise(resolve => cancel = resolve)
+    const run = { running: true, initializing: true, busy: false, frame: null, segmenter: null, output: null, cancel, started: null }
+    const previous = this.tail
+    this.run = run
+    const initialize = (async () => {
+      try {
+        await previous
+        if (!this.active(run))
+          return
+        if (!this.video || !this.layer?.style)
+          throw new Error('Danmuku mask requires core video and danmuku template nodes')
+        run.segmenter = await loadSegmenter(this.config, () => this.active(run))
+        if (!this.active(run))
+          return
+        if (!run.segmenter) {
+          run.running = false
+          return
+        }
+        run.output = createOutput(this.layer)
+      }
+      catch (error) {
+        if (this.active(run)) {
+          run.running = false
+          throw error
+        }
+      }
+      finally {
+        run.initializing = false
+        if (!this.active(run))
+          await this.release(run)
+      }
+      if (this.active(run))
+        this.tick(run)
+    })()
+    this.tail = initialize.catch(() => {})
+    // stop settles public start promptly; eventual SDK results remain observed by initialize.
+    run.started = Promise.race([initialize, cancelled])
+    return run.started
+  }
+
+  halt() {
+    const run = this.run
+    if (run) {
+      run.running = false
+      run.cancel()
+      if (run.frame !== null) {
+        cancelAnimationFrame(run.frame)
+        run.frame = null
+      }
+      if (!run.initializing && !run.busy)
+        this.tail = this.release(run).catch(error => console.warn('Failed to release danmuku mask resources:', error))
+    }
+  }
+
+  stop() {
+    this.halt()
+    if (this.layer?.style)
+      this.layer.style.maskImage = 'none'
+  }
+
+  destroy() {
+    if (this.closed)
+      return
+    this.closed = true
+    try {
+      this.stop()
+    }
+    finally {
+      try {
+        this.art.off('ready', this.ready)
+      }
+      finally {
+        this.art.off('destroy', this.destroy)
+      }
+    }
+  }
+}

packages/artplayer-plugin-danmuku-mask/src/index.js

@@ -1,155 +1,16 @@
-import * as bodySegmentation from '@tensorflow-models/body-segmentation'
-import * as tf from '@tensorflow/tfjs-core'
-import '@tensorflow/tfjs-backend-webgl'
-import '@tensorflow/tfjs-backend-cpu'
+import maskConfig from './config'
+import MaskController from './controller'
 
 export default function artplayerPluginDanmukuMask(option = {}) {
   return (art) => {
-    const {
-      template: { $video, $danmuku },
-    } = art
-
-    let segmenter = null
-    let canvas = null
-    let ctx = null
-    let animationFrameId = null
-    let isInitialized = false
-
-    const config = {
-      solutionPath: option.solutionPath || 'https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation',
-      modelSelection: option.modelSelection || 1,
-      smoothSegmentation: option.smoothSegmentation !== undefined ? option.smoothSegmentation : true,
-      minDetectionConfidence: option.minDetectionConfidence || 0.5,
-      minTrackingConfidence: option.minTrackingConfidence || 0.5,
-      selfieMode: option.selfieMode || false,
-      drawContour: option.drawContour || false,
-      foregroundThreshold: option.foregroundThreshold || 0.5,
-      opacity: option.opacity || 1,
-      maskBlurAmount: option.maskBlurAmount || 3,
-    }
-
-    async function initTensorFlow() {
-      try {
-        await tf.setBackend('webgl')
-      }
-      catch (error) {
-        console.warn('WebGL backend not available, falling back to CPU', error.message)
-        await tf.setBackend('cpu')
-      }
-    }
-
-    async function initSegmenter() {
-      await initTensorFlow()
-      const model = bodySegmentation.SupportedModels.MediaPipeSelfieSegmentation
-
-      const segmenterConfig = {
-        runtime: 'mediapipe',
-        modelType: 'general',
-        solutionPath: config.solutionPath,
-        modelSelection: config.modelSelection,
-        smoothSegmentation: config.smoothSegmentation,
-        minDetectionConfidence: config.minDetectionConfidence,
-        minTrackingConfidence: config.minTrackingConfidence,
-        selfieMode: config.selfieMode,
-      }
-
-      try {
-        segmenter = await bodySegmentation.createSegmenter(model, segmenterConfig)
-        isInitialized = true
-      }
-      catch (error) {
-        console.error('Error initializing segmenter:', error)
-        isInitialized = false
-      }
-    }
-
-    function setupDanmukuStyle() {
-      Object.assign($danmuku.style, {
-        maskMode: 'alpha',
-        maskSize: 'contain',
-        maskRepeat: 'no-repeat',
-        backgroundSize: 'contain',
-        backgroundRepeat: 'no-repeat',
-      })
-    }
-
-    function createCanvas() {
-      canvas = document.createElement('canvas')
-      ctx = canvas.getContext('2d')
-    }
-
-    function makeWhiteTransparent(imageData) {
-      const data = imageData.data
-      for (let i = 0; i < data.length; i += 4) {
-        if (data[i] > 250 && data[i + 1] > 250 && data[i + 2] > 250) {
-          data[i + 3] = 0
-        }
-      }
-      return imageData
-    }
-
-    async function segmentBody() {
-      if (!isInitialized || $video.paused || $video.ended) {
-        animationFrameId = requestAnimationFrame(segmentBody)
-        return
-      }
-
-      try {
-        canvas.width = $video.videoWidth
-        canvas.height = $video.videoHeight
-
-        const segmentation = await segmenter.segmentPeople($video)
-
-        if (!segmentation || segmentation.length === 0) {
-          animationFrameId = requestAnimationFrame(segmentBody)
-          return
-        }
-
-        const foregroundColor = { r: 255, g: 255, b: 255, a: 255 }
-        const backgroundColor = { r: 0, g: 0, b: 0, a: 255 }
-        const mask = await bodySegmentation.toBinaryMask(
-          segmentation,
-          foregroundColor,
-          backgroundColor,
-          config.drawContour,
-          config.foregroundThreshold,
-        )
-
-        await bodySegmentation.drawMask(canvas, $video, mask, config.opacity, config.maskBlurAmount)
-
-        const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height)
-        ctx.putImageData(makeWhiteTransparent(imageData), 0, 0)
-        $danmuku.style.maskImage = `url(${canvas.toDataURL()})`
-      }
-      catch (error) {
-        console.error('Error in segmentBody:', error)
-      }
-
-      animationFrameId = requestAnimationFrame(segmentBody)
-    }
-
+    const { template: { $video, $danmuku } } = art
+    const controller = new MaskController(art, maskConfig(option), { $video, $danmuku })
     async function startSegmentation() {
-      if (!isInitialized) {
-        await initSegmenter()
-      }
-      if (!canvas) {
-        createCanvas()
-      }
-      setupDanmukuStyle()
-      segmentBody()
+      await controller.start()
     }
-
     function stopSegmentation() {
-      $danmuku.style.maskImage = 'none'
-      if (animationFrameId) {
-        cancelAnimationFrame(animationFrameId)
-        animationFrameId = null
-      }
+      controller.stop()
     }
-
-    art.on('ready', startSegmentation)
-    art.on('destroy', stopSegmentation)
-
     return {
       name: 'artplayerPluginDanmukuMask',
       start: startSegmentation,

packages/artplayer-plugin-danmuku-mask/src/output.js

@@ -0,0 +1,65 @@
+import { drawMask, toBinaryMask } from './sdk'
+
+export function makeWhiteTransparent(imageData) {
+  const data = imageData.data
+  for (let i = 0; i < data.length; i += 4) {
+    if (data[i] > 250 && data[i + 1] > 250 && data[i + 2] > 250)
+      data[i + 3] = 0
+  }
+  return imageData
+}
+
+export function createOutput(layer) {
+  const canvas = document.createElement('canvas')
+  try {
+    const ctx = canvas.getContext('2d')
+    if (!ctx)
+      throw new Error('Danmuku mask requires a 2D canvas context')
+    Object.assign(layer.style, {
+      maskMode: 'alpha',
+      maskSize: 'contain',
+      maskRepeat: 'no-repeat',
+      backgroundSize: 'contain',
+      backgroundRepeat: 'no-repeat',
+    })
+    return { canvas, ctx }
+  }
+  catch (error) {
+    canvas.width = 0
+    canvas.height = 0
+    throw error
+  }
+}
+
+export async function renderMask(output, video, layer, segmenter, config, active) {
+  const { canvas, ctx } = output
+  canvas.width = video.videoWidth
+  canvas.height = video.videoHeight
+  const segmentation = await segmenter.segmentPeople(video)
+  if (!active() || !segmentation || !segmentation.length)
+    return
+  const mask = await toBinaryMask(
+    segmentation,
+    { r: 255, g: 255, b: 255, a: 255 },
+    { r: 0, g: 0, b: 0, a: 255 },
+    config.drawContour,
+    config.foregroundThreshold,
+  )
+  if (!active())
+    return
+  await drawMask(canvas, video, mask, config.opacity, config.maskBlurAmount)
+  if (!active())
+    return
+  const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height)
+  ctx.putImageData(makeWhiteTransparent(imageData), 0, 0)
+  const url = canvas.toDataURL()
+  if (active())
+    layer.style.maskImage = `url(${url})`
+}
+
+export function releaseOutput(output) {
+  if (output) {
+    output.canvas.width = 0
+    output.canvas.height = 0
+  }
+}

packages/artplayer-plugin-danmuku-mask/src/sdk.js

@@ -0,0 +1,47 @@
+import * as bodySegmentation from '@tensorflow-models/body-segmentation'
+import * as tf from '@tensorflow/tfjs-core'
+import '@tensorflow/tfjs-backend-webgl'
+import '@tensorflow/tfjs-backend-cpu'
+
+export async function loadSegmenter(config, active) {
+  try {
+    await tf.setBackend('webgl')
+  }
+  catch (error) {
+    if (!active())
+      return null
+    console.warn('WebGL backend not available, falling back to CPU', error.message)
+    await tf.setBackend('cpu')
+  }
+  if (!active())
+    return null
+  try {
+    return await bodySegmentation.createSegmenter(bodySegmentation.SupportedModels.MediaPipeSelfieSegmentation, {
+      runtime: 'mediapipe',
+      modelType: 'general',
+      solutionPath: config.solutionPath,
+      modelSelection: config.modelSelection,
+      smoothSegmentation: config.smoothSegmentation,
+      minDetectionConfidence: config.minDetectionConfidence,
+      minTrackingConfidence: config.minTrackingConfidence,
+      selfieMode: config.selfieMode,
+    })
+  }
+  catch (error) {
+    if (active())
+      console.error('Error initializing segmenter:', error)
+    return null
+  }
+}
+
+export async function releaseSegmenter(segmenter) {
+  try {
+    await segmenter.dispose()
+  }
+  catch (error) {
+    console.warn('Failed to dispose danmuku mask segmenter:', error)
+  }
+}
+
+export const toBinaryMask = (...args) => bodySegmentation.toBinaryMask(...args)
+export const drawMask = (...args) => bodySegmentation.drawMask(...args)