mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-11 02:46:16 -08:00
docs(skills): refresh Nx and SQLite ownership
This commit is contained in:
1 parent
6bb7e764d8
commit
fa6f988dd3
4 files changed
+333
-159
No files matched your search
@@ -1,44 +1,78 @@
|
||||
---
|
||||
name: iptvnator-nx-architecture
|
||||
description: Repository-specific Nx monorepo structure, library placement rules, scoped path aliases, and migration guardrails for portal/workspace/app code.
|
||||
description: Use when deciding where IPTVnator code belongs, creating or moving Nx projects, changing scoped aliases or tags, editing lint targets, or validating module boundaries.
|
||||
---
|
||||
|
||||
# IPTVnator Nx Architecture
|
||||
|
||||
Use this skill when deciding where code belongs, extracting libraries, changing imports, editing project tags, or refactoring portal/workspace/app boundaries.
|
||||
## Discover Before Deciding
|
||||
|
||||
## Project Shape
|
||||
|
||||
- `apps/web`: Angular renderer application.
|
||||
- `apps/electron-backend`: Electron main process and native/runtime integration.
|
||||
- `apps/*-e2e`: Playwright E2E projects.
|
||||
- `apps/*-mock-server`: local development and E2E mock servers.
|
||||
- `libs/playlist/*`: M3U/import/shared playlist functionality.
|
||||
- `libs/portal/*`: Xtream, Stalker, and provider-neutral portal functionality.
|
||||
- `libs/workspace/*`: workspace shell and dashboard.
|
||||
- `libs/ui/*`: provider-neutral UI, playback, EPG, remote control, and pipes.
|
||||
- `libs/shared/*`: contracts, database, and pure utility code.
|
||||
|
||||
## Import Policy
|
||||
|
||||
- Use scoped aliases from `tsconfig.base.json`, for example `@iptvnator/services` and `@iptvnator/shared/interfaces`.
|
||||
- Do not reintroduce legacy bare aliases such as `services`, `components`, or `shared-interfaces`.
|
||||
- Prefer library public APIs (`src/index.ts`) over deep imports unless a sub-entrypoint is explicitly configured.
|
||||
- Keep app-only code in `apps/*`; shared behavior belongs in a domain library.
|
||||
|
||||
## Tag Policy
|
||||
|
||||
- Every project should have `scope:*`, `domain:*`, and `type:*` tags.
|
||||
- Use `type:feature` for route/component orchestration, `type:data-access` for stores/API/persistence, `type:ui` for reusable components, and `type:util` for pure helpers/contracts.
|
||||
- Keep `type:data-access` from depending on `type:feature` or `type:ui`.
|
||||
- Add or update `@nx/enforce-module-boundaries` constraints when introducing new tag families.
|
||||
|
||||
## Validation
|
||||
In a fresh worktree, install first:
|
||||
|
||||
```bash
|
||||
pnpm install --frozen-lockfile
|
||||
pnpm nx show projects
|
||||
pnpm nx lint <project>
|
||||
pnpm nx test <project>
|
||||
pnpm nx show project <name>
|
||||
pnpm nx show projects --withTarget test
|
||||
pnpm nx show projects --withTarget e2e
|
||||
```
|
||||
|
||||
When dependencies are not installed in a fresh worktree, run `pnpm install --frozen-lockfile` first.
|
||||
Discovery is authoritative. The current app groups are `web`,
|
||||
`electron-backend`, `web-backend`, `remote-control-web`, `website`, `web-e2e`,
|
||||
`electron-backend-e2e`, `stalker-mock-server`, and `xtream-mock-server`. Current
|
||||
tool projects are `eslint-tools`, `packaging`, `release-tools`, and
|
||||
`repository-skills`.
|
||||
|
||||
## Place Code by Ownership
|
||||
|
||||
- `apps/`: runtime, development, E2E, and mock-server applications.
|
||||
- `tools/`: repository automation; tag its projects `scope:tools`.
|
||||
- `libs/`: domain libraries.
|
||||
- `type:feature`: route and screen orchestration.
|
||||
- `type:ui`: reusable visual components.
|
||||
- `type:data-access`: injectable state, API, persistence, or orchestration.
|
||||
- `type:util`: the destination for **new pure** helpers and contracts only.
|
||||
|
||||
Provider-neutral collection services that coordinate persistence belong in
|
||||
`libs/portal/shared/data-access`; pure collection helpers belong in
|
||||
`libs/portal/shared/util`; reusable views belong in
|
||||
`libs/portal/shared/ui`. Existing injectable/stateful services under a `util`
|
||||
path are legacy debt, not placement precedent.
|
||||
|
||||
## Preserve Boundaries
|
||||
|
||||
Every project keeps one `scope:*`, `domain:*`, and `type:*` tag. Enforced type
|
||||
directions are:
|
||||
|
||||
| Source | Allowed dependencies |
|
||||
| ----------------- | ------------------------------ |
|
||||
| app, E2E, dev-app | feature, UI, data-access, util |
|
||||
| website | UI, util |
|
||||
| feature | feature, UI, data-access, util |
|
||||
| UI | UI, data-access, util |
|
||||
| data-access | data-access, util |
|
||||
| util | util |
|
||||
|
||||
Domain constraints are additive. Never weaken either constraint to solve a
|
||||
placement problem. Preserve the documented `workspace-shell-util` path/tag
|
||||
exception: its injectable services require `type:data-access` so app routes can
|
||||
eagerly import them without loading the lazy shell feature.
|
||||
|
||||
Use aliases from `tsconfig.base.json` and public `src/index.ts` barrels. Do not
|
||||
add legacy bare aliases or deep imports. For a buildable library that has a
|
||||
local `package.json`, its package name must match its scoped alias.
|
||||
|
||||
## Validate the Change
|
||||
|
||||
Production TypeScript targets under 300 lines; 400 is the hard limit. Tests are
|
||||
limited to 1200. The legacy baseline may only shrink. See
|
||||
`tools/eslint/max-lines-config.mjs` and regenerate after a split with
|
||||
`tools/eslint/generate-max-lines-baseline.mjs`; never add a new baseline entry.
|
||||
|
||||
Quote recursive globs in command-based lint targets, for example
|
||||
`eslint "apps/<project>/**/*.ts"`, then compare coverage with
|
||||
`find apps/<project> -name '*.ts' | wc -l`.
|
||||
|
||||
Discover targets, then run affected lint/test/build targets and the closest
|
||||
available E2E target; do not invent targets. Canonical guidance:
|
||||
`docs/architecture/nx-workspace-boundaries.md`.
|
||||
@@ -1,31 +1,69 @@
|
||||
---
|
||||
name: iptvnator-sqlite-db-worker
|
||||
description: Repository-specific guidance for Electron non-EPG SQLite worker boundaries, request-scoped DB progress events, and validation for slow DB operations.
|
||||
description: Use when changing Electron SQLite IPC, database-worker operations, request-scoped progress or cancellation, worker packaging, or runtime verification of non-EPG database work.
|
||||
---
|
||||
|
||||
# IPTVnator SQLite DB Worker
|
||||
|
||||
Use this skill when changing Electron SQLite operations, worker-backed database flows, DB progress events, or Xtream/playlist import/search/delete persistence.
|
||||
## Ownership and Flow
|
||||
|
||||
## Ownership
|
||||
- Renderer service: `libs/services/src/lib/database-electron.service.ts`
|
||||
- Preload API: `apps/electron-backend/src/app/api/main.preload.ts`
|
||||
- IPC handlers: `apps/electron-backend/src/app/events/database/`
|
||||
- Client: `apps/electron-backend/src/app/services/database-worker-client.ts`
|
||||
- Protocol: `apps/electron-backend/src/app/workers/database-worker.types.ts`
|
||||
- Thin dispatcher: `apps/electron-backend/src/app/workers/database.worker.ts`
|
||||
- Connection: `apps/electron-backend/src/app/workers/database.worker-connection.ts`
|
||||
- Runtime paths: `apps/electron-backend/src/app/workers/worker-runtime-paths.ts`
|
||||
- SQL operations: `apps/electron-backend/src/app/database/operations/`
|
||||
- Shared schema: `libs/shared/database/src/lib/schema.ts`
|
||||
- Bundler: `apps/electron-backend/build-worker.js`
|
||||
- Canonical guide: `docs/architecture/sqlite-db-worker.md`
|
||||
|
||||
- Runtime worker client: `apps/electron-backend/src/app/services/database-worker-client.ts`
|
||||
- Worker protocol: `apps/electron-backend/src/app/workers/database-worker.types.ts`
|
||||
- Worker dispatcher: `apps/electron-backend/src/app/workers/database.worker.ts`
|
||||
- SQL operation modules: `apps/electron-backend/src/app/database/operations/`
|
||||
- Shared schema/path helpers: `libs/shared/database/src/`
|
||||
- Architecture doc: `docs/architecture/sqlite-db-worker.md`
|
||||
The chain is renderer `DatabaseService` → preload IPC → main event handler →
|
||||
`DatabaseWorkerClient` and its request protocol → worker dispatcher →
|
||||
worker connection → operation module → shared
|
||||
`@iptvnator/shared/database/schema`, then response or request-scoped event back
|
||||
to the originating renderer. Keep SQL-heavy logic in operation modules and the
|
||||
worker entrypoint focused on dispatch/orchestration.
|
||||
|
||||
## Rules
|
||||
The build script produces three bundles: EPG parser, database, and playlist
|
||||
refresh. EPG parsing stays in its dedicated worker. Do not silently migrate
|
||||
lightweight download handlers or EPG-specific main-process query/mapping/fetch
|
||||
handlers as part of unrelated database work.
|
||||
|
||||
- Keep heavy non-EPG SQLite work off the Electron main thread.
|
||||
- Keep SQL-heavy logic in operation modules; keep the worker entrypoint as dispatcher/orchestration.
|
||||
- Emit request-scoped `DB_OPERATION_EVENT` progress for long-running operations.
|
||||
- Preserve cancellation as cooperative and chunk-based.
|
||||
- Do not break existing preload API method names without a coordinated renderer migration.
|
||||
## Identity, Progress, and Cancellation
|
||||
|
||||
## Validation
|
||||
`requestId` is generated for every client request and correlates worker
|
||||
event/response transport. `operationId` is the renderer-visible identity for
|
||||
long-operation progress and cooperative cancellation.
|
||||
|
||||
- Run `pnpm nx test electron-backend` for worker/client/operation changes.
|
||||
- Run targeted Electron E2E for import, search, delete, backup/restore, or downloads flows when touched.
|
||||
- Use `IPTVNATOR_TRACE_DB=1` or `IPTVNATOR_TRACE_SQL=1` for manual debugging.
|
||||
Tracked operations are save content, delete Xtream content, restore Xtream user
|
||||
data, delete playlist, and delete all playlists. The first four are
|
||||
cancellable. Delete-all is deliberately tracked with `cancellable: false`.
|
||||
|
||||
Cancellation is cooperative at chunk checkpoints. Committed chunks remain
|
||||
committed and the request finally rejects with `AbortError`. For the
|
||||
operation/busy lifecycle, only a terminal completed/error/cancelled event
|
||||
settles UI state; a cancel-requested flag may update immediately.
|
||||
|
||||
Inside synchronous `better-sqlite3` transactions, prepared writes must use
|
||||
`.run()`. `.execute()` defers work and can commit a silent no-op.
|
||||
|
||||
## Rebuild and Verify
|
||||
|
||||
After worker source changes:
|
||||
|
||||
```bash
|
||||
pnpm nx test electron-backend
|
||||
pnpm nx run electron-backend:build-worker
|
||||
stat dist/apps/electron-backend/workers/database.worker.js
|
||||
```
|
||||
|
||||
Confirm the artifact timestamp, restart Electron, then run the closest
|
||||
Electron E2E or CDP workflow.
|
||||
|
||||
For SQL output, `IPTVNATOR_TRACE_DB=1` and `IPTVNATOR_TRACE_SQL=1` emit only
|
||||
fixed, allowlisted statement types through the shared redacting summary in
|
||||
`libs/shared/logging/src/lib/sql-trace-summary.ts`; never log expanded SQL or
|
||||
bound values. DB transport traces remain separately redacted.
|
||||
@@ -1,100 +1,130 @@
|
||||
# Nx Workspace Boundaries
|
||||
|
||||
This document records the current monorepo boundary conventions for IPTVnator.
|
||||
This document records the current monorepo placement, tagging, and validation
|
||||
contract for IPTVnator. Nx discovery is the canonical project inventory; avoid
|
||||
copying an exhaustive project list into documentation.
|
||||
|
||||
## Fresh Worktree Bootstrap
|
||||
## Fresh Worktree Bootstrap and Discovery
|
||||
|
||||
Install dependencies before using Nx discovery or targets:
|
||||
Install dependencies before relying on Nx:
|
||||
|
||||
```bash
|
||||
pnpm install --frozen-lockfile
|
||||
pnpm nx show projects
|
||||
```
|
||||
|
||||
`pnpm nx show projects` depends on the workspace-local Nx packages under
|
||||
`node_modules`. In a fresh worktree without dependencies it will fail before it
|
||||
can inspect project metadata.
|
||||
`pnpm nx show projects` requires the workspace-local Nx packages in
|
||||
`node_modules`. Inspect project ownership and available validation targets
|
||||
before choosing commands:
|
||||
|
||||
```bash
|
||||
pnpm nx show project <name>
|
||||
pnpm nx show projects --withTarget test
|
||||
pnpm nx show projects --withTarget e2e
|
||||
```
|
||||
|
||||
Do not invent a `test`, `build`, or `e2e` target because a similarly named
|
||||
project has one. Run affected lint/test/build targets that exist and the closest
|
||||
available E2E target for the changed behavior.
|
||||
|
||||
## Placement Decision
|
||||
|
||||
- `apps/` owns runtime applications, development servers, E2E applications,
|
||||
and provider mock servers.
|
||||
- `libs/` owns reusable code grouped by product domain and architectural role.
|
||||
- `tools/` owns repository automation such as lint, packaging, release, and
|
||||
repository-skill validation. Nx projects there use `scope:tools`.
|
||||
|
||||
Inside `libs/`, choose the role before the path:
|
||||
|
||||
- `type:feature` owns routes, screens, and feature orchestration.
|
||||
- `type:ui` owns reusable visual components.
|
||||
- `type:data-access` owns injectable state, API access, persistence, and
|
||||
orchestration.
|
||||
- `type:util` is the destination for new pure helpers and contracts only.
|
||||
|
||||
For example, provider-neutral collection services that coordinate favorites,
|
||||
recents, EPG, or playback persistence belong in
|
||||
`libs/portal/shared/data-access`. Pure collection types and transformations stay
|
||||
in `libs/portal/shared/util`, while reusable collection views stay in
|
||||
`libs/portal/shared/ui`. Existing injectable or stateful services in a `util`
|
||||
path are legacy debt, not precedent for new placement.
|
||||
|
||||
## Project Tags
|
||||
|
||||
Every Nx project should carry three tag families in `project.json`:
|
||||
Every Nx project keeps one tag from each family in `project.json`:
|
||||
|
||||
1. `scope:*` - ownership area, for example `scope:portal`, `scope:workspace`,
|
||||
`scope:shared`, `scope:electron`, `scope:e2e`, or `scope:dev-tools`.
|
||||
2. `domain:*` - product/runtime domain, for example `domain:xtream`,
|
||||
`domain:stalker`, `domain:m3u`, `domain:playback`, `domain:web`, or
|
||||
`domain:shared-runtime`.
|
||||
3. `type:*` - architectural role, for example `type:app`, `type:e2e`,
|
||||
`type:dev-app`, `type:feature`, `type:ui`, `type:data-access`,
|
||||
`type:util`, `type:tool`, or `type:website`.
|
||||
1. `scope:*` records ownership, such as `scope:portal`, `scope:workspace`,
|
||||
`scope:shared`, `scope:electron`, `scope:e2e`, or `scope:tools`.
|
||||
2. `domain:*` records the product/runtime domain.
|
||||
3. `type:*` records the architectural role.
|
||||
|
||||
`eslint.config.mjs` uses these tags with `@nx/enforce-module-boundaries`.
|
||||
When adding a project, choose tags before adding imports so dependency direction
|
||||
is clear from the start.
|
||||
`eslint.config.mjs` enforces these type directions:
|
||||
|
||||
## Import Aliases
|
||||
| Source tag | Allowed dependency type tags |
|
||||
| ------------------ | ------------------------------ |
|
||||
| `type:app` | feature, UI, data-access, util |
|
||||
| `type:e2e` | feature, UI, data-access, util |
|
||||
| `type:dev-app` | feature, UI, data-access, util |
|
||||
| `type:website` | UI, util |
|
||||
| `type:feature` | feature, UI, data-access, util |
|
||||
| `type:ui` | UI, data-access, util |
|
||||
| `type:data-access` | data-access, util |
|
||||
| `type:util` | util |
|
||||
|
||||
Use scoped `@iptvnator/*` aliases from `tsconfig.base.json`.
|
||||
Domain constraints in the same rule are additive to type constraints. If an
|
||||
import violates either family, move the contract or implementation to its
|
||||
proper owner instead of weakening a constraint.
|
||||
|
||||
Examples:
|
||||
`workspace-shell-util` is a deliberate path/tag exception:
|
||||
`libs/workspace/shell/util` is tagged `type:data-access` because it exports
|
||||
injectable services that depend on `@iptvnator/services`. The web app imports
|
||||
those services eagerly from `apps/web/src/app/app.routes.ts` without pulling
|
||||
the lazy workspace shell feature into the initial bundle.
|
||||
|
||||
```ts
|
||||
import { SettingsStore } from '@iptvnator/services';
|
||||
import { Playlist } from '@iptvnator/shared/interfaces';
|
||||
import { DialogService } from '@iptvnator/ui/components';
|
||||
## Import Aliases and Public APIs
|
||||
|
||||
Use scoped aliases from `tsconfig.base.json` and expose public imports through a
|
||||
library's `src/index.ts`. Do not introduce legacy bare aliases such as
|
||||
`services`, `components`, `shared-interfaces`, or `database`, and avoid deep
|
||||
imports unless a sub-entrypoint is explicitly configured.
|
||||
|
||||
For a buildable library that has a local `package.json`, its `name` must match
|
||||
the scoped alias. Nx uses that package name when rewriting buildable dependency
|
||||
paths to `dist/` during `@nx/js:tsc` builds.
|
||||
|
||||
## TypeScript File Size
|
||||
|
||||
`tools/eslint/max-lines-config.mjs` is the single source of truth:
|
||||
|
||||
- production TypeScript should stay below 300 lines and has a hard maximum of
|
||||
400;
|
||||
- tests, E2E specs, and E2E infrastructure have a maximum of 1200;
|
||||
- blank lines and comments are not counted.
|
||||
|
||||
Pre-existing violations live in
|
||||
`tools/eslint/max-lines-baseline.mjs`. That baseline may only shrink. After
|
||||
splitting a baselined file, run
|
||||
`node tools/eslint/generate-max-lines-baseline.mjs`; never add a new file to the
|
||||
baseline. A genuinely inseparable new file needs a justified file-wide
|
||||
directive, which the generator deliberately skips.
|
||||
|
||||
## Command-Based Lint Targets
|
||||
|
||||
Quote recursive globs so POSIX and Windows hosts lint the same files:
|
||||
|
||||
```bash
|
||||
eslint "apps/<project>/**/*.ts"
|
||||
find apps/<project> -name '*.ts' | wc -l
|
||||
```
|
||||
|
||||
Do not introduce legacy bare aliases such as:
|
||||
|
||||
- `components`
|
||||
- `m3u-state`
|
||||
- `m3u-utils`
|
||||
- `services`
|
||||
- `shared-interfaces`
|
||||
- `shared-portals`
|
||||
- `remote-control`
|
||||
- `database`
|
||||
- `database-schema`
|
||||
- `database-path-utils`
|
||||
- `workspace-dashboard-feature`
|
||||
- `workspace-dashboard-data-access`
|
||||
|
||||
The lint config blocks these aliases so new code uses the same visible
|
||||
namespace and ownership convention.
|
||||
|
||||
Buildable libraries that have a local `package.json` should use the same public
|
||||
name as their scoped alias. Nx uses `package.json.name` when it rewrites
|
||||
buildable dependency paths to `dist/` during `@nx/js:tsc` builds.
|
||||
|
||||
## Dependency Direction
|
||||
|
||||
- `type:feature` may use `type:feature`, `type:ui`, `type:data-access`, and
|
||||
`type:util`.
|
||||
- `type:ui` may use `type:ui`, `type:data-access`, and `type:util`.
|
||||
- `type:data-access` may use `type:data-access` and `type:util`.
|
||||
- `type:util` may use only `type:util`.
|
||||
|
||||
If a change needs a dependency in the opposite direction, move the shared
|
||||
contract into a lower-level library instead of weakening boundaries.
|
||||
|
||||
Portal collection orchestration that reads/writes favorites, recent items, live
|
||||
playback, or EPG data belongs in `libs/portal/shared/data-access`, not
|
||||
`libs/portal/shared/util`. That keeps pure collection helpers importable by
|
||||
Xtream/Stalker data-access libraries while allowing shared UI to use
|
||||
provider-specific collection services without creating cycles.
|
||||
|
||||
Note: `workspace-shell-util` (`libs/workspace/shell/util`) is tagged
|
||||
`type:data-access` despite its path. It exports injectable services such as
|
||||
`WorkspaceStartupPreferencesService` that depend on `@iptvnator/services`, and
|
||||
it must stay eagerly importable from `apps/web/src/app/app.routes.ts` without
|
||||
pulling the lazy-loaded workspace shell feature bundle into the initial chunk.
|
||||
An unquoted `**` can expand to a shallow subset on POSIX while still returning
|
||||
success. After editing such a target, compare ESLint's linted-file count with
|
||||
the `find` count.
|
||||
|
||||
## CI Enforcement
|
||||
|
||||
The `Lint` job in `.github/workflows/ci.yml` runs
|
||||
`pnpm nx affected --target=lint` on PRs and
|
||||
`pnpm nx run-many --target=lint --all` on master pushes, so
|
||||
`@nx/enforce-module-boundaries` violations, legacy bare-alias imports, and
|
||||
`max-lines` violations fail CI. Root config or lockfile changes mark every
|
||||
project affected, so the boundary rules cannot be dodged on PRs. Run
|
||||
`pnpm run lint` locally before pushing.
|
||||
The CI lint job runs affected projects on pull requests and all projects on
|
||||
master pushes. Root config or lockfile changes affect every project, so module
|
||||
boundaries, legacy-alias restrictions, and max-lines enforcement apply across
|
||||
the workspace.
|
||||
@@ -10,7 +10,9 @@ Related:
|
||||
|
||||
## Summary
|
||||
|
||||
- Heavy non-EPG SQLite work no longer runs on Electron's main thread.
|
||||
- Heavy non-EPG SQLite work no longer runs on Electron's main thread. The
|
||||
explicitly lightweight download and EPG-specific SQLite handlers remain in
|
||||
main.
|
||||
- A dedicated long-lived database worker now handles the slow Xtream and
|
||||
playlist database operations that were freezing the UI.
|
||||
- Renderer APIs stay stable. The main change is that progress and long-running
|
||||
@@ -28,6 +30,14 @@ The worker cutover addresses three concrete problems:
|
||||
|
||||
## Current Ownership
|
||||
|
||||
### Renderer and preload boundary
|
||||
|
||||
These files own the renderer-facing database service and stable Electron
|
||||
bridge:
|
||||
|
||||
1. `libs/services/src/lib/database-electron.service.ts`
|
||||
2. `apps/electron-backend/src/app/api/main.preload.ts`
|
||||
|
||||
### Main-process runtime wiring
|
||||
|
||||
These files own worker lifecycle and IPC bridging:
|
||||
@@ -48,7 +58,7 @@ These files own the worker protocol and the SQLite work itself:
|
||||
3. `apps/electron-backend/src/app/workers/database.worker-connection.ts`
|
||||
4. `apps/electron-backend/src/app/workers/worker-runtime-paths.ts`
|
||||
|
||||
### Pure database operation modules
|
||||
### Database operation and support modules
|
||||
|
||||
Keep SQL-heavy logic here so the worker entry remains a thin dispatcher:
|
||||
|
||||
@@ -63,25 +73,53 @@ Keep SQL-heavy logic here so the worker entry remains a thin dispatcher:
|
||||
9. `apps/electron-backend/src/app/database/operations/title-match.operations.ts`
|
||||
10. `apps/electron-backend/src/app/database/operations/tmdb.operations.ts`
|
||||
11. `apps/electron-backend/src/app/database/operations/epg-mapping.operations.ts`
|
||||
12. `apps/electron-backend/src/app/database/operations/title-sources.operations.ts`
|
||||
13. `apps/electron-backend/src/app/database/operations/vod-source-pin.operations.ts`
|
||||
|
||||
(plus the shared cancellation helper `operation-control.ts` in the same directory)
|
||||
Focused helpers in the same directory keep the operation modules and dispatcher
|
||||
small:
|
||||
|
||||
1. `content-search.util.ts`
|
||||
2. `title-token-glob.ts`
|
||||
3. `operation-control.ts`
|
||||
4. `performance-phase-capture.ts`
|
||||
5. `xtream-content-operation-steps.ts`
|
||||
|
||||
Worker operations use the shared table contract in
|
||||
`libs/shared/database/src/lib/schema.ts`, imported by worker code through
|
||||
`@iptvnator/shared/database/schema`.
|
||||
|
||||
## Worker Architecture
|
||||
|
||||
### Request flow
|
||||
|
||||
1. Renderer calls the existing preload API such as `window.electron.dbSaveContent`.
|
||||
2. `ipcMain.handle(...)` in the Electron backend builds a payload and delegates
|
||||
to `DatabaseWorkerClient`.
|
||||
3. `DatabaseWorkerClient` lazily starts one long-lived `worker_threads` worker
|
||||
1. Renderer `DatabaseService` calls the stable preload API, such as
|
||||
`window.electron.dbSaveContent`.
|
||||
2. The preload invokes an IPC channel owned by the focused event modules under
|
||||
`apps/electron-backend/src/app/events/database/`.
|
||||
3. `ipcMain.handle(...)` builds a payload and delegates to
|
||||
`DatabaseWorkerClient`.
|
||||
4. `DatabaseWorkerClient` lazily starts one long-lived `worker_threads` worker
|
||||
and correlates requests with a generated `requestId`.
|
||||
4. The worker executes SQLite work and sends back either:
|
||||
5. The protocol reaches the worker dispatcher, which obtains the worker
|
||||
connection and delegates SQL work to an operation module using the shared
|
||||
schema.
|
||||
6. The worker sends back either:
|
||||
1. `ready`
|
||||
2. `event`
|
||||
3. `response`
|
||||
5. The main process resolves the IPC request and forwards worker events back to
|
||||
7. The main process resolves the IPC request and forwards worker events back to
|
||||
the originating renderer process.
|
||||
|
||||
`requestId` and `operationId` have different scopes:
|
||||
|
||||
1. `DatabaseWorkerClient` generates a fresh `requestId` for every request. It
|
||||
correlates worker `event` and `response` messages with the pending main-side
|
||||
transport request and is not renderer-visible operation state.
|
||||
2. A renderer supplies an `operationId` for tracked long-running work. Progress
|
||||
events and cooperative cancellation use that stable identity across the
|
||||
renderer, preload, main process, and worker.
|
||||
|
||||
### Why one long-lived worker
|
||||
|
||||
- It avoids worker startup cost on every search/delete/import.
|
||||
@@ -309,6 +347,11 @@ Current shipped operation names:
|
||||
4. `delete-playlist`
|
||||
5. `delete-all-playlists`
|
||||
|
||||
All five are tracked. Save content, delete Xtream content, restore Xtream user
|
||||
data, and delete playlist are cancellable. Delete all playlists deliberately
|
||||
uses `cancellable: false`: its renderer-visible progress is tracked, but a
|
||||
cancel request does not interrupt it.
|
||||
|
||||
The event is forwarded to the renderer as `DB_OPERATION_EVENT`.
|
||||
|
||||
### Cancellation contract
|
||||
@@ -329,7 +372,10 @@ If a worker operation is canceled:
|
||||
3. the UI clears its busy state without treating the operation as success
|
||||
|
||||
Cancellation is cooperative and chunk-based. Already committed SQLite batches
|
||||
stay committed.
|
||||
stay committed. For an operation's busy lifecycle, only the terminal
|
||||
`completed`, `error`, or `cancelled` event settles UI state. The UI may set a
|
||||
separate cancel-requested flag immediately so the cancel action cannot be
|
||||
clicked twice while it waits for the authoritative terminal event.
|
||||
|
||||
With the exact `IPTVNATOR_PERF_WORKER_PROFILING=1` opt-in, receipt of a cancel
|
||||
for a correlated active request also emits a
|
||||
@@ -370,8 +416,27 @@ falls back to the legacy progress API if the newer event channel is missing.
|
||||
|
||||
## Migrated Operations
|
||||
|
||||
The worker now owns all heavy non-EPG SQLite paths plus the remaining portal
|
||||
state handlers that still used direct main-thread SQLite access.
|
||||
The worker owns heavy non-EPG SQLite paths and portal state operations that
|
||||
would otherwise block Electron main.
|
||||
|
||||
### Deliberate direct-main exceptions
|
||||
|
||||
Lightweight work that coordinates main-process runtime or remains
|
||||
EPG-specific is not migrated incidentally:
|
||||
|
||||
1. `apps/electron-backend/src/app/events/database/downloads.events.ts` keeps
|
||||
small download-row reads/writes beside native dialogs, filesystem cleanup,
|
||||
and the main-process download runtime.
|
||||
2. `apps/electron-backend/src/app/events/database/epg-db.events.ts` keeps the
|
||||
EPG programme-search IPC handler.
|
||||
3. `apps/electron-backend/src/app/events/epg-fetch.service.ts`,
|
||||
`apps/electron-backend/src/app/events/epg-mapping.service.ts`, and
|
||||
`apps/electron-backend/src/app/events/epg-query.service.ts` keep EPG
|
||||
freshness, mapping, and lookup behavior in their focused main-process
|
||||
owners, while EPG parsing/import remains in its dedicated worker.
|
||||
|
||||
Do not move these paths as part of unrelated database work. Reassess the
|
||||
boundary if a handler becomes heavy enough to block the main process.
|
||||
|
||||
### Categories
|
||||
|
||||
@@ -613,8 +678,8 @@ Current implementation paths:
|
||||
2. `libs/portal/xtream/data-access/src/lib/with-favorites.feature.ts`
|
||||
3. `libs/portal/xtream/data-access/src/lib/with-recent-items.ts`
|
||||
4. `libs/portal/xtream/feature/src/lib/portal-channels-list/portal-channels-list.component.ts`
|
||||
5. `libs/portal/shared/util/src/lib/collection/unified-recent-data.service.ts`
|
||||
6. `libs/portal/shared/util/src/lib/collection/unified-favorites-data.service.ts`
|
||||
5. `libs/portal/shared/data-access/src/lib/collection/unified-recent-data.service.ts`
|
||||
6. `libs/portal/shared/data-access/src/lib/collection/unified-favorites-data.service.ts`
|
||||
|
||||
### Busy states
|
||||
|
||||
@@ -635,10 +700,14 @@ renderer can actually paint the loading state instead of freezing.
|
||||
|
||||
### Worker bundling
|
||||
|
||||
`apps/electron-backend/build-worker.js` now bundles both:
|
||||
`apps/electron-backend/build-worker.js` produces three bundles:
|
||||
|
||||
1. `epg-parser.worker.ts`
|
||||
2. `database.worker.ts`
|
||||
1. `apps/electron-backend/src/app/workers/epg-parser.worker.ts` →
|
||||
`dist/apps/electron-backend/workers/epg-parser.worker.js`
|
||||
2. `apps/electron-backend/src/app/workers/database.worker.ts` →
|
||||
`dist/apps/electron-backend/workers/database.worker.js`
|
||||
3. `apps/electron-backend/src/app/workers/playlist-refresh.worker.ts` →
|
||||
`dist/apps/electron-backend/workers/playlist-refresh.worker.js`
|
||||
|
||||
The worker build also aliases:
|
||||
|
||||
@@ -666,12 +735,16 @@ artifacts for:
|
||||
2. macOS app bundles
|
||||
3. Windows unpacked app resources
|
||||
|
||||
The script checks:
|
||||
The verifier's `workerFiles` list currently checks:
|
||||
|
||||
1. `epg-parser.worker.js`
|
||||
2. `database.worker.js`
|
||||
3. `better-sqlite3` in one approved unpacked node_modules location
|
||||
4. Snap packaging compatibility settings for `better-sqlite3`:
|
||||
|
||||
The playlist refresh bundle is produced by the worker build, but is not yet a
|
||||
third explicit `workerFiles` check. The verifier also checks:
|
||||
|
||||
1. `better-sqlite3` in one approved unpacked node_modules location
|
||||
2. Snap packaging compatibility settings for `better-sqlite3`:
|
||||
- `snap.base = core22`
|
||||
- Snap launch args keep the X11 fallback
|
||||
- Snap and the other non-Flatpak Linux artifacts build on Ubuntu 22.04, while Flatpak builds on a separate Ubuntu 24.04 CI runner
|
||||
@@ -853,11 +926,15 @@ Available trace flags:
|
||||
Logs `window.electron.*` method calls crossing the preload bridge so you can
|
||||
see whether the renderer is still reaching Electron main.
|
||||
3. `IPTVNATOR_TRACE_DB=1`
|
||||
Logs `DatabaseWorkerClient` request dispatch, completion timing, and emitted
|
||||
`DB_OPERATION_EVENT` payloads.
|
||||
Logs redacted `DatabaseWorkerClient` request dispatch, completion timing,
|
||||
and emitted `DB_OPERATION_EVENT` summaries. It also enables the safe SQL
|
||||
statement-type summaries described below.
|
||||
4. `IPTVNATOR_TRACE_SQL=1`
|
||||
Logs SQLite statements for the shared main-process connection and the DB
|
||||
worker connection using `better-sqlite3` verbose hooks.
|
||||
Logs only a fixed allowlisted statement type such as `SELECT`, `INSERT`, or
|
||||
`UPDATE` for the shared main-process and worker connections. The verbose
|
||||
hook output passes through
|
||||
`libs/shared/logging/src/lib/sql-trace-summary.ts`; expanded SQL text and
|
||||
bound values are never emitted.
|
||||
5. `IPTVNATOR_TRACE_WINDOW=1`
|
||||
Logs BrowserWindow loading, navigation, `unresponsive`, and
|
||||
`render-process-gone` transitions.
|
||||
@@ -882,19 +959,14 @@ CI=1 NX_TASKS_RUNNER_DYNAMIC_OUTPUT=false pnpm nx run electron-backend:build --s
|
||||
|
||||
## Current Limitations
|
||||
|
||||
These are intentionally still out of scope for this first cut:
|
||||
These remain intentionally out of scope:
|
||||
|
||||
1. moving network-heavy Xtream fetches off the current path
|
||||
2. migrating every remaining small SQLite IPC handler to the worker
|
||||
2. migrating explicitly lightweight download and EPG-specific main-process
|
||||
handlers without evidence that they block Electron main
|
||||
3. richer delete progress reporting for bulk destructive operations
|
||||
4. repo-wide Angular/Jest cleanup for the currently failing web test baseline
|
||||
|
||||
(Request cancellation, originally listed here, has since shipped — see the
|
||||
"Cancellation contract" section above: `DB_CANCEL_OPERATION` in
|
||||
`apps/electron-backend/src/app/api/main.preload.ts`, `AbortError` production in
|
||||
`database.worker.ts`, and `DatabaseService.cancelOperation` in
|
||||
`libs/services/src/lib/database-electron.service.ts`.)
|
||||
|
||||
## Extending The Worker
|
||||
|
||||
When adding another heavy SQLite operation:
|
||||
|
||||
Reference in new issue
Block a user