diff --git a/.changes/stalker-session-compatibility.md b/.changes/stalker-session-compatibility.md new file mode 100644 index 000000000..c58719f30 --- /dev/null +++ b/.changes/stalker-session-compatibility.md @@ -0,0 +1,6 @@ +--- +type: feature +area: stalker +--- + +Stalker portals discover endpoints, retain cookies, request credentials only when needed, and refresh once. Native playback keeps authorization out of the renderer; reused MPV isolates stream headers and authenticated VLC avoids RC reuse. Backups redact credentials and explicit identity by default, validate missing Xtream credentials, and retain portal URLs and MACs. diff --git a/.codex/skills/stalker-portal/SKILL.md b/.codex/skills/stalker-portal/SKILL.md index d59fa7a20..b025eee7e 100644 --- a/.codex/skills/stalker-portal/SKILL.md +++ b/.codex/skills/stalker-portal/SKILL.md @@ -1,6 +1,6 @@ --- name: stalker-portal -description: Repository guidance for Stalker/Ministra portal catalogs, VOD/series shapes, playback metadata, collections, EPG, and remote control. +description: Repository guidance for Stalker/Ministra sessions, catalogs, VOD/series shapes, playback metadata, collections, EPG, and remote control. --- # Stalker Portal @@ -18,7 +18,13 @@ views, playback, favorites/recent activity, EPG, or remote control. - Feature UI: `libs/portal/stalker/feature/src/lib/` - Store/API data access: `libs/portal/stalker/data-access/src/lib/` -- Electron requests: `apps/electron-backend/src/app/events/stalker.events.ts` +- Pure session protocol: `libs/portal/stalker/protocol/src/lib/` +- Typed full-session IPC: + `apps/electron-backend/src/app/events/stalker-session.events.ts` +- Main-owned session runtime: + `apps/electron-backend/src/app/services/stalker-session/` +- Legacy simple/PWA-compatible requests: + `apps/electron-backend/src/app/events/stalker.events.ts` - Shared Stalker item normalization: `libs/shared/interfaces/src/lib/stalker-item.normalizer.ts` - Dashboard aggregation: `libs/workspace/dashboard/data-access/src/lib/` @@ -27,6 +33,42 @@ Keep provider-specific API and normalization behavior in Stalker data access. Keep shared portal layouts/utilities provider-neutral. Preserve full-portal session auth and simple IPC request paths. +## Session Compatibility Invariants + +1. Electron full portals cross preload only through typed open, continue, + request, and control operations. The renderer retains opaque lease, + challenge, attempt, and playback-context references only. +2. Credentials, Bearer tokens, handshake randoms, cookies, internal session + keys, and playback headers stay out of preload/renderer state, persisted + playlist metadata, and diagnostics. Electron main owns them and supplies + authorized HTTP/native-player requests directly. +3. Discovery is anonymous before origin approval. Unknown/malformed response + shapes fail closed; only a narrowly recognized benign HTML endpoint-shape + miss or an explicit unsupported status may advance, and only explicit + unsupported auth can contribute stateless evidence. Explicit private portal + sources are allowed. Anonymous public-to-private redirects must be rejected + before contact; identity-bearing cross-origin redirects must pause before + target preparation/contact and require approval. +4. First profile uses `auth_second_step=0`. Submit exact credentials only after + status `2`; a second profile with step `1` follows canonical `do_auth` + success. The three-submission budget belongs to the connection attempt, not + a replaceable auth object. Do not invent serial/device/hash values. +5. Provisional flows persist the verified playlist atomically before commit. + Cancel, navigation, stale async outcomes, deletion, and failed promotion + must discard/clean the corresponding main-owned state. +6. Embedded MPV and external MPV/VLC consume `create_link` authorization + through a sender-bound main-owned playback context. Built-in web players + may carry but do not consume the opaque reference, and authenticated + downloads are not yet a context consumer. Do not expose headers or cookies + to make an unsupported surface work. +7. A request recipe with the current classifier version is authoritative over + legacy `isFullStalkerPortal`. Missing/stale recipes classify provisionally; + a current stateless recipe may perform one single-flight endpoint-shape + re-detection, persist/commit the new recipe, and reissue the operation once. +8. Backup import never trusts learned Stalker connection state. Every restore, + including a matched redacted or secret-bearing merge, clears the landing, + recipe, classifier-version, and last-verification fields before reconnecting. + ## `is_series` Cross-Surface Checklist Treat VOD items with `is_series` as series across every downstream surface. @@ -56,6 +98,15 @@ Do not stop after making the detail view render. `pnpm nx test portal-stalker-feature` - Stalker shape/store behavior: `pnpm nx test portal-stalker-data-access` +- Pure protocol and Electron session runtime: + `pnpm nx test portal-stalker-protocol` + `pnpm nx test electron-backend` +- Import and connection lifecycle: + `pnpm nx test playlist-import-feature` + `pnpm nx test portal-stalker-feature` +- Stateful replay validation: + `pnpm run stalker:fixtures:validate` + `pnpm nx test stalker-fixture-tools` - Dashboard classification and position lookup: `pnpm nx test workspace-dashboard-data-access` - Dashboard badge rendering when changed: diff --git a/AGENTS.md b/AGENTS.md index 014efff5b..03059df4b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -424,8 +424,8 @@ Key files: File: `.codex/skills/iptvnator-sqlite-db-worker/SKILL.md` - `stalker-portal` - Repository-specific guidance for Stalker/Ministra catalogs, all three VOD/series modes, cross-surface `is_series` behavior, playback metadata, collections, EPG, and remote control. - Use when changing Stalker routes, stores, detail views, playback, favorites/recent activity, EPG, or remote control. + Repository-specific guidance for Stalker/Ministra main-owned sessions, catalogs, all three VOD/series modes, cross-surface `is_series` behavior, playback metadata, collections, EPG, and remote control. + Use when changing Stalker authentication, routes, stores, detail views, playback, favorites/recent activity, EPG, or remote control. File: `.codex/skills/stalker-portal/SKILL.md` - `xtream-electron` diff --git a/CLAUDE.md b/CLAUDE.md index cb8cbbe18..9d5c22894 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -244,7 +244,10 @@ This is an Nx monorepo with the following structure: - **playlist/m3u/feature-player** - M3U video player page and `/workspace/playlists/:id` routes - **playlist/shared/{ui,util}** - Shared playlist UI and utilities - **portal/xtream/{data-access,feature}** - XtreamStore, services, data sources; routed Xtream components - - **portal/stalker/{data-access,feature}** - StalkerStore and routed Stalker components + - **portal/stalker/protocol** - Pure Stalker endpoint, response, + identity-profile, state-machine, and reserved-request policy + - **portal/stalker/{data-access,feature}** - Opaque renderer session + facade, StalkerStore, connection recovery, and routed Stalker components - **portal/catalog/feature** - Portal catalog UI - **portal/downloads/feature** - Download manager UI - **portal/shared/{data-access,ui,util}** - Cross-portal shared code @@ -607,7 +610,10 @@ This project uses modern Angular signal-based APIs and patterns. **ALWAYS** use - `playlist.events.ts` - Playlist import/update - `epg.events.ts` - EPG IPC registration and freshness/fetch orchestration; worker lifecycle lives in `epg-worker.service.ts`, DB lookups in `epg-query.service.ts` - `xtream.events.ts` - Xtream Codes API - - `stalker.events.ts` - Stalker portal API + - `stalker.events.ts` - Legacy simple/PWA-compatible Stalker portal API + - `stalker-session.events.ts` - Typed full-portal session IPC; the + main-process runtime in `services/stalker-session/` owns discovery, + cookies, credentials, refresh, watchdogs, and playback authorization - `player.events.ts` - External player IPC registration; MPV/VLC lifecycle logic lives in `mpv-session.service.ts`, `vlc-session.service.ts`, and shared `external-player-*` helpers - `settings.events.ts` - App settings - `electron.events.ts` - App version, etc. @@ -624,7 +630,14 @@ This project uses modern Angular signal-based APIs and patterns. **ALWAYS** use - M3U/M3U8 files (local or URL) - Xtream Codes API (`username`, `password`, `serverUrl`) -- Stalker portal (`macAddress`, `url`) +- Stalker portal (`macAddress`, source URL): Electron full portals use bounded + endpoint discovery, conditional status-2 credentials, a portal-scoped cookie + jar, single-flight refresh, and opaque main-owned playback contexts. Explicit + stateless portals and the PWA retain the legacy compatibility path. Playlist + backups retain portal addresses and MACs but exclude credentials and explicit + identity overrides by default; authenticated web playback and downloads + remain outside the playback-context consumer contract. Canonical details: + `docs/architecture/stalker-portal.md`. **Video Players**: diff --git a/apps/stalker-mock-server/README.md b/apps/stalker-mock-server/README.md index 8a299a823..6c899ad57 100644 --- a/apps/stalker-mock-server/README.md +++ b/apps/stalker-mock-server/README.md @@ -1,14 +1,25 @@ # Stalker Mock Server -A local mock implementation of the Stalker/Ministra portal API for development and end-to-end testing of IPTVnator. +A local mock implementation of the Stalker/Ministra portal API for development +and end-to-end testing of IPTVnator. It has two separate test surfaces: + +- the long-running generated catalog server on port `3210`; +- short-lived, stateful replay listeners used by Electron authentication E2E. ## Overview -The mock server speaks the same `portal.php` HTTP protocol as a real Stalker portal, generating deterministic fake data using `@faker-js/faker` seeded from the connecting MAC address. This means: +The mock server speaks the same `portal.php` HTTP protocol as a real Stalker +portal. Its scenario, IDs, and catalog structure are seeded from the connecting +MAC address. This means: -- The **same MAC address always returns the same data** (consistent across page refreshes and test runs). -- **Different MAC addresses produce different datasets** — use predefined scenario MACs for specific test conditions. -- Data is generated once per MAC on first request and cached in memory for the server's lifetime. **Restart to regenerate.** +- The **same MAC address stays consistent between reset/restart events** and + keeps the same seed-driven structure after regeneration. +- Ratings, some dates, and current-day EPG are volatile, so regenerated payloads + are not byte-for-byte identical. +- Different MAC addresses have isolated state and seeded catalogs — use + predefined scenario MACs for specific test conditions. +- Data is generated once per MAC on first request and cached until `/reset` or + process restart. ## Quick Start @@ -39,7 +50,7 @@ Then in IPTVnator, add a new Stalker portal: | `00:1A:79:00:00:04` | **is-series** | 60% of VOD items have `is_series=1` — tests the Ministra lazy-season flow | | `00:1A:79:00:00:05` | **embedded-series** | 50% of VOD items have embedded `series[]` arrays — tests the embedded series flow | | `00:1A:79:00:00:06` | **legacy-pagination** | No `get_all_channels` support — tests the paginated `get_ordered_list` crawl fallback for the full ITV channel list | -| `` | **auto** | MAC bytes used as seed → deterministic unique dataset | +| `` | **auto** | MAC bytes choose the seed for an isolated generated catalog | ## Configuration @@ -101,6 +112,37 @@ nx e2e web-e2e --grep "@stalker" The test suite uses `00:1A:79:00:00:01` (default scenario) for most tests, and calls `POST /reset` in `beforeEach` to ensure a clean state between tests. +## Stateful Authentication Replay + +Authentication, redirects, cookies, response-classifier edge cases, refresh, +and custom endpoint layouts use fixtures under `fixtures/replay/`. These +fixtures are not served by the development listener on port `3210`. + +The Electron replay harness starts a capability-protected loopback control +plane, creates one allowlisted fixture run, and receives distinct ephemeral +listeners for every named origin. The Electron app receives only synthetic +portal inputs. Each run must be finalized for exact request cardinality and +terminal state, then disposed in `finally`. + +Validate the complete committed corpus with: + +```bash +pnpm run stalker:fixtures:validate +pnpm nx test stalker-mock-server +pnpm nx test stalker-fixture-tools +``` + +The local HAR converter is only a draft aid: + +```bash +pnpm run stalker:fixtures:draft -- /absolute/input.har /absolute/output.json +``` + +It rejects repository inputs, unsafe files, unknown origins, oversized or +deep payloads, and secret-like evidence. Never commit real portal URLs, +credentials, account data, catalogs, artwork, or stream links. Review and +validate every generated draft before moving it into `fixtures/replay/`. + ## EPG Behavior The mock server generates a 7-day EPG schedule for every ITV channel using @@ -121,9 +163,12 @@ See [`docs/architecture/stalker-mock-server.md`](../../docs/architecture/stalker ``` apps/stalker-mock-server/ +├── fixtures/replay/ # Secret-scanned stateful fixtures ├── src/ -│ ├── main.ts # Express bootstrap +│ ├── main.ts # Generated catalog CLI bootstrap +│ ├── app.ts # Import-safe Express app factory │ └── app/ +│ ├── replay/ # Isolated replay/control-plane runtime │ ├── scenarios.ts # MAC → scenario config mapping │ ├── data-generator.ts # Seeded faker data generation │ ├── data-store.ts # Lazy per-MAC in-memory cache diff --git a/docs/architecture/embedded-inline-playback.md b/docs/architecture/embedded-inline-playback.md index 3acd31898..0c9e4b045 100644 --- a/docs/architecture/embedded-inline-playback.md +++ b/docs/architecture/embedded-inline-playback.md @@ -345,7 +345,14 @@ diagnostic and explicit MPV/VLC fallback. Portal VOD and episode payloads with `contentInfo` are treated as non-live by the inline players unless `isLive` is explicitly set. If Chromium leaves the underlying MediaSource duration at `Infinity` for a finite TS VOD, the Video.js wrapper normalizes its UI duration from the finite `seekable` or `buffered` range. Embedded MPV uses the same live decision rule and shows an unknown duration placeholder for VOD/episode snapshots until MPV reports a finite duration. This removes the misleading `LIVE` control state without changing stream decoding, diagnostics, or external fallback behavior. -When a diagnostic is actionable in Electron, the diagnostic surface may offer `Open in MPV`, `Open in VLC`, `Copy URL`, technical details, and `Retry`. Web builds only expose copy/help text and retry. MPV/VLC fallback requests carry the original `ResolvedPortalPlayback` payload so headers, referer, origin, user-agent, content metadata, and resume offset stay intact. Retry clears the current diagnostic and rebuilds the active inline player inputs; it does not change the saved player setting. +When a diagnostic is actionable in Electron, the diagnostic surface may offer +`Open in MPV`, `Open in VLC`, `Copy URL`, technical details, and `Retry`. Web +builds only expose copy/help text and retry. MPV/VLC fallback carries the +original `ResolvedPortalPlayback`. Legacy request metadata remains intact; +typed Stalker playback carries only `playbackContextRef`, which native-player +IPC consumes once to obtain main-owned headers. Retry clears the current +diagnostic and rebuilds the active inline player inputs; it does not change the +saved player setting. `PortalPlayer.openExternalPlayback(playback, player)` is the forced external launch API. It sends the playback payload to MPV or VLC regardless of the current saved player setting, so fallback buttons do not mutate preferences. @@ -363,9 +370,27 @@ with spaces safe, and preserves existing settings for users who never configured extra arguments. The arguments apply only when IPTVnator spawns a new external player process. If -MPV or VLC instance reuse is active and an existing process is reused, subsequent -streams are loaded through MPV IPC or VLC RC commands and new process arguments -are not re-applied until a fresh process starts. +MPV or custom-header-free VLC instance reuse is active and an existing process +is reused, subsequent streams are loaded through MPV IPC or VLC RC commands and +new process arguments are not re-applied until a fresh process starts. + +For reused MPV processes, every `loadfile` carries file-local `user-agent`, +`referrer`, and `http-header-fields` values and waits for acknowledgement before +the next queued source switch. Plain sources send empty file-local values, and +the initial reusable source is wrapped in MPV's local option scope. Prior +Authorization/Cookie headers therefore cannot become process defaults or leak +to the next stream. + +VLC RC reuse is restricted to streams whose merged custom-header map is empty. +Before a launch carrying Authorization/Cookie or any other entry in that map, +Electron kills any tracked reusable VLC process and starts a fresh untracked +process with `shell: false`, passing each header as one argv entry. +Authorization/Cookie values never enter an RC `add` command, so spaces, +newlines, or VLC-looking input-option text inside such a header value cannot +become a second RC option or command. Scalar user-agent, referrer, and origin +settings are not part of this map and remain per-input RC options for reusable +streams. A per-launch RC port may still be used only for constant +progress-polling commands when resume tracking is required. ## Electron External Player Ownership @@ -401,7 +426,11 @@ Current contract: - AppImage, deb/rpm, snap, macOS, and Windows keep the existing direct process spawn flow. - VLC keeps the current external-session flow in Flatpak, including the RC port used for progress polling. - MPV is intentionally reduced in Flatpak: the app does not reuse an existing MPV instance there and does not open the Unix socket bridge used for non-Flatpak progress polling. -- VLC instance reuse is also gated off in Flatpak. Outside Flatpak the user can opt in via the "Reuse VLC instance" setting; the app then keeps a single tracked VLC process and drives subsequent stream loads through its RC interface (`clear` + `add :http-*`) instead of spawning a new window per click. +- VLC instance reuse is also gated off in Flatpak. Outside Flatpak the user can + opt in via the "Reuse VLC instance" setting; the app keeps a single tracked + VLC process only for sources without a custom-header map and drives subsequent + eligible stream loads through its RC interface. A source with + Authorization/Cookie or another custom header always gets a fresh process. This keeps non-Flatpak behavior unchanged while allowing Flatpak builds to open host-installed external players. @@ -409,7 +438,7 @@ This keeps non-Flatpak behavior unchanged while allowing Flatpak builds to open Shared playback payloads live in: -- `/Users/4gray/Code/iptvnator/libs/shared/interfaces/src/lib/portal-playback.interface.ts` +- `libs/shared/interfaces/src/lib/portal-playback.interface.ts` Types introduced: @@ -423,6 +452,7 @@ These provide a single shape for: - optional thumbnail and resume start time - playback-position metadata - optional external-player headers and request metadata +- optional opaque `playbackContextRef` for native Stalker playback ## Xtream Behavior diff --git a/docs/architecture/playlist-backup-restore.md b/docs/architecture/playlist-backup-restore.md index 2f3f6f045..56f9ba77c 100644 --- a/docs/architecture/playlist-backup-restore.md +++ b/docs/architecture/playlist-backup-restore.md @@ -5,13 +5,13 @@ settings screen. ## Entry Points -- UI: `/Users/4gray/Code/iptvnator/apps/web/src/app/settings/settings-backup-section.component.ts` +- UI: `apps/web/src/app/settings/settings-backup-section.component.ts` (embedded in `settings.component.html`), with the file read/handoff in - `/Users/4gray/Code/iptvnator/apps/web/src/app/settings/settings-backup.facade.ts` -- Backup service: `/Users/4gray/Code/iptvnator/libs/services/src/lib/playlist-backup.service.ts` -- Manifest types: `/Users/4gray/Code/iptvnator/libs/shared/interfaces/src/lib/playlist-backup.interface.ts` + `apps/web/src/app/settings/settings-backup.facade.ts` +- Backup service: `libs/services/src/lib/playlist-backup.service.ts` +- Manifest types: `libs/shared/interfaces/src/lib/playlist-backup.interface.ts` - Xtream pending restore storage: - `/Users/4gray/Code/iptvnator/libs/services/src/lib/xtream-pending-restore.service.ts` + `libs/services/src/lib/xtream-pending-restore.service.ts` ## Manifest Contract @@ -30,6 +30,12 @@ Top-level shape: The manifest is portable across machines because it stores playlist definitions and portable user state, while excluding cache-only database content. +`includeSecrets` is an enforced security boundary, not just a UI hint. New +exports set it to `false` unless the user explicitly opts in. An import that +declares `includeSecrets: false` but contains gated credential/device fields is +rejected before writes. Older version-1 manifests that omitted the field retain +their legacy secret-bearing interpretation for backward compatibility. + ## Export Scope ### M3U @@ -54,10 +60,10 @@ playlist object graph is not the backup format. Xtream backups export only connection metadata plus portable user state. -- Connection metadata: - - `serverUrl` - - `username` - - `password` +- Connection metadata always includes `serverUrl`. +- Default redacted export adds `credentialsOmitted: true` and contains neither + username nor password. +- Explicit secret export includes `username` and `password`. - User state: - hidden categories by `{ categoryType, xtreamId }` - favorites by `{ contentType, xtreamId, addedAt?, position? }` @@ -75,16 +81,22 @@ Explicitly excluded: Stalker backups export connection metadata plus playlist-scoped favorites/recent state. -- Exported connection fields: +- Required exported connection fields: - `portalUrl` - `macAddress` +- Optional non-secret connection fields, exported when present: + - original `sourceUrl` - `isFullStalkerPortal` - - `username` - - `password` + - profile preset + - non-secret transport configuration - `userAgent` - `referrer` - `origin` - - serial/device/signature fields when present +- Explicit secret export additionally includes: + - `username` + - `password` + - structured identity overrides + - legacy serial/device/signature fields when present - Exported user state: - favorites snapshots - recently viewed snapshots @@ -93,8 +105,17 @@ Explicitly excluded: - `stalkerToken` - `stalkerAccountInfo` +- `stalkerLandingUrl`, `stalkerRequestRecipe`, + `stalkerRecipeClassifierVersion`, and `stalkerLastVerifiedAt` +- live cookies, handshake randoms, leases, challenges, and playback contexts - playback positions in v1 +Compatibility `portalUrl` remains exported as connection metadata. Restore +re-resolves from `sourceUrl` when it is present. + +The opt-in warning remains necessary even for a redacted backup: portal hosts, +MAC addresses, and private M3U source URLs can still be sensitive. + ### App Settings Only EPG source URLs are backed up at the app-settings level. @@ -116,12 +137,17 @@ The service: 4. Upserts playlists into app playlist storage. 5. Restores provider-specific user state. -Fingerprint rules: +Base fingerprint rules: - M3U URL playlists: normalized URL - M3U without URL: hash of canonical `rawM3u` -- Xtream: normalized `serverUrl + username` -- Stalker: normalized `portalUrl + macAddress` +- Xtream: normalized server URL, then the exact principal when credentials are + present +- Secret-bearing Stalker: normalized source + MAC + profile + exact effective + identity/transport. A matching exported ID is preferred; otherwise exactly + one exact username/principal match is required. The password is never part + of the fingerprint, so an exported-ID match may patch it. Structured + identity/transport wins, with equivalent legacy fields as fallback If a fingerprint matches an existing playlist: @@ -135,6 +161,31 @@ If no fingerprint matches: - reuse `exportedId` only when it is unused - otherwise generate a new UUID +Redacted provider entries are deliberately stricter: + +- redacted Xtream preserves credentials only when an exact exported-ID/server + match has both a usable username and password. Otherwise, including an exact + row with an incomplete credential pair, import prompts for credentials, + validates the exact submitted values against an active portal response + without using the status cache, and can be skipped without creating or + overwriting a row. A validated pair patches that exact row instead of + creating a duplicate +- redacted Stalker preserves local credentials only on an exact exported-ID, + source, MAC, and profile match; otherwise it creates a credential-less row, + which later follows the normal status-2 Stalker connection flow +- ambiguous legacy Stalker matches create a separate row rather than merging + two possible devices/accounts + +Restore fields use patch semantics: present values replace, allowed empty +values clear, and omitted fields preserve the existing value on any merge. +Redacted entries can merge only under the stricter exact-match rules above, so +unmatched redacted rows cannot inherit local secrets. The excluded learned +Stalker fields are an intentional exception: every Stalker restore clears +`stalkerLandingUrl`, `stalkerRequestRecipe`, +`stalkerRecipeClassifierVersion`, and `stalkerLastVerifiedAt`, including on a +matched merge, so the next connection re-resolves and verifies the imported +definition. + ## Xtream Restore Contract Xtream restore is type-aware end to end. The app no longer stores plain @@ -180,6 +231,10 @@ raw JSON application dump. - Export filename: `iptvnator-playlist-backup-YYYY-MM-DD.json` +- “Include portal credentials and device identity” is off by default. +- Import prompts for missing Xtream credentials; the user may validate them or + skip that entry. Blank, inactive, expired, and unavailable credentials are + rejected before playlist persistence. - Import summary reports: - imported - merged diff --git a/docs/architecture/stalker-authentication-compatibility-audit.md b/docs/architecture/stalker-authentication-compatibility-audit.md new file mode 100644 index 000000000..b8cabc393 --- /dev/null +++ b/docs/architecture/stalker-authentication-compatibility-audit.md @@ -0,0 +1,892 @@ +# Stalker/Ministra Authentication and Client Compatibility Audit + +Status: Option B Stage 1 implemented; this audit remains the evidence record, +not the runtime contract. + +Audit date: 2026-07-26 +IPTVnator baseline: `9ae53e451570538770e831e2838a776796284907` + +## Scope + +This audit compares IPTVnator's Stalker/Ministra implementation with: + +- [`Schrittfisch2000/Stalker-Client`](https://github.com/Schrittfisch2000/Stalker-Client) +- public Stalker Middleware/MAG client code +- Team Kodi's active + [`pvr.stalker`](https://github.com/kodi-pvr/pvr.stalker) repository, currently + [without a dedicated Stalker maintainer](https://github.com/kodi-pvr/pvr.stalker/issues/255#issuecomment-4400506581) +- other active clients and proxies with substantial adoption or test coverage +- IPTVnator's public Stalker issues and pull requests +- public STBEmu and StbEmuTV documentation and release history + +The review covers endpoint discovery, handshake/profile authentication, device +identity, cookies, session refresh, pagination, caching, playback session +lifecycle, persistence, diagnostics, and ideas that can be safely adapted. + +Only public source code, documentation, and redacted issue reports were used. +No credentials, private portal traces, or access-control bypass techniques are +part of this audit. + +## Executive Summary + +At the audited baseline, the main compatibility problem was not simply a +missing serial number. IPTVnator sent an internally inconsistent MAG identity +and skipped the canonical conditional second authentication step: + +- the initial `get_profile` incorrectly claims `auth_second_step=1` +- `profile.status=2` is not handled as `do_auth` followed by a second profile +- `do_auth` exists but sends empty credentials and is not called +- `prehash` is `SHA1(uppercase MAC)`, which does not match the public MAG client +- the request says `MAG250` in metrics while omitting or contradicting the + corresponding firmware, hardware, `stb_type`, `X-User-Agent`, and Referer +- server-issued cookies are not retained in a portal-scoped cookie jar +- endpoint selection depends on URL string patterns instead of discovering the + actual portal entry point + +The strongest public evidence supported keeping IPTVnator's MAC-only default. +Real MAG-only values such as `GetUID()` and +`GetHashVersion1(...)` are opaque native functions, not standard hashes of the +MAC. Invented device IDs can become persistently bound to the MAC; invented +serial or signature values can also fail custom access filters. Recovery may +require restoring the original identity or a provider-side reset. + +At the audited baseline, IPTVnator's live-channel pagination was already +stronger than most reviewed clients: it tried `get_all_channels`, fell back to +a bounded concurrent page crawl, calculated page count correctly, +de-duplicated IDs, stopped on repeated pages, retried once, and kept a +last-good in-memory ITV catalog cache. The next useful caching step is a +semantic SQLite snapshot with stale-while-revalidate, not a generic HTTP cache +for authentication or stream links. + +`Schrittfisch2000/Stalker-Client` is useful as a small idea source, but it is +not an authentication reference: at the audited commit it had no license, no +issues, no stars or forks, and its first pull request explicitly noted the lack +of a real portal E2E test. Its profile flow repeats several of IPTVnator's +mistakes and its fallback pagination treats `max_page_items` as a page count. + +Difference severity is high for credential-required, strict-fingerprint, and +custom-path portals, because request order or endpoint selection can fail +before catalog access. It is moderate for permissive MAC-only portals: the +handshake/Bearer foundation is present, so servers that ignore the inconsistent +profile can still work. This explains why compatibility can look random across +providers without serial number being the single missing field. + +## Implementation Outcome + +The approved Option B foundation was implemented on 2026-07-27. The canonical +runtime contract is now +[Stalker Portal Architecture](./stalker-portal.md); this document intentionally +keeps the audited baseline and source comparisons unchanged. + +Stage 1 now provides: + +- bounded root, landing, custom-prefix, `portal.php`, and `server/load.php` + discovery with explicit cross-origin approval +- a main-process RFC cookie jar and server-issued cookie rotation +- first profile with `auth_second_step=0`, conditional status-2 `do_auth`, and + second profile with step `1` +- one versioned MAC-only-by-default identity preset with explicit, stable + overrides and no invented serial/device/hash values +- main-owned token generations, principal coordination, single-flight refresh, + profile-derived watchdogs, and sender-bound playback contexts +- typed IPC operations that expose opaque references and sanitized outcomes, + while preserving the simple/PWA compatibility adapter +- save-before-commit import and lazy route migration, with credentials requested + only after status `2` +- default-redacted provider backups and deterministic, secret-scanned replay + fixtures +- final acceptance hardening that rejects anonymous public-to-private redirect + hops before contact, treats unknown HTML denials as terminal, keeps the + credential budget across coordinator races, and invalidates learned recipe + state on every backup restore + +Semantic SQLite catalog snapshots/FTS and adaptive or resumable pagination +remain Stage 2; Stage 1 deliberately keeps the existing bounded in-memory +crawl. Evidence-driven HTTP 462 handling, previous-stream release, and broader +playback-session lifecycle remain Stage 3. Authenticated web-player/download +context consumers, the signed image proxy, and exportable compatibility +reports are also not part of Stage 1. + +## Evidence Hierarchy + +| Confidence | Source | What it can establish | Important limitation | +| ---------- | -------------------------------------------------------- | ---------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- | +| High | Public Stalker Middleware server and MAG frontend source | Request order, status branches, token storage policy, server-side identity binding | Historical versions; proprietary portal forks may differ | +| High | Kodi `pvr.stalker` | A mature independent state machine, retries, Referer/X-UA, watchdog timing | Kodi's default identity values are not a replacement for a real MAG fingerprint | +| Medium | Active clients with tests and users | Defensive endpoint handling, persistence, synchronization, UX patterns | Popularity does not prove protocol correctness | +| Medium | IPTVnator and Kodi issue reports | Real failure modes and compatibility symptoms | Reports often lack sanitized traces or a confirmed root cause | +| Low | Reverse-engineered hash recipes and scanner tools | Candidate hypotheses for controlled experiments | Frequently synthetic, mutually inconsistent, or unsafe to send to a real account | +| Low | Closed-source app release notes | Existence of user-facing cache or login features | Cannot establish what is cached or how authentication is implemented | + +Exact identity hash recipes are deliberately excluded from the recommended +design unless they can be verified against a user-owned device trace and a +sanitized fixture. + +## Authentication Core and Compatibility Wrappers + +The public MAG frontend and Kodi agree on the authentication core below. +Endpoint discovery, scoped cookie ownership, classified 403 handling, and +serialized refresh are recommended compatibility and safety wrappers rather +than canonical MAG steps. + +```mermaid +flowchart TD + A["User enters a landing or API URL"] --> B["Recommended wrapper: resolve final /c/ landing and validate API origin"] + B --> C["Recommended wrapper: create portal + identity scoped cookie/session context"] + C --> D["handshake with empty or portal-approved stored token"] + D --> E["get_profile with auth_second_step=0"] + E --> F{"profile.status"} + F -->|"0 / normal"| G["Session ready"] + F -->|"1 / rejected or blocked"| H["Show portal rejection/block message; do not rotate identity"] + F -->|"2 / credentials required"| I["do_auth with login/password plus configured IDs when available"] + I --> J{"do_auth succeeded"} + J -->|"yes"| K["get_profile with auth_second_step=1"] + K --> G + J -->|"no"| L["Report credential failure"] + G --> M["Authenticated API calls + profile-derived watchdog"] + M -->|"401, Authorization failed., or evidenced token-auth 403"| N["Recommended wrapper: one serialized full session refresh"] + M -->|"Access denied. without invalid-token evidence"| O["Report account/profile authorization denial"] + N --> D +``` + +The initial profile is always the first step. `do_auth` is conditional; it is +not a ritual request that every portal requires. A successful second profile +is what proves that the second step completed. + +Primary evidence: + +- MAG handshake and first profile: + [`xpcom.common.js`](https://github.com/iptvhakr/stalker_portal/blob/72deceee1e32ea00cf33ecf2376b80902ab11134/c/xpcom.common.js#L850-L967) +- MAG status and credential branch: + [`xpcom.common.js`](https://github.com/iptvhakr/stalker_portal/blob/72deceee1e32ea00cf33ecf2376b80902ab11134/c/xpcom.common.js#L296-L359) +- server-side `doAuth`: + [`stb.class.php`](https://github.com/iptvhakr/stalker_portal/blob/0eb23e4995222e7c3daa4b945a4e962703ebf0cc/server/lib/stb.class.php#L2140-L2175) +- Kodi session flow: + [`SessionManager.cpp`](https://github.com/kodi-pvr/pvr.stalker/blob/07989d2d8e5542135c5c7107ffeb4c316f7f65fc/src/stalker/SessionManager.cpp#L30-L183) + +There is no canonical hard-coded 10-, 15-, or 60-minute token lifetime in +these sources. Refresh should be outcome-driven and serialized. + +## Audited Baseline Differences (`9ae53e45`) + +### Priority Matrix + +| Priority | Area | Audited behavior | Evidence-based target | +| -------- | ------------------- | ----------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | +| P0 | First profile | Always sends `auth_second_step=1` | First profile uses `0`; only a successful `do_auth` permits `1` | +| P0 | Credential branch | Ignores `profile.status`; unused `do_auth` sends empty login/password | On status `2`, use imported credentials plus configured IDs when available, then fetch the second profile | +| P0 | Endpoint resolution | Infers mode from `/stalker_portal` and rewrites only a few suffixes | Validate the final origin, then use bounded MAC-only probes before any token/credentials | +| P0 | Device profile | MAG250 metrics conflict with blank/missing firmware and transport fields | One coherent, versioned device-profile preset plus an explicit traced-identity mode | +| P0 | `prehash` | Sends uppercase `SHA1(MAC)` and labels it real-client compatible | Omit when not applicable; only send a profile-version algorithm backed by evidence | +| P1 | Challenge | Invents a random value when handshake omits one | Preserve absence; do not manufacture challenge-derived fields | +| P1 | Headers | Full browser UA is duplicated into `X-User-Agent`; no API Referer | Coherent browser UA, `X-User-Agent: Model: ...; Link: ...`, and resolved `/c/` Referer | +| P1 | Cookies | Rebuilds static cookies and synthesizes `__cfduid`; ignores `Set-Cookie` | Portal/identity-scoped cookie jar plus only the canonical bootstrap cookies | +| P1 | Token | Persists a token at import, ignores it at runtime, keys memory state by playlist ID | Scope by canonical endpoint + MAC + identity revision; persist only when portal requests it | +| P1 | Error taxonomy | Recognizes 401 and one text form; not 403 or `Access denied.` | Classify HTTP and HTTP-200 body failures; refresh once without loops | +| P1 | Watchdog | Fixed 25-second interval | Use bounded profile `timeslot`/`watchdog_timeout` with jitter | +| P1 | Credentials | Dormant username/password controls are hidden, included in form data, and unused | Expose/use them only for status-2 auth and define secure storage/backup behavior | +| P2 | Metadata comments | Interface comments claim generated IDs and useful persisted session token | Correct comments when the runtime contract is implemented | + +### P0: `auth_second_step` and Status Handling + +In +[`stalker-session.service.ts`](https://github.com/4gray/iptvnator/blob/9ae53e451570538770e831e2838a776796284907/libs/portal/stalker/data-access/src/lib/stalker-session.service.ts#L293-L459), +the first profile always sends `auth_second_step: '1'`. `authenticate()` accepts +any profile without branching on `status`. The existing `doAuth()` sends blank +credentials and is never called. + +This is a protocol bug, not merely an optional enhancement. In the public +4.9-era server, `auth_second_step=1` changes the `auth_every_load` branch even +though IPTVnator never completed the second step: + +[`stb.class.php`](https://github.com/iptvhakr/stalker_portal/blob/0eb23e4995222e7c3daa4b945a4e962703ebf0cc/server/lib/stb.class.php#L555-L601). + +### P0: Endpoint Discovery + +The import flow: + +- considers a portal "full" only when the original URL contains + `/stalker_portal` +- rewrites a generic `/c` to `/portal.php` +- leaves a root URL unchanged +- maps a custom `/c` to `/portal.php`, which is wrong for + canonical Middleware installations whose API is + `/server/load.php` + +See +[`stalker-portal-import.component.ts`](https://github.com/4gray/iptvnator/blob/9ae53e451570538770e831e2838a776796284907/libs/playlist/import/feature/src/lib/stalker-portal-import/stalker-portal-import.component.ts#L198-L275). + +The public MAG frontend instead derives sibling `server/load.php` from the +actual loaded `/c/` document path; it does not require the literal directory +name `stalker_portal`: + +[`xpcom.common.js`](https://github.com/iptvhakr/stalker_portal/blob/72deceee1e32ea00cf33ecf2376b80902ab11134/c/xpcom.common.js#L387-L400). + +A safe resolver needs two phases: + +1. normalize a URL that may be a root, `/c`, `/c/`, `portal.php`, or + `server/load.php` +2. follow landing redirects without MAC, token, credentials, or device IDs +3. derive same-origin sibling candidates from the resolved path +4. validate the final origin and require explicit user confirmation before + trusting an origin change +5. only then run a bounded MAC-only handshake probe against the candidates, + still without Bearer token, password, or optional device IDs +6. inspect status, content type, body shape, and known auth outcomes, then save + the selected endpoint, landing Referer, and auth recipe + +MAC cookies, credentials, and Bearer tokens must never be forwarded to an +unvalidated redirect target. + +### P0: Incoherent Device Profile + +IPTVnator currently combines: + +- `metrics.model = MAG250` +- `metrics.type = STB` +- `stb_type = ''` +- no coherent `ver`, `client_type`, `image_version`, `hw_version`, + `hw_version_2`, or timestamp +- `SHA1(MAC)` as `prehash` +- a browser MAG250 UA copied verbatim into `X-User-Agent` +- no API Referer +- hard-coded `stb_lang=en_US@rg=dezzzz`, `timezone=Europe/Berlin`, and + `Accept-Language=en-US` instead of one configurable locale/timezone profile + +See +[`stalker-session.service.ts`](https://github.com/4gray/iptvnator/blob/9ae53e451570538770e831e2838a776796284907/libs/portal/stalker/data-access/src/lib/stalker-session.service.ts#L244-L345) +and +[`stalker-identity.ts`](https://github.com/4gray/iptvnator/blob/9ae53e451570538770e831e2838a776796284907/apps/electron-backend/src/app/events/stalker-identity.ts#L23-L57). + +Strict portals can reject inconsistent firmware/metrics. A Kodi report includes +the portal-returned profile message "Old firmware, missing metrics or hash": + +[`kodi-pvr/pvr.stalker#192`](https://github.com/kodi-pvr/pvr.stalker/issues/192#issuecomment-1256963480). + +The fix is not to fill every field with an invented hash. The safe model is: + +- a versioned, internally coherent compatibility profile for non-opaque + browser/model/firmware/header/locale/timezone fields +- MAC-only identity by default +- a separate advanced profile containing stable, user-provided values observed + from the user's own device +- no automatic identity cycling after failure + +### Why Serial and Device IDs Must Not Be Invented + +In the public MAG client: + +- `device_id` comes from native `stb.GetUID()` +- `signature` comes from a challenge form of the same opaque native function +- `prehash` is version-dependent; in one public version it is + `GetHashVersion1(model, firmwarePrefix)`, while older variants differ + +The YASEM maintainer could not reproduce `GetUID` and therefore exposed +configured values rather than claiming a standard JavaScript hash: + +- [`gstb.cpp`](https://github.com/mvasilchuk/yasem-mag-api/blob/f3646885e899a9b7e94905ebda3043eec5276473/gstb.cpp#L2325-L2351) +- [`yasem-mag-api#1`](https://github.com/mvasilchuk/yasem-mag-api/issues/1) + +The public server can bind the first non-empty device IDs to a MAC and reject a +later mismatch: + +[`stb.class.php`](https://github.com/iptvhakr/stalker_portal/blob/0eb23e4995222e7c3daa4b945a4e962703ebf0cc/server/lib/stb.class.php#L467-L531). + +This matches IPTVnator's own history: + +- [`#860`](https://github.com/4gray/iptvnator/issues/860): a generated device ID + caused the account to stop working in STBEmu +- [`#927`](https://github.com/4gray/iptvnator/issues/927): values entered by the + user were not forwarded, producing a device-ID conflict +- [`#941`](https://github.com/4gray/iptvnator/pull/941): fixed passthrough and + removed blank-field generation + +Keep that policy. Serial remains valuable as an explicit field because custom +access filters may require it, but a fake default serial is not safer than no +serial. + +### P1: Cookies and the Synthetic `__cfduid` + +The stock bootstrap cookies are `mac`, `stb_lang`, and `timezone`; a real +browser also retains server-issued `Set-Cookie` values. IPTVnator reconstructs +the bootstrap string on every request and has no portal-scoped cookie jar. + +When a serial is present, IPTVnator also derives a fixed 32-character +`__cfduid`. This is not a canonical Stalker identity mechanism. Cloudflare +deprecated that cookie and stopped setting it on 2021-05-10: + +[`Deprecating the __cfduid cookie`](https://blog.cloudflare.com/deprecating-cfduid-cookie/). + +The target design should: + +- use one cookie jar per canonical endpoint and stable identity revision +- preserve same-origin server-issued session cookies +- clear or migrate the jar when endpoint or identity changes +- remove the synthetic `__cfduid` from the normal profile +- treat the current `SN` HTTP header as an explicit legacy compatibility quirk, + not part of the normal MAG profile; canonical public code uses serial in + `get_profile.sn` and metrics +- treat actual Cloudflare challenges as a diagnostic outcome, not something to + bypass with guessed cookies + +### P1: Token Persistence and Session Scope + +The public MAG client offers a previously stored token during handshake, but +stores it only when the returned profile enables `store_auth_data_on_stb`: + +- client: + [`xpcom.common.js`](https://github.com/iptvhakr/stalker_portal/blob/72deceee1e32ea00cf33ecf2376b80902ab11134/c/xpcom.common.js#L850-L920) + and + [`token policy`](https://github.com/iptvhakr/stalker_portal/blob/72deceee1e32ea00cf33ecf2376b80902ab11134/c/xpcom.common.js#L1118-L1120) +- server: + [`stb.class.php`](https://github.com/iptvhakr/stalker_portal/blob/0eb23e4995222e7c3daa4b945a4e962703ebf0cc/server/lib/stb.class.php#L299-L373) + +IPTVnator currently saves `stalkerToken` during import but deliberately ignores +it in `ensureToken()`. The in-memory token and single-flight maps use playlist +ID as their key. The audited stock server validates and stores the access token +on the user row found by MAC, so two playlist records with the same endpoint +and MAC can create competing sessions. + +Choose one explicit policy: + +- do not persist tokens at all, or +- persist them in protected Electron storage only when the profile requests it, + offer them to handshake as a bootstrap hint, and clear them on `not_valid` or + classified rejection + +In both cases, live session state should be scoped by the tuple of canonical +endpoint, MAC, and stable identity revision rather than playlist ID alone. + +### P1: Error and Refresh Taxonomy + +The public frontend recognizes literal HTTP-200 bodies such as +`Authorization failed.` and `Access denied.`, but they are not equivalent. +Invalid-token evidence can trigger refresh, while `Access denied.` can be an +account/profile authorization cutoff: + +[`xpcom.common.js`](https://github.com/iptvhakr/stalker_portal/blob/72deceee1e32ea00cf33ecf2376b80902ab11134/c/xpcom.common.js#L698-L795). + +IPTVnator's current +[`isAuthorizationError`](https://github.com/4gray/iptvnator/blob/9ae53e451570538770e831e2838a776796284907/libs/portal/stalker/data-access/src/lib/stalker-session.service.ts#L526-L554) +recognizes HTTP 401 and one family of authorization text, but not HTTP 403 or +the plain `Access denied.` outcome. + +Use a bounded classifier: + +| Outcome | Action | +| --------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | +| 401, `Authorization failed.`, or another proven token rejection | Invalidate session and run one serialized full refresh | +| 403 with evidence of portal token rejection | One refresh, then report the result | +| 403 with HTML/challenge markers | Report WAF/challenge; do not loop | +| `Access denied.` or rejected/blocked profile without invalid-token evidence | Show the account/profile denial; do not refresh by default | +| 404, HTML instead of API JSON, or wrong content type | Re-run endpoint discovery without secrets | +| 429 or transient 5xx | Bounded backoff with jitter | +| Device conflict/block status | Keep identity stable and show actionable portal text | +| HTTP 462 during playback | Test recreating `create_link` and releasing a previous stream session; do not rotate global identity | + +IPTVnator already has the right high-level single-flight and retry-once shape. +It needs broader classification and a full +`handshake -> first profile -> conditional second step` refresh. + +### P1: Watchdog + +IPTVnator pings every 25 seconds. Kodi reads the portal's `timeslot` and +watchdog fields, and treats watchdog authorization failure as a session +outcome: + +[`SessionManager.cpp`](https://github.com/kodi-pvr/pvr.stalker/blob/07989d2d8e5542135c5c7107ffeb4c316f7f65fc/src/stalker/SessionManager.cpp#L219-L240). + +The interval should be profile-derived, bounded to safe minimum/maximum values, +and jittered. The canonical MAG lifecycle starts its watchdog for the portal +session, not only during playback. IPTVnator may choose a playback-scoped +watchdog as a traffic optimization, but that scope needs fixture evidence. + +## Audit of `Schrittfisch2000/Stalker-Client` + +Audited commit: +[`ec4d34a919e55cd660004d3ed8a905374b727eb9`](https://github.com/Schrittfisch2000/Stalker-Client/tree/ec4d34a919e55cd660004d3ed8a905374b727eb9). + +### What It Does Well + +- compact separation between backend portal access and browser UI +- same-origin media/image proxying instead of exposing every remote URL to the + browser +- global search across loaded content +- persisted local configuration, favorites, recent items, and playback + progress +- signed image proxy URLs +- redacted diagnostics +- a short live-list memory cache +- restart logic in + [`PR #11`](https://github.com/Schrittfisch2000/Stalker-Client/pull/11) + that recreates a `create_link` result and releases a previous session after + HTTP 462 + +The HTTP 462 work is a useful hypothesis for IPTVnator's live-playback issues, +but it is not evidence that IPTVnator has the same root cause. + +### Where It Is Weaker + +Its main Stalker implementation is in +[`app/stalker.py`](https://github.com/Schrittfisch2000/Stalker-Client/blob/ec4d34a919e55cd660004d3ed8a905374b727eb9/app/stalker.py). + +- API endpoint is effectively hard-coded around `portal.php` +- a new HTTP client is created for requests, so there is no durable cookie jar +- token lifetime is assumed to be 15 minutes +- handshake `random` is discarded +- profile sends a default serial, empty IDs/signature, + `auth_second_step=1`, and no coherent challenge hash +- pagination interprets `max_page_items` as if it were a page count +- the full live list is cached only for 90 seconds in process memory +- the issue tracker has no reports from which compatibility can be inferred +- [`PR #1`](https://github.com/Schrittfisch2000/Stalker-Client/pull/1) + says a real portal E2E test was not possible without private credentials +- the audited repository has no license, so code must not be copied + +Conclusion: borrow product ideas and the HTTP 462 test hypothesis, not its auth +payloads or pagination algorithm. + +## Other Serious Clients and Proxies + +Adoption numbers are a snapshot from 2026-07-26. They help discover candidates; +they are not a protocol correctness score. + +| Project | Adoption/activity | Test evidence | Best ideas | Protocol or legal caveat | +| -------------------------------------------------------------------------------------------------------------- | ---------------------------------------- | --------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | +| [Kodi `pvr.stalker`](https://github.com/kodi-pvr/pvr.stalker/tree/07989d2d8e5542135c5c7107ffeb4c316f7f65fc) | 47 stars, 65 forks; active in July 2026 | No repository-level Stalker test suite | Correct conditional second step, retries, Referer/X-UA, profile-derived watchdog, optional EPG disk cache | GPL-2.0; no dedicated maintainer; do not treat its static default IDs as real MAG values | +| [StreamVault-IPTV](https://github.com/Davidona/StreamVault-IPTV/tree/593333714c43cb1802e14a1452cd9f5906e0a286) | 547 stars, 88 forks; active in July 2026 | Dedicated Stalker tests and 16 sanitized replay fixtures; no live E2E | Room catalog, FTS, WorkManager sync, staged atomic replacement, partial-sync upsert guard, cache TTLs, endpoint/profile hints, response size bounds, defer catalog sync during playback | Custom non-commercial source-available license; auth sends second-step too early, discards random, and uses synthetic metrics | +| [OwnTV](https://github.com/ahXN00/OwnTV/tree/26f68f558f6879045755eea92d614773dd501d5a) | 279 stars, 45 forks; active in July 2026 | Nine test files | Minimal MAC-only auth, per-source mutex/single-flight, endpoint probing, in-memory winning endpoint, bounded one-time refresh | GPL-3.0; no conditional `do_auth`; hard-coded token TTL; incomplete HTTP-200 body-error classification | +| [ynoTV](https://github.com/tbeezy/ynotv/tree/3219ca0c7717c07f40e52a4661fd118a219a6c81) | 209 stars, 9 forks; active in July 2026 | No repository-level test suite | SQLite catalogs, EPG cache, watchlist, reminders, DVR, series/VOD fallback | AGPL-3.0; unverified MD5/SHA identity formulas, token logging, hard-coded TTL/page size, no `do_auth` | +| [uiptv](https://github.com/xixogo5105/uiptv/tree/e3bd949cc7819e9438efadbe45aed6bbf779af28) | 47 stars, 6 forks; active in July 2026 | Broad test tree; endpoint tests preserve known malformed URLs | Reads endpoint hints from `xpcom.common.js`, then probes common endpoints | MIT; sends fresh random signatures that could change device identity | +| [tuliprox](https://github.com/euzu/tuliprox/tree/bcc1badcfeb484ffbb325222f80b19bde850f314) | 512 stars, 52 forks; active in July 2026 | Stalker unit tests, but the integration is recent | Defensive endpoint candidates, cookie/session ownership, JSONP/BOM/HTML parsing, response caps, redacted failure taxonomy, serialized refresh, learned recipe/capability architecture | MIT; configured identity fields appear incomplete and some refresh hooks are not wired outside tests | + +### StreamVault Patterns Worth Re-implementing + +The strongest cache/sync reference is StreamVault, not its auth recipe: + +- content policy: + [`ContentCachePolicy.kt`](https://github.com/Davidona/StreamVault-IPTV/blob/593333714c43cb1802e14a1452cd9f5906e0a286/data/src/main/java/com/streamvault/data/sync/ContentCachePolicy.kt) +- constrained background indexing: + [`StalkerIndexWorker.kt`](https://github.com/Davidona/StreamVault-IPTV/blob/593333714c43cb1802e14a1452cd9f5906e0a286/data/src/main/java/com/streamvault/data/sync/StalkerIndexWorker.kt) +- staged catalog writes and partial-sync protection: + [`SyncCatalogStore.kt`](https://github.com/Davidona/StreamVault-IPTV/blob/593333714c43cb1802e14a1452cd9f5906e0a286/data/src/main/java/com/streamvault/data/sync/SyncCatalogStore.kt) +- persistent per-category hydration state: + [`Entities.kt`](https://github.com/Davidona/StreamVault-IPTV/blob/593333714c43cb1802e14a1452cd9f5906e0a286/data/src/main/java/com/streamvault/data/local/entity/Entities.kt#L850-L908) +- adaptive page loading with resume and page fingerprints: + [`SyncManager.kt`](https://github.com/Davidona/StreamVault-IPTV/blob/593333714c43cb1802e14a1452cd9f5906e0a286/data/src/main/java/com/streamvault/data/sync/SyncManager.kt#L1933-L2180) +- sanitized capture-to-fixture workflow: + [`fixtures/README.md`](https://github.com/Davidona/StreamVault-IPTV/blob/593333714c43cb1802e14a1452cd9f5906e0a286/data/src/test/resources/stalker/fixtures/README.md#L1-L29) +- auth payload caveat: + [`OkHttpStalkerApiService.kt`](https://github.com/Davidona/StreamVault-IPTV/blob/593333714c43cb1802e14a1452cd9f5906e0a286/data/src/main/java/com/streamvault/data/remote/stalker/OkHttpStalkerApiService.kt) + +Because of the custom license, these are architecture observations only. +Its hydration/resume model is useful, but its pagination completion semantics +are not: `totalPages()` silently caps a result at 200 pages: + +[`OkHttpStalkerApiService.kt`](https://github.com/Davidona/StreamVault-IPTV/blob/593333714c43cb1802e14a1452cd9f5906e0a286/data/src/main/java/com/streamvault/data/remote/stalker/OkHttpStalkerApiService.kt#L1588-L1601). + +At a safety cap, prefer tuliprox's explicit incomplete outcome. + +### tuliprox Patterns Worth Re-implementing + +tuliprox is valuable for defensive transport design: + +- preserve custom path prefixes while trying `server/load.php`, `portal.php`, + and `/c` +- keep a stateful cookie context +- tolerate JSONP, BOMs, and HTML error responses +- cap response bodies +- redact tokens, MACs, and credentials in diagnostics +- serialize session refresh behind a lock +- separate learned endpoint capabilities from identity-bound data + +Its generic HLS segment cache is not evidence for a Stalker catalog cache, and +its current identity/profile values should not be copied as a protocol oracle. + +Its catalog pagination does add two useful contracts: + +- accept `total_items`, `max_page_items`, and `max_page` as separate hints +- return an explicit incomplete result at a safety cap instead of silently + presenting a truncated catalog as complete + +See +[`catalog.rs`](https://github.com/euzu/tuliprox/blob/bcc1badcfeb484ffbb325222f80b19bde850f314/backend/src/utils/network/stalker/catalog.rs#L364-L440). +Support for object-keyed `data` payloads is another inexpensive compatibility +improvement worth testing. + +### OwnTV: A Safer Minimal Baseline + +OwnTV intentionally leaves SN/device/signature blank until a real incompatible +portal demonstrates a need. It keeps session state per source, serializes +authentication with a mutex, retries only once, and keeps the endpoint that won +a bounded probe for the current in-memory session: + +- [`StalkerAuthManager.kt`](https://github.com/ahXN00/OwnTV/blob/26f68f558f6879045755eea92d614773dd501d5a/app/src/main/java/tv/own/owntv/core/stalker/StalkerAuthManager.kt#L22-L104) +- [`StalkerClient.kt`](https://github.com/ahXN00/OwnTV/blob/26f68f558f6879045755eea92d614773dd501d5a/app/src/main/java/tv/own/owntv/core/stalker/StalkerClient.kt#L96-L136) + +This supports IPTVnator's MAC-only default, but OwnTV is not a complete protocol +reference: it has no status-2 `do_auth`, assumes a five-minute token TTL, and +does not recognize every plain-text authorization body. + +### Ideas Versus Folklore in ynoTV and uiptv + +ynoTV offers useful product patterns around SQLite, EPG, watchlists, reminders, +DVR, and VOD/series fallback. Its auth identity is not safe evidence: it +derives serial and device fields from undocumented MD5/SHA formulas, duplicates +the device ID, logs a Bearer token, and has no conditional `do_auth`: + +- identity formulas: + [`stalker-client.ts`](https://github.com/tbeezy/ynotv/blob/3219ca0c7717c07f40e52a4661fd118a219a6c81/packages/local-adapter/src/stalker-client.ts#L42-L105) +- auth, token refresh, and token logging: + [`stalker-client.ts`](https://github.com/tbeezy/ynotv/blob/3219ca0c7717c07f40e52a4661fd118a219a6c81/packages/local-adapter/src/stalker-client.ts#L430-L615) +- hard-coded pagination: + [`stalker-client.ts`](https://github.com/tbeezy/ynotv/blob/3219ca0c7717c07f40e52a4661fd118a219a6c81/packages/local-adapter/src/stalker-client.ts#L820-L880) + +uiptv has an appealing endpoint idea: inspect `xpcom.common.js` before probing +common API paths: + +[`PingStalkerPortal.java`](https://github.com/xixogo5105/uiptv/blob/e3bd949cc7819e9438efadbe45aed6bbf779af28/core/src/main/java/com/uiptv/util/PingStalkerPortal.java#L31-L154). + +However, its tests preserve a +[duplicated path](https://github.com/xixogo5105/uiptv/blob/e3bd949cc7819e9438efadbe45aed6bbf779af28/core/src/test/java/com/uiptv/util/PingStalkerPortalTest.java#L20-L35) +and a +[missing slash](https://github.com/xixogo5105/uiptv/blob/e3bd949cc7819e9438efadbe45aed6bbf779af28/core/src/test/java/com/uiptv/util/PingStalkerPortalTest.java#L103-L106). +Its auth also generates fresh UUID-like signature/random values: + +[`HandshakeService.java`](https://github.com/xixogo5105/uiptv/blob/e3bd949cc7819e9438efadbe45aed6bbf779af28/core/src/main/java/com/uiptv/service/HandshakeService.java#L90-L148). + +Borrow endpoint discovery as an independently tested behavior; do not borrow +the generated identity. + +## STBEmu and StbEmuTV + +Two products are easy to conflate: + +- Android STBEmu has public + [profile documentation](https://docs.stbemu.com/en/profiles.html) and + [common settings documentation](https://docs.stbemu.com/en/common_settings.html) +- macOS/iOS StbEmuTV is a separate closed-source product whose public evidence + is its + [App Store page](https://apps.apple.com/ca/app/stbemutv-premium/id1589654283?mt=12) + and version history + +The public Android documentation confirms that a successful emulator presents +a coherent STB profile: model, firmware, UA, MAC, optional serial and device +IDs, signature settings, vendor, and hardware version belong to one profile. +That coherence is a more plausible reason for broad compatibility than merely +sending more fields. + +The documentation also exposes a generic network cache and warns that it may +need to be disabled when authentication errors become too frequent. This is +strong evidence against applying a blind HTTP cache to handshake, profile, or +`create_link`. + +The macOS/iOS StbEmuTV release history mentions network cache settings, clear +cache/reset support, restored fast login, and repeated portal/login fixes. That +supports the user's observation that persistence exists, but the closed source +does not reveal whether the implementation stores HTTP resources, a semantic +catalog, tokens, cookies, or some combination. It should be treated as a +product clue, not a protocol specification. + +## Pagination and Cache Comparison + +### What IPTVnator Already Gets Right + +Current `main`: + +- tries the canonical `itv/get_all_channels` action first +- falls back to `get_ordered_list` +- computes `ceil(total_items / max_page_items)` +- fetches with bounded concurrency +- retries each failed page once +- de-duplicates channel IDs +- stops when a portal repeats a page or contributes no new IDs +- caps a crawl at 30,000 channels +- uses per-playlist-record single-flight state, falling back to portal URL when + no playlist ID exists, plus a 30-second failure cooldown +- serves stale data while a manual refresh is running + +See +[`stalker-itv-channel-loader.ts`](https://github.com/4gray/iptvnator/blob/9ae53e451570538770e831e2838a776796284907/libs/portal/stalker/data-access/src/lib/stalker-itv-channel-loader.ts) +and +[`stalker-itv-cache.service.ts`](https://github.com/4gray/iptvnator/blob/9ae53e451570538770e831e2838a776796284907/libs/portal/stalker/data-access/src/lib/stalker-itv-cache.service.ts). + +The official list logic and Kodi use the same page-count interpretation: + +- MAG list: + [`layer.list.js`](https://github.com/iptvhakr/stalker_portal/blob/72deceee1e32ea00cf33ecf2376b80902ab11134/c/layer.list.js#L230-L275) +- Kodi: + [`ChannelManager.cpp`](https://github.com/kodi-pvr/pvr.stalker/blob/07989d2d8e5542135c5c7107ffeb4c316f7f65fc/src/stalker/ChannelManager.cpp#L18-L58) + +The "search only sees 14 channels" report in +[`#1146`](https://github.com/4gray/iptvnator/issues/1146) was fixed on `main` +by [`#1209`](https://github.com/4gray/iptvnator/pull/1209), merged on +2026-07-23. It is still a valid symptom for the 2026-07-05 v0.22 release. + +### Recommended Persistent Cache + +The audited stock Stalker 4.9.35 `load.php` sets +`Cache-Control: no-store`; proprietary forks may differ: + +[`server/load.php`](https://github.com/iptvhakr/stalker_portal/blob/0eb23e4995222e7c3daa4b945a4e962703ebf0cc/server/load.php#L1-L21). + +Therefore persistent caching should be an explicit app-level semantic snapshot, +not an implicit HTTP cache. + +Recommended design: + +- SQLite last-good snapshots for live channels, categories, VOD/series + metadata, and EPG +- key by a tuple containing canonical endpoint, MAC, identity revision, content + type, category, and normalized request parameters +- render the last-good snapshot immediately and refresh in the background +- preserve the previous complete snapshot when a crawl is partial +- atomically replace a complete catalog; use upsert-only for a known partial + crawl so unseen rows are not deleted +- expose last-updated state, refresh progress, and "clear portal cache" +- invalidate on endpoint change, identity change, portal deletion, or schema + version change +- use separate TTL policies for catalogs and EPG +- defer large sync work while playback is sensitive to portal traffic +- optionally add SQLite FTS for global VOD/series/live search + +Do not persist: + +- Bearer tokens in the catalog cache +- `create_link` results or resolved stream URLs +- authorization error bodies +- server challenge values as reusable identity + +The current in-memory ITV cache should remain the first layer even if a disk +snapshot is added. + +## IPTVnator Issue Signals + +### Endpoint and Empty Catalog + +The strongest unresolved cluster points to URL/endpoint detection: + +- [`#686`](https://github.com/4gray/iptvnator/issues/686): root URL and invalid + URL/empty-content symptoms; the portal works elsewhere +- [`#755`](https://github.com/4gray/iptvnator/issues/755): Stalker request + returns 404 while the same server/MAC works in another client +- [`#850`](https://github.com/4gray/iptvnator/issues/850): root URL rejected, + `/c` accepted +- [`#389`](https://github.com/4gray/iptvnator/issues/389): Ministra 5.6.0 path + rewriting followed by empty content +- [`#343`](https://github.com/4gray/iptvnator/issues/343): long-running empty + catalog reports, weakly closed before the current auth flow existed + +A 404 is much stronger evidence for the wrong path than for a missing serial. +Endpoint discovery should precede deeper fingerprint experiments. + +### Identity + +Issues `#860`, `#927`, and PR `#941` support the current explicit-identity +policy. The remaining part of +[`#345`](https://github.com/4gray/iptvnator/issues/345) is transport identity: +the import form contains a `userAgent` control, but it is not exposed or used by +the Stalker API. + +### Playback Is a Separate Failure Domain + +These reports show successful catalog access but built-in playback trouble: + +- [`#849`](https://github.com/4gray/iptvnator/issues/849) +- [`#910`](https://github.com/4gray/iptvnator/issues/910) +- [`#1158`](https://github.com/4gray/iptvnator/issues/1158) + +If catalog/VOD access works and an external VLC path receives a link, the +initial portal auth probably succeeded. Trace these separately: + +`create_link -> redirect chain -> headers/cookies -> first byte or segment -> +player startup -> stream/session release`. + +Potential causes include one-time links, missing same-origin cookies, codec or +container support, redirect handling, and an unreleased previous stream. The +external HTTP 462 fix is a testable hypothesis, not a confirmed root cause. + +## Ideas to Adopt + +### High Confidence + +1. Canonical conditional second-step state machine. +2. Safe endpoint/landing resolver with persisted learned result. +3. One coherent transport/device profile and explicit traced identity. +4. Portal + identity scoped cookie jar and session single-flight. +5. Body-aware auth/error taxonomy with one bounded refresh. +6. Profile-derived watchdog timing. +7. Redacted connection-test report. +8. Playback-session tracing and failure classification. +9. Bounded response sizes and secret redaction. + +### Useful Product Ideas + +- signed same-origin image proxy +- global indexed search +- background catalog refresh with progress +- semantic last-good SQLite cache with stale-while-revalidate +- last-updated and stale-state indicators +- manual "clear portal cache" and "re-detect endpoint" +- exportable sanitized compatibility report +- delay large catalog synchronization during live playback +- persist learned non-secret endpoint/profile capability hints +- fixture-test tolerant JSONP/BOM/object-keyed response parsing +- fixture-test HTTP 462 recovery by releasing the previous stream session and + recreating `create_link` + +### Do Not Adopt Without New Evidence + +- `SHA1(MAC)` as a real MAG prehash +- `MD5(MAC)` as a serial +- identical `device_id` and `device_id2` +- signatures derived from arbitrary concatenations of MAC, serial, IDs, and + portal URL +- `PHPSESSID=null` +- fake `__cfduid` +- hard-coded token TTLs +- `auth_second_step=1` before successful `do_auth` +- automatic identity rotation or recipe cycling +- generic HTTP caching of handshake/profile/`create_link` + +## Implementation Options + +### Option A: Authentication Correction Only + +Implement the state machine, fix headers/profile coherence, classify 403/body +errors, and use returned watchdog timing. + +Advantages: + +- smallest runtime change +- directly addresses the highest-confidence auth bugs +- easier to cover with contract tests + +Limitations: + +- root/custom-path portals can still fail before authentication +- no real cookie persistence +- no offline/fast-start catalog improvement + +### Option B: Staged Compatibility Foundation (Recommended) + +Stage 1: + +- build sanitized fixture/replay tests +- add endpoint/landing discovery +- introduce a portal/identity session key and cookie jar +- implement the canonical auth state machine and coherent profile presets +- remove unsupported synthetic identity behavior + +Stage 2: + +- add redacted connection diagnostics +- persist learned endpoint/profile capabilities +- add semantic SQLite last-good cache and optional FTS +- evaluate adaptive/resumable pagination against deterministic fixtures + +Stage 3: + +- trace and fixture-test `create_link`/redirect/HTTP 462/session release + behavior, then implement only confirmed recovery rules + +Advantages: + +- addresses the failure order seen in real issues +- isolates auth correctness from caching and playback +- creates evidence before adding more identity recipes +- yields incremental, testable releases + +Limitations: + +- requires session ownership across renderer/main/DB boundaries +- needs migration decisions for existing token and synthetic-cookie data + +### Option C: Full Emulator-Style Recipe Engine + +Add multiple device presets and automatically try endpoint, firmware, hash, and +identity recipes. + +Advantages: + +- may reach unusual proprietary forks +- can expose a powerful advanced compatibility UI + +Risks: + +- highest chance of binding or blocking a real account +- opaque native MAG values cannot be reconstructed reliably +- difficult to test without user-owned traces +- automatic recipe cycling can look like abusive traffic +- much larger security and support surface + +Do not choose this as the default. If pursued later, restrict it to explicit, +stable, user-controlled profiles with strong warnings and no automatic identity +rotation. + +## Verification Strategy for Future Runtime Work + +Before implementation, create redacted fixtures for: + +- root URL, `/c/`, custom-prefix `/c/`, `portal.php`, and + `server/load.php` +- same-origin and cross-origin landing redirects +- handshake with and without `random` +- profile statuses normal, blocked, and credentials-required +- successful and failed `do_auth` +- `store_auth_data_on_stb` enabled and disabled +- HTTP 401, auth-style 403, WAF-style 403, 404 HTML, 429, and 5xx +- HTTP-200 `Authorization failed.` and `Access denied.` +- cookie set/rotation and isolation between two portals +- two playlist records that share endpoint + MAC +- repeated page, partial page failure, wrong totals, and large catalogs +- HTTP 462 and stale one-time `create_link` + +Required regression assertions: + +- first profile always uses `auth_second_step=0` +- second profile cannot occur before successful `do_auth` +- blank optional identity fields remain absent +- changing unrelated settings never rotates identity +- no secret crosses an unvalidated redirect origin +- only one session refresh runs for concurrent failures +- `Access denied.` without invalid-token evidence does not trigger a refresh +- a partial sync cannot delete last-good rows +- token, MAC, credentials, and server cookies are redacted from exported reports + +Fixture replay is necessary but not sufficient. The final compatibility pass +needs opt-in tests against user-owned portals or a controlled middleware +fixture. No private credentials or real catalog artwork should enter the +repository or CI artifacts. + +## Licensing Notes + +- Kodi is GPL: learn from observable protocol behavior and independently + implement it; do not copy code into IPTVnator's MIT codebase. +- StreamVault is source-available under a custom non-commercial license: + architecture ideas only. +- OwnTV is GPL-3.0 and ynoTV is AGPL-3.0: independently re-implement observed + behavior, not code. +- Schrittfisch2000/Stalker-Client had no license at the audited commit: no code + reuse. +- tuliprox and uiptv are MIT, but both still require behavior-by-behavior + verification; tuliprox's Stalker support is recent and uiptv's endpoint tests + include malformed cases. +- Scanner/exploitation-oriented repositories were excluded as implementation + references even when they had substantial stars. + +## Original Recommended Decision (Accepted) + +The accepted recommendation was Option B, with the first implementation stage +focused on: + +1. sanitized fixtures and endpoint resolver +2. canonical auth state machine +3. coherent transport profile and real cookie jar +4. scoped token refresh and diagnostics + +Persistent catalog cache, adaptive/resumable pagination, authenticated +web-player/download context consumers, and the remaining playback-session +lifecycle are intentionally separate follow-up work. This order targeted the +strongest issue evidence, avoided unsafe identity invention, and provided a +testable base for later compatibility work. diff --git a/docs/architecture/stalker-epg.md b/docs/architecture/stalker-epg.md index 437f251be..acced42ee 100644 --- a/docs/architecture/stalker-epg.md +++ b/docs/architecture/stalker-epg.md @@ -256,8 +256,12 @@ EPG requests follow the standard Stalker request path: | Full Stalker portal | `StalkerSessionService.makeAuthenticatedRequest()` | | Simple Stalker portal | generic IPC request path via Electron | -No EPG-specific backend transport was needed; the Electron Stalker request -handler forwards portal params directly. +No EPG-specific backend transport was needed. On the full-session path, +renderer-side `adaptLegacyStalkerRequest` first converts the legacy request +shape into typed `ShortEpg` / `EpgInfo` parameters. Electron main validates +those typed parameters, and `stalker-operation-adapter` reconstructs the +allowlisted portal wire request before authenticated transport. Only the +simple/legacy path forwards portal parameters through generic Electron IPC. ## Fallback Behavior diff --git a/docs/architecture/stalker-mock-server.md b/docs/architecture/stalker-mock-server.md index a15bb061a..c5bc2d103 100644 --- a/docs/architecture/stalker-mock-server.md +++ b/docs/architecture/stalker-mock-server.md @@ -12,8 +12,15 @@ This document describes the design decisions, data flow, and extension points of The mock server enables: 1. **Local development** without access to a real Stalker portal -2. **Playwright E2E testing** with predictable, deterministic data +2. **Playwright E2E testing** with predictable scenario shapes and stable IDs 3. **Scenario-based testing** via predefined MAC addresses that map to specific data shapes +4. **Stateful authentication replay** with exact request order, cookie, + redirect, classifier, refresh, and terminal-state assertions + +The generated catalog server and authentication replay are deliberately +separate. Port `3210` remains a convenient long-running development portal. +Authentication E2E creates isolated ephemeral loopback listeners and never +shares their state with that server. ## Key Design Decisions @@ -23,8 +30,12 @@ Per-request random data would break navigation: if category IDs change between c - Data is generated **once per MAC address** on first request, then cached in memory. - `@faker-js/faker` is seeded with the scenario's `seed` value before generation: predefined scenario MACs use fixed seeds from `scenarios.ts`; unknown MACs derive the seed from the MAC via `macToSeed()`. -- Same MAC → identical data on every server restart. -- Restart the server to reshuffle all data. +- The same MAC keeps the same scenario, IDs, and seed-driven catalog structure + across regeneration, and the in-memory cache keeps requests internally + consistent between `/reset` or process-restart events. +- A restart is not byte-for-byte deterministic: ratings and some dates use + runtime randomness, while EPG is anchored to the current day. Change the + scenario seed to intentionally reshuffle the seed-driven fields. ### MAC Address as Identity @@ -32,7 +43,9 @@ Stalker portals use MAC address as the primary credential. The mock server follo - Each unique MAC gets its own isolated dataset. - Predefined MACs map to specific `ScenarioConfig` shapes (see `src/app/scenarios.ts`). -- Unknown MACs use the sum of their byte values as a seed, producing unique but deterministic data. +- Unknown MACs use the sum of their byte values as a seed for their catalog + structure. Different MACs still have isolated in-memory state; seed sums can + collide, so uniqueness is not promised. ### In-Memory Only @@ -267,7 +280,50 @@ Playwright waits for both servers to be healthy before starting tests. If either Each stalker e2e test calls `POST http://localhost:3210/reset` in `beforeEach` to clear in-memory state. This ensures tests don't bleed favorites or other mutable state into each other. -`resetAll()` clears both the generated-content cache and in-memory favorites (`data-store.ts`). Because generation is seed-deterministic, the next request regenerates identical content, so the observable data does not change across resets. +`resetAll()` clears both the generated-content cache and in-memory favorites +(`data-store.ts`). The next request regenerates the same seeded IDs and scenario +shape, but volatile ratings, dates, and current-day EPG may differ. Tests should +assert the intended structure or behavior rather than byte equality of the full +payload. + +## Authentication Replay Architecture + +Replay fixtures live under +`apps/stalker-mock-server/fixtures/replay/`. A version-1 fixture declares: + +- named origins and an entry URL; +- ordered or unordered phases; +- exact request method, path, headers, query, cookies, and body expectations; +- typed generated inputs and exact references to them; +- request cardinality, barriers, and the required terminal state. + +`createReplayServerRun()` binds one ephemeral `127.0.0.1` listener per named +origin. Every public URL includes a run-specific route prefix, which the +listener strips before matching. Symbols, cookies, counters, and ledgers are +isolated per run. + +Electron E2E does not call the listeners directly through an ambient global +server. `withStalkerReplayRun()` starts a process-local control plane whose +capability is never passed to the app. The control plane accepts only +repository-allowlisted fixture IDs, exact loopback Host headers, bounded JSON +bodies, and create/finalize/dispose operations. Finalization verifies exact +cardinality, barriers, the expected endpoint, and terminal state; the ledger +contains sanitized operation and mismatch counts only. + +Committed fixtures are a security boundary: + +- use only generated locally administered MACs, safe credentials, tokens, and + cookies; +- never include real portal origins, accounts, catalogs, artwork, or streams; +- validate the whole corpus with + `pnpm run stalker:fixtures:validate`; +- run `pnpm nx test stalker-mock-server` and + `pnpm nx test stalker-fixture-tools` after schema/runtime changes. + +`pnpm run stalker:fixtures:draft -- ` creates a +sanitized draft outside the repository. It is not a substitute for review: +the converter fails closed on unsafe files, repository inputs, unknown origins, +oversized/deep structures, and secret-like evidence. ### Recommended Test Structure @@ -292,6 +348,9 @@ test('browse VOD categories', async ({ page }) => { - **New content types**: Add a new generator function in `data-generator.ts` and a new handler in `handlers/`. - **New scenarios**: Add to `SCENARIOS` in `scenarios.ts`. -- **Stateful session tokens**: `handshake.handler.ts` generates a token from the MAC — extend this to track token expiry for testing re-auth flows. -- **Error simulation**: Add a special MAC or query param to trigger error responses (e.g. 401, 500) for testing error handling in the Stalker store. +- **Authentication/session behavior**: Add a validated replay fixture instead + of overloading generated scenario MACs or introducing test-only query + switches. +- **Catalog-only errors**: Add a generated scenario only when the behavior + belongs to the long-running development portal rather than session replay. - **Slow responses**: Add a `MOCK_DELAY_MS` env var and apply it in middleware for testing loading states. diff --git a/docs/architecture/stalker-portal.md b/docs/architecture/stalker-portal.md index 930b2c554..76f09034c 100644 --- a/docs/architecture/stalker-portal.md +++ b/docs/architecture/stalker-portal.md @@ -12,6 +12,7 @@ This document describes the Stalker portal implementation in IPTVnator and where - [Download Manager](./download-manager.md) - [Category Management](./category-management.md) - [Stalker Store API Baseline](./stalker-store-api-baseline.md) +- [Stalker Authentication and Client Compatibility Audit](./stalker-authentication-compatibility-audit.md) ## Scope @@ -44,12 +45,134 @@ Primary route tree lives in ## Runtime Architecture -1. Angular Stalker screens call methods/resources in `StalkerStore`. -2. `StalkerStore` builds request params based on selected content type and current view state. -3. Requests go through `DataService.sendIpcEvent(STALKER_REQUEST, ...)` or `StalkerSessionService` (full portal auth). -4. Electron main process handles `STALKER_REQUEST` in - `apps/electron-backend/src/app/events/stalker.events.ts`. -5. Axios calls Stalker `load.php` API with required headers/cookies and returns the raw `response.data` to the renderer; normalization happens in the store feature slices. +Stalker has two deliberately separate request paths: + +1. Electron full portals use `StalkerSessionService`, which exposes only typed + application operations and opaque lease/challenge references. +2. Electron main handles `STALKER_SESSION_OPEN`, `CONTINUE`, `REQUEST`, and + `CONTROL` in `stalker-session.events.ts`. `StalkerSessionManager` owns + endpoint discovery, the RFC cookie jar, handshake/profile state, credentials, + token generations, serialized refresh, watchdogs, and playback contexts. +3. `stalker-operation-adapter.ts` maps the allowlisted catalog, EPG, search, + detail, and `create_link` operations to portal wire parameters. Renderer + callers cannot submit raw auth actions or managed auth parameters. +4. A successful `create_link` returns the stream URL plus a one-use, + sender-bound `playbackContextRef`. Embedded MPV and external MPV/VLC IPC + consume that reference to obtain main-owned authorization and cookie + headers. Built-in web players currently carry but do not consume the + reference. +5. Explicitly simple Electron portals and the PWA keep the legacy + `STALKER_REQUEST`/HTTP adapter. They do not acquire a main-owned full + session. + +The primary ownership points are: + +- pure URL, response, identity, state-machine, and request-policy rules: + `libs/portal/stalker/protocol/` +- renderer facade and playlist descriptor mapping: + `libs/portal/stalker/data-access/src/lib/stalker-session*.ts` +- route connection orchestration: + `libs/portal/stalker/feature/src/lib/stalker-connection-flow/` and + `stalker-workspace-route-session.service.ts` +- Electron runtime: + `apps/electron-backend/src/app/services/stalker-session/` +- typed IPC registration: + `apps/electron-backend/src/app/events/stalker-session.events.ts` + +Bearer tokens, handshake randoms, server cookies, credentials, internal session +keys, and player headers must never be returned through preload, diagnostics, +or persisted playlist metadata. + +## Connection Classification and Persistence + +Endpoint discovery starts from a source that may be a root URL, landing +directory, `portal.php`, `server/load.php`, or a custom prefix. The landing +request is anonymous. Derived candidates are bounded, de-duplicated, and +probed sequentially with an isolated jar. A validated handshake plus first +profile selects `full-session`; only an explicitly unsupported auth shape plus +a recognized read-only catalog response may select `stateless-mac`. + +An origin-changing redirect pauses before identity-bearing traffic and requires +confirmation showing the exact source and target origins. An explicitly +entered private or loopback source remains supported. Anonymous discovery +rejects a public-to-private redirect before the target hop is contacted; +identity-bearing cross-origin redirects pause before target preparation or +contact and require explicit approval. Profile status `2` opens the +username/password challenge; `do_auth` is sent only then, and +`auth_second_step=1` is sent only after canonical `do_auth` success. Blank +credentials are never submitted. The three-submission credential budget +belongs to the whole connection attempt and survives auth-session replacement +or coordinator epoch changes. + +Imports, explicit re-detection, and lazy migration use provisional attempts: + +1. resolve and authenticate without mutating the stored playlist; +2. build a non-secret persistence draft; +3. atomically save the playlist row; +4. commit the provisional main-process session; +5. report Connected and initialize the Stalker store. + +Cancel or route navigation discards the provisional attempt. A failed local +write retains the bounded ready attempt and draft so Save Again can retry +persistence without repeating discovery or authentication; abandoning the +retry discards it. Existing route content is not initialized before the +connection flow is ready. Terminal open, recovery, and lease-activation +failures keep their stable reason visible in an actionable snackbar; Retry +performs one explicit re-detection instead of silently exposing the playlist +or looping in the background. + +Response handling is fail-closed. Unknown profile statuses, malformed JSONP, +valid JSON with an incompatible MIME type, invalid handshake/catalog shapes, +and ambiguous auth bodies stop as `incompatible-response`; they cannot silently +downgrade a full portal to stateless mode. Only explicit unsupported endpoint +statuses (`404`, `405`, and `501`) or a narrowly classified, non-auth, +plain-HTML endpoint-shape miss with the known Stalker/Ministra landing-shell +markers advances discovery. Generic HTML/XML, account denials, gateway pages, +and unknown protection pages remain terminal. An HTML `404` remains eligible +for the next bounded candidate. A valid API envelope with an incompatible MIME +type remains fail-closed for ordinary candidates; only a persisted learned +endpoint may treat a body rejected solely by media type as a stale-hint miss +and continue bounded discovery. + +Persisted compatibility fields are: + +- `stalkerSourceUrl` +- verified `portalUrl` and `stalkerLandingUrl` +- `stalkerRequestRecipe` and `stalkerRecipeClassifierVersion` +- `stalkerProfilePreset` +- explicit `stalkerIdentityOverrides` +- explicit `stalkerTransportConfiguration` +- `stalkerLastVerifiedAt` +- `isFullStalkerPortal` for legacy compatibility + +`stalkerToken`, cookies, random values, leases, challenges, and playback +contexts are never persisted. Saved credentials are loaded through the non-EPG +database worker as an attempt-scoped candidate only when source, MAC, profile +preset, effective identity, and effective transport configuration match. They +become session credentials only after canonical authentication succeeds; +stateless reclassification clears the stored pair. + +Deleting one playlist waits for the database delete and then cleans all +main-owned attempts, leases, watchdogs, and playback contexts for that playlist. +Delete-all destroys the complete Stalker session runtime. Failed database +deletes do not tear down sessions. + +A current `stalkerRequestRecipe` plus classifier version is authoritative and +overrides the legacy `isFullStalkerPortal` hint. Missing or stale recipes are +classified provisionally. If a current stateless recipe receives `404`, `405`, +`501`, or an incompatible endpoint envelope, the original operation performs +one single-flight provisional re-detection, persists and commits the updated +recipe, then reissues itself once. + +## Session Watchdog + +Each active full-session principal has one main-process watchdog. Its interval +prefers a positive profile `timeslot`, then `watchdog_timeout`, defaults to 25 +seconds, clamps to 10 seconds–5 minutes, and applies ±10% jitter. A token +rejection joins the session's single-flight refresh. Other classified failures +remain attached to the session and are surfaced on the next request until a +later successful profile check clears them. Removing the last active lease +stops the watchdog. ## Main UI Components @@ -115,24 +238,31 @@ blank fields are not generated or forwarded to `get_profile`. - User-provided `sn`, `device_id`, `device_id2`, `signature`, and `signature2` values are trimmed, persisted under the canonical `stalker*` playlist fields, - and reused for initial auth, token refresh, retry auth, normal API requests, - and same-origin playback headers. + and reused for the `get_profile` / `do_auth` steps of initial auth, token + refresh, and retry auth. Full-session catalog requests and same-origin + playback receive the session-owned Bearer token and cookies, not these + optional identity fields. - Empty optional identity fields remain absent. IPTVnator must not generate a device ID from the MAC address or duplicate `device_id2` from `device_id1`. - The legacy default serial value `BEDACD4569BAF` is treated as absent at runtime so older blank imports do not keep sending a synthetic serial number. - Playback headers use the same serial normalization, so the legacy default is - not sent as `SN` or as a serial-derived `__cfduid`. MAC-only API and - playback requests do not synthesize `__cfduid`; when a real serial is - present, same-origin playback uses a canonical 32-character `__cfduid` - protocol cookie. + not sent as `SN` or as a serial-derived `__cfduid`. The main-owned + full-session path never synthesizes `__cfduid`; it retains real + portal-issued cookies instead. The stateless/simple compatibility path still + derives the 32-character value when an explicit serial is present, so it + remains a legacy quirk rather than a canonical identity mechanism. The + [compatibility audit](./stalker-authentication-compatibility-audit.md) + records why it must not be reintroduced into the full-session profile. - Generated MAG-like identity remains a future explicit setting. It must not be the default because strict portals can bind accounts to the first device fingerprint they receive. -- Stalker workspace routes must initialize `StalkerStore` from a playlist object - with an explicit `isFullStalkerPortal` mode. If the active route metadata is a - lightweight playlist record without that field, the route session must load the - full playlist by id before category/content resources run. Stalker auth +- Stalker workspace routes may initialize `StalkerStore` directly from active + route metadata only when it contains a request recipe produced by the current + classifier version. A legacy `isFullStalkerPortal` boolean, even when + present, is not enough: the route session must load the full playlist by id + and finish any lazy connection migration before category/content resources + run. Stalker auth metadata is independent from M3U playlist EPG metadata and must not depend on M3U-specific EPG fields. @@ -363,6 +493,11 @@ Navigation rule to preserve: - Stalker radio favorites/recent items are the exception to VOD/series inline detail opening: they are normalized as live items and must open through the shared live collection audio-player path. +- Global live/radio collection playback loads the stored playlist and follows + its current request recipe. Eligible legacy or stale Electron records + classify single-flight and persist before commit; explicitly simple legacy + records remain on the simple adapter. Migrated `create_link` results retain + their opaque playback context. - VOD-backed series favorites can be displayed in series collections, but detail opening must preserve their VOD origin: `is_series=1` favorites set the selected content type to `vod` so the lazy Ministra season/episode resources @@ -403,28 +538,55 @@ snapshot-first + background re-fetch contract: ## Backup and Restore Versioned playlist backups include Stalker connection metadata plus playlist- -scoped favorites/recent snapshots. +scoped favorites/recent snapshots. Secret export is off by default. -Exported fields: +Required exported connection fields: - `portalUrl` - `macAddress` -- `isFullStalkerPortal` -- optional `username` / `password` -- optional request headers (`userAgent`, `referrer`, `origin`) -- full-portal serial/device/signature fields when present - favorites and recently viewed collections -Excluded fields: +Optional non-secret connection fields are exported when present: + +- original `sourceUrl` +- `isFullStalkerPortal` +- profile preset and transport configuration +- legacy request headers (`userAgent`, `referrer`, `origin`) + +Exported only after the user explicitly enables credential and explicit +identity-override export: + +- `username` and `password` +- explicit serial, device IDs, signatures, prehash, API signature, and custom + firmware/hardware tuple + +Always excluded: - `stalkerToken` - `stalkerAccountInfo` +- `stalkerLandingUrl`, `stalkerRequestRecipe`, + `stalkerRecipeClassifierVersion`, and `stalkerLastVerifiedAt` +- cookies, handshake randoms, leases, challenges, and playback contexts - playback positions in backup v1 +Compatibility `portalUrl` remains exported as connection metadata, but restore +re-resolves from `sourceUrl` when it is present. + Import rule: - backups restore the saved portal definition and replace the stored favorites/recent state for the matched playlist +- redacted Stalker backups preserve local credentials only for an exact + exported-id/source/MAC/profile match; otherwise they create a credential-less + row that follows the normal status-2 connection flow +- secret-bearing Stalker matches normalize source, MAC, profile, and effective + identity/transport. A matching exported ID is preferred; otherwise exactly + one exact username/principal match is required. Password is never part of + the fingerprint, so an exported-ID match may patch it. Structured + identity/transport takes precedence, with equivalent legacy + serial/device/signature and UA/Referer/Origin fields as fallback +- every Stalker restore clears learned landing/recipe/classifier/verification + fields, including on matched merges - a fresh handshake must happen after import for full-portal sessions; imported backups never trust a serialized token @@ -465,6 +627,45 @@ Stalker reuses some Xtream UI infrastructure deliberately: This reduces duplicate UI logic across portal types and keeps compatibility behavior aligned. +The main-owned authenticated playback consumer currently covers Embedded MPV +and external MPV/VLC. Built-in HTML5, Video.js, and ArtPlayer playback can +carry the opaque `playbackContextRef` but does not consume it, so streams that +require header/cookie authorization beyond their URL remain unsupported there. +The shared download flow likewise uses its older stream-URL contract. +Authenticated Stalker web playback and downloads remain explicitly deferred +until those surfaces can consume a main-owned context without exposing or +persisting headers and cookies. + +## Authentication Replay Fixtures + +Stateful authentication fixtures live under +`apps/stalker-mock-server/fixtures/replay/`. A fixture declares named +loopback origins, ordered or unordered phases, exact request expectations, +typed generated symbols, cardinality, barriers, and a terminal state. Every +run is isolated and must be finalized and disposed; the ledger contains only +sanitized operation and mismatch counts. + +The Electron E2E harness starts the replay control plane with a process-local +capability, requests only repository-allowlisted fixtures, and gives the app +only synthetic portal inputs. Real portal URLs, credentials, catalogs, stream +links, and artwork must never be committed as fixtures. + +Use: + +```bash +pnpm run stalker:fixtures:validate +pnpm nx test stalker-mock-server +pnpm nx test stalker-fixture-tools +pnpm nx run electron-backend-e2e:e2e-ci--src/stalker-auth.e2e.ts +pnpm nx run electron-backend-e2e:e2e-ci--src/stalker-route-auth.e2e.ts +pnpm nx run electron-backend-e2e:e2e-ci--src/backup-security.e2e.ts +``` + +The HAR converter is a local draft aid, not a sanitizer of last resort. It +rejects unsafe paths, oversized/deep inputs, unrecognized origins, and +secret-like evidence before atomically writing output outside the repository. +Review and validate every draft before moving it into the fixture tree. + ## Regression Coverage Focused regression tests for Stalker VOD mode branching and the cross-surface @@ -485,3 +686,18 @@ Covered scenarios include: - Inline and external episode handoffs carry resolved season/episode metadata - Dashboard activity classifies `is_series` VOD as series and resolves its saved episode position + +Session compatibility coverage additionally lives in: + +- `portal-stalker-protocol`: URL recipes, response classifiers, auth state, + identity revision, and reserved request policy +- `stalker-mock-server` and `stalker-fixture-tools`: stateful replay, + schema/cardinality enforcement, control-plane security, and secret scanning +- `electron-backend`: redirects/SSRF, cookie ownership, resolver, auth, + coordinator, manager, watchdog, playback context, IPC validation, and saved + credential lookup +- `portal-stalker-data-access`, `portal-stalker-feature`, and + `playlist-import-feature`: opaque facade, recovery, challenges, lazy route + migration, and save-before-commit behavior +- `services` and `web`: backup exclusion, safe restore matching, and explicit + secret-export UX