Files
iptvnator/libs/playlist
4grayandClaude Fable 5 b92503feae feat(stalker): endpoint probing + behavior-based portal mode with lazy repair (#1344)
* feat(stalker): endpoint probing + behavior-based portal mode with lazy repair

Replace the URL-shape guess behind isFullStalkerPortal with real endpoint
discovery: at import, probe portal.php -> server/load.php ->
stalker_portal/server/load.php (the pasted .php endpoint first) and
classify the portal by observed behavior — a token-less itv/get_genres
answering data proves a token-free panel, the middleware's plain-text
auth failure proves the endpoint enforces the token, confirmed by the
real handshake + get_profile. The proven endpoint and mode are persisted.

The three diverging portal-mode predicates (import, session service,
legacy migration) collapse into one shared helper in
@iptvnator/shared/interfaces; executeStalkerRequest becomes the single
request choke point (search and the collection stream resolver fold in),
and the production-dead makeStalkerRequest copy is removed.

Existing misclassified playlists repair themselves lazily: only after a
request actually fails with the plain-text auth bodies, HTTP 404, or a
terminal handshake error, at most once per playlist per session, and only
a configuration discovery proved to answer is persisted — via a minimal
portalUrl/isFullStalkerPortal patch, so favorites, recents and playback
positions survive. Working reseller panels are never probed or rewritten;
there is deliberately no eager one-shot migration, because tolerant
portal.php panels cannot be told apart from misclassified canonical
portals without probing.

The Electron handler now embeds the HTTP status code in the error message
(ipcRenderer.invoke strips custom properties from rejections), and probe
requests carry silent:true so expected 404s do not toast error snackbars.
The stalker mock gains a portal.php-less /ministra host so e2e can prove
the 404 fallthrough end to end.

Fixes #850, #686, #755.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): sync watchdog, PWA proxy errors and cmd resolution with lazy repair

Review round 1 (Greptile P1, Codex P1/P2):

- A successful repair now re-syncs the ACTIVE watchdog playlist via the new
  StalkerSessionService.refreshActiveWatchdogPlaylist(): a simple-to-full
  repair starts the required keepalive mid-session, full-to-simple stops it,
  and an endpoint change repoints the pings instead of leaving them on the
  activation-time snapshot.
- PwaService.forwardStalkerRequest surfaces the web-backend proxy's
  normalized { message, status } no-payload envelope as an HTTP error
  carrying the status, so endpoint discovery and the lazy repair can
  classify upstream 404s in the PWA too (previously payload unwrapping
  returned undefined and dead endpoints were unrepairable there). Probe
  requests pass silent:true and skip the error snackbar.
- fetchStalkerPlaybackLink and the collection StreamResolverService re-apply
  the repair override AFTER the request, so a relative create_link reply
  resolves against the endpoint that actually answered (the resolver keeps
  the /stalker_portal path segment as base, so this matters beyond origin).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): parse candidate URLs and tie repair overrides to their source config

Review round 2 (Codex P2 x2):

- Endpoint candidates are now derived from the parsed origin + pathname:
  a pasted URL carrying a query or fragment (host/c?key=value) no longer
  gets /portal.php bolted onto the query, which made every probe hit /c
  and persisted the non-API URL.
- A repair override is tied to the failing configuration it replaced.
  Playlists carrying anything else (the user edited the portal URL or mode
  through the playlist dialog) drop the override and re-arm the
  once-per-session probe latch, so edited metadata is used verbatim and
  may repair again if it fails.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): auth-gated probes, normalized offline fallback, mock docs sync

Review round 3 (Codex P1 x2, P2):

- A probe answered with HTTP 401/403 now classifies the endpoint as
  auth-required and attempts the real handshake instead of skipping the
  candidate: non-standard middlewares answer 401 where the stock server
  sends HTTP 200 + plain text, and such portals authenticated fine before
  discovery existed.
- The unreachable-host import fallback normalizes the pasted URL (origin +
  pathname) before the legacy /c -> portal.php rewrite, so a query or
  fragment can no longer make it persist the browser page URL - a 200 HTML
  answer from /c is not a repair trigger, which would have left the
  playlist empty for good.
- The stalker mock-server README and architecture doc now describe
  behavior-based discovery and the /ministra host instead of the retired
  URL-shape rule and its "known inconsistency" note.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): recognize JSON auth failures and guard repairs against mid-probe edits

Review round 4 (Codex P1 + P2):

- isStalkerAuthFailureResponse() recognizes the JSON envelope some panels
  answer instead of the plain-text body ({js:{error:"Authorization
  failed"}} / {js:{msg:...}}). Probe classification treats it as
  auth-required instead of token-free data, and the lazy-repair trigger
  fires on it at runtime — previously such a portal was persisted simple
  with no repair path at all.
- A repair is committed only after re-reading the persisted row and
  verifying it still carries the configuration that failed: a user who
  edits the portal URL (or deletes the playlist) during the multi-second
  probe now wins over the in-flight repair result for the old URL.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): probe past endpoint 5xx, sibling fallbacks, identity-aware repair guard

Review round 5 (Codex P2 x3):

- A probe that fails with a RESOLVABLE HTTP status keeps discovery going:
  a broken /portal.php handler answering 500 must not hide a healthy
  sibling endpoint. Only status-less failures (true network level) stop
  the loop. The Electron handler now gives real HTTP 5xx responses the
  same parseable "HTTP Error <code>" message shape as 4xx, so the
  renderer can tell them apart from ECONNREFUSED/timeouts after
  ipcRenderer strips the object shape.
- Standard fallback candidates for a nonstandard pasted endpoint
  (.../cp/api.php) derive from its DIRECTORY, so recovery probes hit
  /cp/portal.php instead of /cp/api.php/portal.php.
- The repair's row re-verification also compares the MAC and all Stalker
  identity fields: a probe authenticated as the old identity must not
  install its token/watchdog or persist onto a row whose credentials were
  edited mid-probe.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): reactivation-safe watchdog, wider JSON auth phrases, per-config probe latch

Review round 6 (Greptile 4/5 concern + Codex P1/P2):

- setCurrentPlaylist applies the repair override before feeding the
  watchdog and store state: re-activating the portal route with the stale
  NgRx meta no longer stops or repoints the repaired keepalive back to
  the broken configuration.
- The structured js.error/js.msg fields accept the full phrase set the
  session service recognizes (Invalid token, Auth failed, bare
  unauthorized/authorization) — panels answering those envelopes were
  still classified token-free. Plain-text body matching stays narrow on
  purpose (HTML false positives).
- The once-per-session probe latch is keyed by the SOURCE configuration
  fingerprint (endpoint, mode, MAC, identity) instead of the playlist id:
  a repair discarded because of a mid-probe edit no longer blocks the
  edited configuration from repairing, while stale snapshots of an
  already-probed configuration still cannot loop the probe.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): identity-aware override invalidation and timeout-tolerant probing

Review round 7 (Greptile P1 + Codex P2):

- The repair override records the identity fingerprint the probe
  authenticated as. Editing the MAC or any Stalker identity field
  afterwards drops the override, the per-config probe latch AND the cached
  token, so requests and watchdog pings never pair the edited identity
  with a session negotiated for the previous one.
- A status-less probe failure that is a TIMEOUT (renderer budget, axios
  request timeout, ETIMEDOUT) continues to the next candidate — one
  hanging handler must not hide healthy siblings; connection-level
  failures (refused, unresolvable host) still stop discovery, so dead
  hosts keep failing fast.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): watchdog pings authenticate as the persisted row

Review round 8 (Greptile 4/5 concern):

The watchdog held its activation-time playlist snapshot for the whole
session, so portal metadata edited (or repaired) mid-session kept the
keepalive authenticating as the previous identity/endpoint — its pings
could keep the old session alive and repopulate the playlist-scoped token
cache with a token for the pre-edit identity.

Each ping now resolves the playlist from the persisted row first (the
single source of truth), falling back to the snapshot only when the store
cannot be read, and refreshes the snapshot on every successful read. Any
edit — identity, endpoint or mode — reaches the keepalive within one ping
cycle; a row now marked simple (or deleted) stops the watchdog. The
in-flight guard is claimed before the row read so overlapping pings
cannot double-fire.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): identity-tagged tokens, watchdog override overlay, retire-on-failure

Review round 9 (Greptile 4/5 concern + Codex P2):

- The session token cache is tagged with the identity fingerprint (MAC +
  all Stalker identity fields) the session was negotiated for; ensureToken
  re-authenticates instead of handing an edited identity the previous
  token. The fingerprint helper is shared (stalker-identity.utils) with
  the repair layer's override/latch checks.
- Watchdog pings overlay the repair layer's in-session override on the
  resolved row (registered decorator, no import cycle): a simple-to-full
  repair whose persistence is pending or failed no longer reads the stale
  row and stops the freshly started keepalive.
- makeAuthenticatedRequest retires a failed token even on the no-retry
  path (watchdog pings), so a dead session is never handed to the next
  caller.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): pending authentications are identity-scoped

Review round 10 (Greptile 4/5 concern):

pendingAuth entries carry the identity fingerprint they authenticate as.
A request for an edited identity no longer adopts an in-flight result
negotiated for the previous identity: it waits the old authentication out
(a competing handshake would strand it with a dead token on strict
portals) and then negotiates its own session. This was the last
id-only-keyed session structure — override, probe latch, token cache,
watchdog snapshot and pending auth are now all identity-aware.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): atomic repair persistence, full probe history, normalized offline classify

Review round 11 (Codex P2 x3 + P1 docs):

- The repair's row verification and patch now run ATOMICALLY inside the
  per-playlist write queue via the new
  PlaylistsService.transformPlaylistMeta(): a user edit that is queued but
  not yet committed wins over the repair — the transform sees the edited
  row and aborts instead of overwriting it. Write failures after a
  successful verification keep the session-only override, read failures
  discard the repair.
- The per-playlist probe latch keeps EVERY attempted source fingerprint,
  so alternating edits (A -> B -> A) cannot evict a fingerprint and let
  stale snapshots re-run discovery.
- The unreachable-host import fallback classifies the normalized
  origin+pathname, so a query merely mentioning /server/load.php cannot
  make a panel URL look canonical and abort the offline import.
- docs/architecture/stalker-portal.md documents the actual probe
  sequencing: any resolvable HTTP status (incl. 5xx) and timeouts continue,
  401/403 classify as auth-required, only connection-level failures abort.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): collision-proof session fingerprints

Review round 12 (Greptile P1): identity values are unrestricted strings,
so the delimiter-joined fingerprint could alias distinct identity tuples
(serial "a|b" + empty device vs serial "a" + device "b") and bypass the
identity invalidation. Both the identity fingerprint and the repair
source fingerprint are JSON-encoded now; regression test pins the exact
aliasing pair.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): preserve URL authority in normalization; document per-config latch

Review round 13 (Codex P1 docs + P2):

- normalizeStalkerPortalInputUrl mutates the parsed URL (clear query/
  fragment, trim pathname) instead of rebuilding from origin, and the
  candidate builder swaps only the path — file: URLs (origin "null") no
  longer make the builder throw, and basic-auth credentials are not
  silently dropped before probing.
- The canonical docs and the repair service JSDoc now describe the actual
  loop guard: at most one probe per SOURCE CONFIGURATION (endpoint, mode,
  MAC, identity) per playlist per session, not once per playlist.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): HTTP 401/403 failures trigger the lazy repair

Review round 14 (Codex P1): discovery classifies 401/403 endpoints as
auth-required, but the repair trigger accepted only 404 — a legacy
playlist misclassified token-free against an HTTP-auth-gated middleware
could never reach discovery and stayed unusable. 401/403 now qualify;
endpoint-specific 5xx still do not.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): re-enter repair for edited configurations after a pending probe

Review round 15 (Codex P2): a request carrying an edited configuration
that raced an in-flight probe only awaited it and inherited its outcome —
the edited fingerprint stayed unattempted and the first request failed
without triggering its own discovery. repairPortal now re-enters after
awaiting the pending probe, so the per-config latch decides: already
attempted -> reapply, never attempted -> own probe.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): probe history remembers outcomes so restored configs repair again

Review round 16 (Greptile P1): the per-config latch kept A's fingerprint
after an edit to B dropped A's override, so restoring A left it latched
with nothing to reapply — broken until restart. The history now stores
each probe's OUTCOME (override or null): a restored configuration
reinstalls its remembered repair without a second discovery, and the
anti-ping-pong property (A<->B alternation never re-runs discovery from
stale snapshots) is preserved.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(playlist): serialize deletion behind the per-playlist write queue

Review round 17 (Codex P2): deletePlaylist bypassed
serializePlaylistWrite, so a queued mutation (e.g. the Stalker portal
repair's conditional transform) finishing after an unserialized delete
could upsert the row back and resurrect the playlist. Deletion now runs
through the same queue: queued writes commit first, the delete lands
last, and a transform enqueued after the delete reads a missing row and
aborts. Regression test pins the write-then-delete ordering.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): reinstalled repairs re-sync the watchdog like fresh ones

Review round 18 (Greptile P1): the restored-configuration branch
reinstalled the remembered override without the watchdog refresh the
fresh-repair path performs — if the intermediate edit stopped the
keepalive, the restored full-portal session recovered requests but never
its pings. The reinstall now calls refreshActiveWatchdogPlaylist with the
override applied, symmetric with a fresh repair.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): discarded probes retry once their configuration is restored

Review round 19 (Greptile P1): the pre-probe history reservation survived
the row-mismatch discard, so restoring the original configuration hit the
latch with nothing to reinstall — lazy repair stayed disabled for the
session. Probe records are now explicit (override / no-change /
discarded): a discarded configuration probes again once one cheap row
read confirms the row was RESTORED to it, while stale snapshots of it
stay declined without a discovery run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): IPC-safe transport errors, repairable profile path, nested base paths

Review round 20 (Codex P2 x4):

- The Electron handler throws a real Error for axios failures without a
  response: Electron serializes rejections via toString(), so a plain
  object arrived as "[object Object]" and discovery could not tell a
  timeout (keep probing) from a dead host (stop).
- isAuthorizationError parses HTTP 401/403 out of the IPC-wrapped message,
  so an expired-token 403 retires the token and re-authenticates instead
  of surfacing as a plain failure.
- The account-info full-profile path (which bypasses
  executeStalkerRequest) routes repair-trigger failures through
  StalkerPortalRepairService and retries with the repaired playlist, so
  opening the dialog can fix a stale endpoint.
- resolveStalkerPlaybackUrl derives the installation base from the
  endpoint's API suffix instead of a fixed stalker_portal|c|portal
  allowlist: relative create_link replies now resolve correctly under
  arbitrary discovered installations such as /cp/server/load.php.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): strict probe data shape, mode-aware profile retry, docs API name

Review round 21 (Codex P1 docs + P2 x2):

- Probe classification requires the real get_genres shape (array, or a
  {data: []} envelope without an error) instead of a bare `js` key: a 200
  error envelope ({js:{error:"Unknown action"}}, {js:false}) no longer
  ends discovery on a broken candidate and persists an empty catalog.
- After a repair that flips the portal to simple mode, the account-info
  retry re-enters the mode routing and uses get_main_info instead of
  handshaking against a token-free panel again.
- docs/architecture/stalker-portal.md names transformPlaylistMeta and its
  atomic source-check invariant (plus the serialized deletion) rather than
  the race-prone updatePlaylistMeta.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): account dialog re-routes after a simple-to-full repair

Review round 22 (Codex P2): fetchViaMainInfo runs through
executeStalkerRequest, whose lazy repair retries the SAME action, so a
repair proving the portal is actually full left the dialog calling
get_main_info — canonical installations publish subscription details only
through handshake + get_profile, leaving the dialog empty. The routing is
now symmetric with the full-to-simple case: an empty main-info result
whose repair flipped the mode re-enters the profile flow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): row-gate override reinstall; document mode-based account routing

Review round 23 (Codex P2 + P1 docs):

- Reinstalling a remembered override now requires the persisted row to
  actually carry that configuration again. A stale request for A while the
  row holds an unrelated C no longer resurrects A's override, which would
  retry against B and repoint the active watchdog away from C. (The
  edit-back-to-A case stays as documented: there the row IS A.)
- docs/architecture/stalker-portal.md and CLAUDE.md describe account-info
  routing by the observed portal MODE instead of the endpoint shape — a
  token-enforcing portal.php is a full portal now — and note the
  mode-change re-routing in both directions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): share the auth-failure predicate; prefer profile over partial main-info

Review round 24 (Codex P1 + P2):

- isAuthorizationError now reuses isStalkerAuthFailureResponse, so the
  phrases discovery and the lazy repair already classify as auth failures
  (Access denied., Unauthorized request., and their JSON envelopes) also
  retire the session token. Previously a full portal expiring with either
  phrase kept its dead token: the repair rediscovered the same
  endpoint/mode, recorded no-change, and every later request stayed broken.
- After a simple-to-full repair, even a PARTIAL get_main_info answer no
  longer wins over the profile flow — expiry and tariff live only behind
  handshake + get_profile. The partial facts are kept only if the profile
  path itself publishes nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): keep a literal c installation directory in candidate derivation

Review round 25 (Codex P2): the /c landing-page rewrite ran after the
endpoint file was stripped, so `/tenant/c/portal.php` collapsed to
`/tenant` and the sibling probes went one level too high, rejecting a
valid portal whose installation directory is literally named `c`. The
rewrite now applies only when the pathname itself ends in `/c` (no
endpoint file); pasted endpoints strip only the file part.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): route rejected post-repair main-info retries to the profile flow

Review round 26 (Codex P2): a simple-to-full repair during
fetchViaMainInfo makes executeStalkerRequest retry the same action against
the repaired full portal, and installations that do not implement
get_main_info answer 404 — the rejection escaped before the repaired-mode
check, so the dialog failed instead of switching to get_profile. The
rejection is captured and reaches the same check; without a mode change it
is rethrown unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): full predicate for wrapped denials; record the removed store prop

Review round 27 (Codex P2 + P1 docs):

- The repair trigger applies the shared auth-failure predicate to the error
  MESSAGE too, so authentication's wrapped structured denials
  (Error('Profile error: Access denied.')) reach the repair instead of
  bypassing it and leaving a healthy sibling endpoint unprobed.
- docs/architecture/stalker-store-api-baseline.md records makeStalkerRequest
  as removed, with the reason it gets no facade alias: it was
  production-dead and held a fourth private copy of the portal-mode branch
  that the shared predicate exists to prevent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): complete auth predicate for wrapped error messages

Review round 28 (Codex P2): the plain-text BODY matcher deliberately knows
only the three middleware phrases, so passing an error message through it
let authenticate()'s wrapped denials — Error('Profile error: Invalid
token') / 'Auth failed' — bypass both the repair trigger and the session
auth predicate. A dedicated isStalkerAuthFailureMessage() applies the wide
phrase set to controlled error strings, while arbitrary portal bodies keep
the narrow matcher that cannot false-positive on HTML pages.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(stalker): reject denied profiles during confirmation; document all repair triggers

Review round 29 (Codex P2 + P1 docs):

- Full-portal confirmation validates the get_profile envelope with the
  shared structured predicate: a handshake can hand out a token whose
  profile still answers {js:{error:"Invalid token"}}, and authenticate()
  inspects only msg/block_msg — discovery would have persisted an unusable
  endpoint and stopped before the healthy sibling. authenticate() now
  returns the raw profile response for that check.
- The canonical lazy-repair contract lists the complete trigger set: the
  plain-text bodies AND their JSON envelopes, HTTP 404, HTTP 401/403, and
  terminal handshake/profile errors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 18:01:45 +02:00
..