fix(portals): time out and fast-fail PWA proxy requests to dead hosts (#1424)

* refactor(portals): hoist the connectivity guard into libs/shared/host-health

The breaker was written dependency-free so both processes that talk to
portals could share it. Move the part that has no Electron in it — the
state machine, the failure classification, the redirect-attribution
helpers and the fast-fail error — into `@iptvnator/shared/host-health`
(`scope:shared` / `domain:shared-runtime` / `type:util`).

What stays in `apps/electron-backend` is the genuinely main-process part:
one guard for the whole process, so both portal IPC handlers see each
other's evidence, and the console warning that announces it. Every call
site is unchanged; the wrapper re-exports the two types they import.

The spec splits the same way — the state machine moves with the class,
the singleton and its redirect attribution stay with the wrapper.

Register the new project in the coverage policy, which every project with
a test target must declare a tier for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(portals): time out and fast-fail PWA proxy requests to dead hosts

The web backend's proxy routes were bare `axios.get()` calls with no
`timeout`, so a provider that accepted a connection and then went silent
held the request until the OS gave up on the TCP connection — minutes,
rather than the 15/30 s budget the Electron handlers use. Add the same
per-route timeouts (Xtream 30 s, Stalker 15 s / 30 s for `create_link`,
playlist and XMLTV 30 s).

Those numbers are safe for large downloads: on axios' default transport
`timeout` bounds the time to response headers and then continues as the
socket's inactivity timeout, so a multi-megabyte XMLTV file that keeps
delivering bytes is never cut off — only a stalled one is.

With requests bounded, run `/xtream` and `/stalker` through the shared
breaker, injected via `WebBackendAppOptions.hostGuard` so specs drive it
with a clock they own. Playlist and XMLTV downloads keep the timeout but
no breaker, matching Electron: a download is one request rather than a
catalog fan-out, it is usually the direct result of the user asking for
it, and it can outlive the half-open trial window.

The breaker is checked before the Xtream URL revalidation, which resolves
the hostname — a dead host is where DNS is slow too, and a request
admitted and then abandoned by the URL policy hands its token back rather
than holding the trial slot.

`resetHostConnectivityGuard()` no longer no-ops in the PWA: the breaker
lives in the backend process, so it travels to a new
`POST /connectivity-guard/reset`, which reads only the origin and never
logs the credential-bearing URL. `skipConnectionGuard` now survives the
PWA transport too, so Stalker endpoint discovery keeps the exemption it
has on the desktop instead of tripping the breaker with its own probes.

A fast-fail keeps each route's HTTP 200 `{message, status}` envelope. The
Stalker path needs one extra step: `forwardStalkerRequest` turns that
envelope into `HTTP Error <code>: …` with a numeric `status`, and the
renderer reads both as "the endpoint answered" — which would make
discovery walk every candidate and fire lazy repair at a host just
declared dead. A prior branch keyed on the shared
`isHostConnectivityFastFailMessage` rethrows it bare instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(portals): report exempt Stalker probe responses to the PWA breaker

Flagged by the author of #1421 as one of the twelve fixes that landed
there after this branch cherry-picked the pre-review commit: an exempt
discovery probe must still REPORT, it just must not COUNT.

The web backend was skipping the report entirely for a probe, which loses
the case that matters. A failure carrying an HTTP response proves the
endpoint answered, and this route sets no `validateStatus`, so axios
rejects every non-2xx with `error.response` attached — a probe answered
with 404 or 500 was therefore dropped instead of clearing the record.
Two counted failures either side of it then read as consecutive and
opened the breaker in the middle of discovery, which is exactly what the
exemption exists to prevent.

`reportProviderRequestFailure` now takes `countFailures`, matching the
Electron reporter, and both routes always report.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(portals): read the redirect hop from the transport that followed it

`failedAfterRedirect` decided whether a failure belonged to a redirect
destination by reading `error.config.url`. That is right for the Electron
transport, which sets `maxRedirects: 0` and reissues every hop as its own
request, so the hop IS the config URL. It is blind on the web backend,
which uses axios' default transport: follow-redirects walks the chain
inside one request and `config` is built once, so `config.url` stays the
URL we asked for.

Verified against the installed axios 1.19.0 with a live server that 302s
to a dead port:

    asked for          : http://127.0.0.1:63953/player_api.php
    config.url         : http://127.0.0.1:63953/player_api.php
    request._currentUrl: http://127.0.0.1:1/dead

So the comparison was original-vs-original, found no redirect, and
charged two dead destinations to the provider that had answered both
times with a 302 — then fast-failed it. Read `request._currentUrl` first
and fall back to `config.url`, which covers both transports; Electron's
native per-hop requests expose no `_currentUrl` and are unaffected.

Also check redirect attribution BEFORE suppressing failure counting for
an exempt probe. A 3xx from the guarded endpoint is an answer, so a probe
that observed one must clear the record; otherwise a timeout, a probe
redirected to a dead destination, and another timeout still read as two
consecutive failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(portals): stop axios query params reading as a redirect

Codex found that the Xtream breaker never opened at all, and it was
right. The web backend passes credentials and the action through axios'
`params`, so axios sends `…/player_api.php?username=…&action=…` while the
baseline handed to `failedAfterRedirect` is the query-less URL the route
built. Verified against axios 1.19.0 with a plain ECONNREFUSED and no
redirect anywhere in sight:

    baseline           : http://127.0.0.1:1/player_api.php
    request._currentUrl: http://127.0.0.1:1/player_api.php?username=demo&…

The two normalized URLs differ, so every ordinary failure looked like a
post-redirect failure, credited the endpoint, and the breaker could never
trip.

Compare origin and path, not the whole URL. That keeps what the check is
for — an endpoint that answered and sent us elsewhere, including the
same-origin `/player_api.php` → `/slow/player_api.php` case — and gives
up only a redirect that changes nothing but the query, which is then
counted as an ordinary failure. Erring towards counting is the safe
direction here.

The reason 57 tests passed over a dead feature is the real lesson:
`StubHttpClient` threw bare `Error`s, so the guard's redirect check saw
neither `config.url` nor `request._currentUrl` and quietly did nothing.
The stub now shapes its rejections like axios does, including the query
axios appends. With that alone, four existing tests fail against the old
comparison.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(portals): count a hostname that stops resolving, and fix a stale docblock

Two from review, one behavioural and one documentation.

A name that will not resolve is the host failing to answer — the same
evidence as the ENOTFOUND the transport would have raised a moment later.
But the SSRF validation turns a lookup failure into a 400 "host could not
be resolved", and the release path added earlier handed the token back as
inconclusive, so the breaker could never open for a host whose DNS died
and every request kept paying for the same dead lookup.

`ProviderUrlError` now carries the underlying lookup error internally.
A refusal that has one is counted; a genuine policy refusal — private
address, bad scheme, credentials in the URL — still only releases the
half-open slot, because that says nothing about reachability. The field
is internal: `providerUrlErrorBody()` strips it at both call sites, so
the client sees exactly the body it saw before, which the test asserts.

The docblock on `resetHostConnectivityGuard` still said the PWA channel
is unknown and the call no-ops. That stopped being true when this branch
implemented `CONNECTIVITY_GUARD_RESET` over HTTP, and a stale contract
there is how the next caller silently skips the PWA path. (The edit was
in an earlier commit and was lost when the branch was rebuilt on master.)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(portals): record the transport-specific redirect contract

The redirect section still described one transport: hop-by-hop requests,
`error.config.url`, whole-URL comparison. Two of those three are now
wrong for the web backend, and this document is the canonical contract —
leaving it stale is how the attribution bugs fixed in the last two
commits get reintroduced.

Says what is actually true: which field holds the failed hop on each
transport and why the helper reads both, and that the comparison is
origin + path because the web backend's credentials ride in axios'
`params` and a whole-URL comparison therefore reported a redirect for
every ordinary failure.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(portals): scope the guard summary to both processes

The opening line still defined the breaker as an Electron main-process
concern, which contradicted the ownership section below it and is the
part a reader skims to decide whether the document applies to them.

Names both processes, and records that the web backend had the worse
version of the problem first — no request timeout at all — since that is
why the timeouts and the breaker had to land there together.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(portals): declare the shared-interfaces dependency of host-health

The new library's manifest listed only `tslib`, but its emitted JavaScript
does `require('@iptvnator/shared/interfaces')` — the guard builds its
fast-fail message with `buildHostConnectivityFastFailMessage`. Anything
resolving the built artifact from its own manifest would have failed with
MODULE_NOT_FOUND.

The manifest was copied from `shared/logging`, which imports nothing
across libraries and therefore needs nothing beyond `tslib`.
`shared/m3u-utils` is the right precedent: it imports the same library
and declares `"@iptvnator/shared/interfaces": "0.0.1"`.

Verified against the build output rather than by inspection — the emitted
`host-connectivity-guard.js` requires the module, and the generated
`dist/libs/shared/host-health/package.json` now declares it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(portals): bound the PWA connectivity-guard reset

The reset was a bare `fetch` with no timeout, which is the exact failure
this change exists to remove, reintroduced one layer up. Every caller
awaits the reset BEFORE issuing the request it is clearing the way for —
`retryContentInitialization` awaits it first by design — so a backend or
reverse proxy that accepts the POST and then goes quiet would leave
Retry doing nothing at all, for as long as the socket stayed open.

Bound it with an AbortController and a 5 s timer. The abort rejects,
`resetHostConnectivityGuard` swallows it as it already does for any
other failure, and the caller proceeds to its real request — which is
what "best effort" was supposed to mean. The timer is cleared in a
`finally`, and it covers the body read as well as the headers.

Five seconds because this talks to the user's own backend rather than a
provider: it should answer immediately, and a slow one must not hold up
the retry that asked for it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

apps/web/src/app/services/pwa.service.spec.ts

@@ -9,10 +9,16 @@ import { Store } from '@ngrx/store';
 import { TranslateService } from '@ngx-translate/core';
 import { config as rxjsConfig, EMPTY } from 'rxjs';
 import {
+    buildHostConnectivityFastFailMessage,
+    CONNECTIVITY_GUARD_RESET,
     PLAYLIST_PARSE_BY_URL,
     PLAYLIST_UPDATE,
     STALKER_REQUEST,
 } from '@iptvnator/shared/interfaces';
+import {
+    getStalkerRequestErrorStatus,
+    isStalkerProbeTimeout,
+} from '@iptvnator/portal/stalker/data-access';
 import { PwaService } from './pwa.service';
 
 describe('PwaService', () => {
@@ -238,4 +244,191 @@ describe('PwaService', () => {
             delete globalWithFetch.fetch;
         }
     });
+
+    it('surfaces a connectivity-guard fast-fail as a status-less connection failure', async () => {
+        // The backend refuses to contact a host that stopped answering, using
+        // the same HTTP 200 `{ message, status }` envelope as every other
+        // provider failure. It must NOT come out of here as the test above
+        // does: an `HTTP Error <code>` message or a numeric `status` both read
+        // as "the endpoint answered", so endpoint discovery would keep walking
+        // candidates, and a 4xx reading would fire lazy portal repair against
+        // a host just declared unreachable.
+        const message = buildHostConnectivityFastFailMessage(
+            'portal.example:8080'
+        );
+        const originalFetch = globalThis.fetch;
+        globalThis.fetch = jest.fn().mockResolvedValue({
+            ok: true,
+            json: async () => ({ message, status: 502 }),
+        } as unknown as Response) as unknown as typeof fetch;
+
+        const request = service.sendIpcEvent(STALKER_REQUEST, {
+            url: 'http://portal.example:8080/portal.php',
+            macAddress: '00:1A:79:AA:BB:CC',
+            params: { action: 'get_genres' },
+            silent: true,
+        }) as Promise<unknown>;
+        const outcome = request.then(
+            () => {
+                throw new Error('expected rejection');
+            },
+            (error: unknown) => error
+        );
+
+        await Promise.resolve();
+        http.expectOne((candidate) =>
+            candidate.url.endsWith('/provider-targets')
+        ).flush({ targetId: 'target-1' });
+
+        const error = (await outcome) as Error & { status?: number };
+        expect(error.message).toBe(message);
+        expect(error.status).toBeUndefined();
+        expect(getStalkerRequestErrorStatus(error)).toBeUndefined();
+        expect(isStalkerProbeTimeout(error)).toBe(false);
+
+        globalThis.fetch = originalFetch;
+    });
+
+    it('carries the discovery guard bypass through to the /stalker proxy', async () => {
+        // The Electron transport honours `skipConnectionGuard` on the payload.
+        // Dropping it here let two probe timeouts open the backend's breaker
+        // and fast-fail the healthy candidate discovery was looking for.
+        const fetchMock = jest.fn().mockResolvedValue({
+            ok: true,
+            json: () => Promise.resolve({ payload: { js: [] } }),
+        } as unknown as Response);
+        const globalWithFetch = globalThis as { fetch?: typeof fetch };
+        globalWithFetch.fetch = fetchMock as unknown as typeof fetch;
+
+        try {
+            const request = service.forwardStalkerRequest({
+                url: 'http://portal.example/portal.php',
+                macAddress: '00:1A:79:AA:BB:CC',
+                params: { action: 'get_genres' },
+                silent: true,
+                skipConnectionGuard: true,
+            });
+
+            http.expectOne((req) =>
+                req.url.endsWith('/provider-targets')
+            ).flush({ targetId: 'target-1' });
+
+            await expect(request).resolves.toEqual({ js: [] });
+
+            const requestUrl = new URL(String(fetchMock.mock.calls[0][0]));
+            expect(requestUrl.searchParams.get('skipConnectionGuard')).toBe(
+                'true'
+            );
+        } finally {
+            delete globalWithFetch.fetch;
+        }
+    });
+
+    it('omits the discovery guard bypass for ordinary Stalker requests', async () => {
+        const fetchMock = jest.fn().mockResolvedValue({
+            ok: true,
+            json: () => Promise.resolve({ payload: { js: [] } }),
+        } as unknown as Response);
+        const globalWithFetch = globalThis as { fetch?: typeof fetch };
+        globalWithFetch.fetch = fetchMock as unknown as typeof fetch;
+
+        try {
+            const request = service.forwardStalkerRequest({
+                url: 'http://portal.example/portal.php',
+                macAddress: '00:1A:79:AA:BB:CC',
+                params: { action: 'get_categories' },
+            });
+
+            http.expectOne((req) =>
+                req.url.endsWith('/provider-targets')
+            ).flush({ targetId: 'target-1' });
+
+            await expect(request).resolves.toEqual({ js: [] });
+
+            const requestUrl = new URL(String(fetchMock.mock.calls[0][0]));
+            expect(requestUrl.searchParams.has('skipConnectionGuard')).toBe(
+                false
+            );
+        } finally {
+            delete globalWithFetch.fetch;
+        }
+    });
+
+    it('gives up on a reset the backend never answers', async () => {
+        // Every caller awaits the reset BEFORE the request it is clearing the
+        // way for, and `fetch` has no timeout of its own — so a backend that
+        // accepts the POST and goes quiet would leave Retry doing nothing at
+        // all, which is the failure this whole change exists to stop.
+        jest.useFakeTimers();
+        const globalWithFetch = globalThis as { fetch?: typeof fetch };
+        let capturedSignal: AbortSignal | undefined;
+        globalWithFetch.fetch = jest.fn((_url: unknown, init: RequestInit) => {
+            capturedSignal = init.signal ?? undefined;
+            // Never settles on its own; only the abort can end this.
+            return new Promise<Response>((_resolve, reject) => {
+                init.signal?.addEventListener('abort', () =>
+                    reject(new DOMException('Aborted', 'AbortError'))
+                );
+            });
+        }) as unknown as typeof fetch;
+
+        try {
+            const request = service.sendIpcEvent(CONNECTIVITY_GUARD_RESET, {
+                url: 'http://portal.example:8080/portal.php',
+            }) as Promise<unknown>;
+            const outcome = request.then(
+                () => 'resolved',
+                (error: unknown) => (error as Error).name
+            );
+
+            expect(capturedSignal).toBeDefined();
+            expect(capturedSignal?.aborted).toBe(false);
+
+            jest.advanceTimersByTime(5_000);
+
+            await expect(outcome).resolves.toBe('AbortError');
+            expect(capturedSignal?.aborted).toBe(true);
+        } finally {
+            jest.useRealTimers();
+            delete globalWithFetch.fetch;
+        }
+    });
+
+    it('asks the backend to forget a host when the connectivity guard is reset', async () => {
+        const fetchMock = jest.fn().mockResolvedValue({
+            ok: true,
+            json: async () => ({ reset: true }),
+        } as unknown as Response);
+        const globalWithFetch = globalThis as { fetch?: typeof fetch };
+        globalWithFetch.fetch = fetchMock as unknown as typeof fetch;
+
+        try {
+            // The breaker lives in the backend process, not the browser, so
+            // the reset has to travel over HTTP. Before this it was a no-op,
+            // and a PWA user's "retry" never cleared anything.
+            await expect(
+                service.sendIpcEvent(CONNECTIVITY_GUARD_RESET, {
+                    url: 'http://portal.example:8080/portal.php',
+                }) as Promise<{ reset: boolean }>
+            ).resolves.toEqual({ reset: true });
+
+            expect(fetchMock).toHaveBeenCalledTimes(1);
+            const [url, init] = fetchMock.mock.calls[0];
+            expect(String(url)).toContain('/connectivity-guard/reset');
+            expect(init.method).toBe('POST');
+            expect(JSON.parse(init.body)).toEqual({
+                url: 'http://portal.example:8080/portal.php',
+            });
+
+            // Nothing to reset without a URL, and nothing sent either.
+            await expect(
+                service.sendIpcEvent(CONNECTIVITY_GUARD_RESET, {}) as Promise<{
+                    reset: boolean;
+                }>
+            ).resolves.toEqual({ reset: false });
+            expect(fetchMock).toHaveBeenCalledTimes(1);
+        } finally {
+            delete globalWithFetch.fetch;
+        }
+    });
 });

apps/web/src/app/services/pwa.service.ts

@@ -15,7 +15,9 @@ import {
 } from 'rxjs';
 import { DataService } from '@iptvnator/services';
 import {
+    CONNECTIVITY_GUARD_RESET,
     ERROR,
+    isHostConnectivityFastFailMessage,
     Playlist,
     PLAYLIST_PARSE_BY_URL,
     PLAYLIST_UPDATE,
@@ -35,6 +37,16 @@ import {
 } from '@iptvnator/portal/shared/util';
 import { getRuntimeBackendUrl } from './runtime-config';
 
+/**
+ * How long to wait for the backend to forget a host before giving up.
+ *
+ * This talks to the user's own backend, not a provider, so it should answer
+ * immediately; the bound exists so a stuck one cannot hold up the retry that
+ * asked for the reset. Short on purpose — the reset is best effort, and the
+ * caller's own request reports the real state either way.
+ */
+const CONNECTIVITY_GUARD_RESET_TIMEOUT_MS = 5_000;
+
 interface PwaXtreamResponse {
     readonly payload?: unknown;
     readonly status?: number;
@@ -140,13 +152,69 @@ export class PwaService extends DataService {
                     token?: string;
                     serialNumber?: string;
                     silent?: boolean;
+                    skipConnectionGuard?: boolean;
                 }
             ) as T;
         }
 
+        if (type === CONNECTIVITY_GUARD_RESET) {
+            return this.resetConnectivityGuard(
+                payload as { url?: string }
+            ) as T;
+        }
+
         return undefined as T;
     }
 
+    /**
+     * Clears the web backend's per-host failure record, so the next request
+     * contacts the host for real instead of being fast-failed.
+     *
+     * The breaker lives in the backend process, not the browser, so this is an
+     * HTTP call rather than a local reset. Best effort by design — the caller's
+     * own request reports the real state, and `resetHostConnectivityGuard`
+     * swallows whatever this rejects with.
+     */
+    private async resetConnectivityGuard(payload: {
+        url?: string;
+    }): Promise<{ reset: boolean }> {
+        if (!payload?.url) {
+            return { reset: false };
+        }
+
+        // Bounded, because every caller awaits this BEFORE issuing the request
+        // it is clearing the way for. `fetch` has no timeout of its own, so a
+        // backend or reverse proxy that accepts the POST and then goes quiet
+        // would leave Retry doing nothing at all — the failure this whole
+        // change exists to stop, reintroduced one layer up. The abort rejects,
+        // `resetHostConnectivityGuard` swallows it, and the caller proceeds.
+        const controller = new AbortController();
+        const abortTimer = setTimeout(
+            () => controller.abort(),
+            CONNECTIVITY_GUARD_RESET_TIMEOUT_MS
+        );
+
+        try {
+            const response = await fetch(
+                `${this.corsProxyUrl}/connectivity-guard/reset`,
+                {
+                    body: JSON.stringify({ url: payload.url }),
+                    headers: { 'content-type': 'application/json' },
+                    method: 'POST',
+                    signal: controller.signal,
+                }
+            );
+
+            if (!response.ok) {
+                return { reset: false };
+            }
+
+            return (await response.json()) as { reset: boolean };
+        } finally {
+            clearTimeout(abortTimer);
+        }
+    }
+
     refreshPlaylist(payload?: Partial<Playlist & { id: string }>) {
         if (!payload?.url || !payload?.id) {
             return;
@@ -503,6 +571,13 @@ export class PwaService extends DataService {
         serialNumber?: string;
         /** Endpoint-discovery probes expect failures; no error snackbar. */
         silent?: boolean;
+        /**
+         * Endpoint-discovery probes are exempt from the backend's per-host
+         * connectivity guard: they walk several candidates on one host and
+         * expect most to fail, so counting them would declare a working portal
+         * unreachable. Mirrors the Electron `STALKER_REQUEST` payload flag.
+         */
+        skipConnectionGuard?: boolean;
     }) {
         let context = createPortalDebugRequestContext({
             provider: 'stalker',
@@ -530,6 +605,13 @@ export class PwaService extends DataService {
                 ...(payload.serialNumber
                     ? { serialNumber: payload.serialNumber }
                     : {}),
+                // Endpoint-discovery probes are exempt from the backend's
+                // connectivity guard. Dropping this here would let two probes
+                // that time out open the breaker and fast-fail the healthy
+                // candidate discovery is looking for.
+                ...(payload.skipConnectionGuard
+                    ? { skipConnectionGuard: 'true' }
+                    : {}),
             };
             const params = new URLSearchParams(requestParams);
             const requestUrl = `${this.corsProxyUrl}/stalker?${params.toString()}`;
@@ -564,6 +646,23 @@ export class PwaService extends DataService {
             // can classify them — unwrapping `payload` here silently
             // returned `undefined`, making a dead endpoint look like an
             // empty answer and unreachable to the repair.
+            // The connectivity guard refused to contact the host. This one must
+            // NOT go through the branch below: an `HTTP Error <code>` prefix
+            // and a numeric `status` both read as "the endpoint answered", so
+            // endpoint discovery would keep walking candidates instead of
+            // aborting, and a 4xx reading would additionally fire lazy portal
+            // repair against a host just declared unreachable. Thrown bare, the
+            // message lands in the connection-level-failure slot — which is
+            // exactly what a tripped breaker means.
+            if (
+                responseBody &&
+                typeof responseBody === 'object' &&
+                !('payload' in responseBody) &&
+                isHostConnectivityFastFailMessage(responseBody.message)
+            ) {
+                throw new Error(String(responseBody.message));
+            }
+
             if (
                 responseBody &&
                 typeof responseBody === 'object' &&