mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-08 17:06:15 -08:00
fix(web-backend): validate and pin provider redirect hops (#1553)
* fix(web-backend): validate and pin every provider redirect hop * fix(web-backend): separate provider metadata from connection authority
This commit is contained in:
1 parent
9de480826c
commit
5febe28eba
24 files changed
+1694
-331
No files matched your search
@@ -0,0 +1,7 @@
|
||||
---
|
||||
type: fix
|
||||
area: web-backend
|
||||
issues: [1436]
|
||||
---
|
||||
|
||||
The self-hosted backend now checks every provider redirect and pins connections to validated addresses, preventing redirects or DNS changes from bypassing private-network restrictions. Trusted LAN access remains available through the existing opt-in.
|
||||
@@ -207,6 +207,19 @@ invalidation prevent late results from restoring removed programmes. Provider
|
||||
EPG is independent. See `docs/architecture/m3u-playlist-module.md`
|
||||
("XMLTV source lifecycle").
|
||||
|
||||
## Web Backend Provider Redirects
|
||||
|
||||
All four provider proxy routes use `ValidatedHttpClient`: automatic redirects
|
||||
are disabled, the initial URL and at most five redirect hops pass full URL/DNS
|
||||
validation, and fresh agents pin each connection to that hop's validated IPs.
|
||||
Host/SNI and TLS verification remain intact; outbound environment proxies are
|
||||
disabled. Private-network opt-in applies to the chain. Cross-origin redirects
|
||||
strip session headers; original query params are not replayed. One portal
|
||||
admission owns the entire chain and final body, with explicit redirect evidence
|
||||
preventing destination failures from penalizing the initial endpoint. Contracts:
|
||||
`docs/architecture/pwa-self-hosted.md` and
|
||||
`docs/architecture/host-connectivity-guard.md`.
|
||||
|
||||
## Portal Connectivity Preference
|
||||
|
||||
- Half-open trial slots follow the complete request lifetime with no elapsed-time
|
||||
|
||||
@@ -1692,6 +1692,19 @@ invalidation prevent late results from restoring removed programmes. Provider
|
||||
EPG is independent. See `docs/architecture/m3u-playlist-module.md`
|
||||
("XMLTV source lifecycle").
|
||||
|
||||
## Web Backend Provider Redirects
|
||||
|
||||
All four provider proxy routes use `ValidatedHttpClient`: automatic redirects
|
||||
are disabled, the initial URL and at most five redirect hops pass full URL/DNS
|
||||
validation, and fresh agents pin each connection to that hop's validated IPs.
|
||||
Host/SNI and TLS verification remain intact; outbound environment proxies are
|
||||
disabled. Private-network opt-in applies to the chain. Cross-origin redirects
|
||||
strip session headers; original query params are not replayed. One portal
|
||||
admission owns the entire chain and final body, with explicit redirect evidence
|
||||
preventing destination failures from penalizing the initial endpoint. Contracts:
|
||||
`docs/architecture/pwa-self-hosted.md` and
|
||||
`docs/architecture/host-connectivity-guard.md`.
|
||||
|
||||
## Portal Connectivity Preference
|
||||
|
||||
- Half-open trial slots follow the complete request lifetime with no elapsed-time
|
||||
|
||||
@@ -13,9 +13,9 @@
|
||||
* see `@iptvnator/shared/host-health`) fast-fails the rest once a host has
|
||||
* refused to answer twice in a row.
|
||||
*
|
||||
* On the axios timeout semantics: with the default (follow-redirects)
|
||||
* transport, `timeout` is NOT a wall-clock deadline for the whole response. It
|
||||
* bounds the time to response headers, and then continues as the socket's
|
||||
* On the axios timeout semantics: with axios 1.20's native HTTP
|
||||
* transport (`maxRedirects: 0`), each hop's `timeout` is NOT a wall-clock
|
||||
* deadline for the whole response. It bounds the time to response headers, and then continues as the socket's
|
||||
* inactivity timeout for the body. A large XMLTV or M3U download that keeps
|
||||
* delivering bytes is therefore never cut off mid-transfer — only a stalled one
|
||||
* is, which is exactly the case these numbers are meant to catch.
|
||||
@@ -29,6 +29,7 @@ import {
|
||||
portalEndpointKeyOf,
|
||||
} from '@iptvnator/shared/host-health';
|
||||
import { buildHostConnectivityFastFailMessage } from '@iptvnator/shared/interfaces';
|
||||
import { ProviderRequestError } from './provider-request-error';
|
||||
import type { NormalizedProviderError } from './provider-error';
|
||||
|
||||
/**
|
||||
@@ -176,8 +177,8 @@ export function resetProviderHost(
|
||||
* `countFailures: false` is for requests exempt from the guard (endpoint
|
||||
* discovery). Their failures are expected and must not count — but the report
|
||||
* must still happen, because an error that carries an HTTP response proves the
|
||||
* endpoint answered. This route sets no `validateStatus`, so axios rejects every
|
||||
* non-2xx WITH `error.response`; dropping those instead of reporting them is
|
||||
* endpoint answered. The redirect wrapper accepts 3xx, but axios still rejects
|
||||
* other non-2xx WITH `error.response`; dropping those instead of reporting them is
|
||||
* what would let the breaker open in the middle of discovery.
|
||||
*/
|
||||
export function reportProviderRequestFailure(
|
||||
@@ -190,6 +191,14 @@ export function reportProviderRequestFailure(
|
||||
return;
|
||||
}
|
||||
|
||||
const manualChain = error instanceof ProviderRequestError;
|
||||
if (error instanceof ProviderRequestError) {
|
||||
if (error.initialResponded) {
|
||||
guard.reportSuccess(token);
|
||||
return;
|
||||
}
|
||||
error = error.cause;
|
||||
}
|
||||
const countFailures = options.countFailures ?? true;
|
||||
switch (classifyHostRequestFailure(error)) {
|
||||
case 'host-level':
|
||||
@@ -199,7 +208,10 @@ export function reportProviderRequestFailure(
|
||||
// other response — otherwise an ordinary timeout, a probe that was
|
||||
// redirected to a dead destination, and another ordinary timeout
|
||||
// still read as two consecutive failures.
|
||||
if (failedAfterRedirect(error, token, options.requestUrl)) {
|
||||
if (
|
||||
!manualChain &&
|
||||
failedAfterRedirect(error, token, options.requestUrl)
|
||||
) {
|
||||
guard.reportSuccess(token);
|
||||
break;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
import axios from 'axios';
|
||||
import {
|
||||
request as httpRequest,
|
||||
ClientRequest,
|
||||
IncomingMessage,
|
||||
RequestOptions,
|
||||
} from 'node:http';
|
||||
import {
|
||||
request as httpsRequest,
|
||||
Agent as HttpsAgent,
|
||||
AgentOptions,
|
||||
} from 'node:https';
|
||||
import { isIP } from 'node:net';
|
||||
import { checkServerIdentity } from 'node:tls';
|
||||
import type {
|
||||
ProviderTransportOptions,
|
||||
WebBackendHttpClient,
|
||||
WebBackendHttpResponse,
|
||||
} from './validated-http-client';
|
||||
|
||||
/**
|
||||
* Keep HTTP authority metadata separate from socket destination selection.
|
||||
* Only the hop's pinned agent can select an address; axios receives a fixed
|
||||
* logical hostname, never a user-selected connection authority. Host and TLS
|
||||
* identity still describe the validated provider, including IP certificates.
|
||||
*/
|
||||
export class ProviderAxiosTransport implements WebBackendHttpClient {
|
||||
async get<T>(
|
||||
url: string,
|
||||
options: ProviderTransportOptions = {}
|
||||
): Promise<WebBackendHttpResponse<T>> {
|
||||
const target = new URL(axios.getUri({ url, params: options.params }));
|
||||
const secure = target.protocol === 'https:';
|
||||
const hostname = target.hostname.replace(/^\[|\]$/g, '');
|
||||
const port = Number(target.port || (secure ? 443 : 80));
|
||||
const root = secure
|
||||
? 'https://provider.invalid/'
|
||||
: 'http://provider.invalid/';
|
||||
if (!options.httpAgent || !options.httpsAgent) {
|
||||
throw new Error('Provider transport requires pinned agents');
|
||||
}
|
||||
const agent = options.httpsAgent as HttpsAgent & {
|
||||
options: AgentOptions;
|
||||
};
|
||||
agent.options.servername = isIP(hostname) ? '' : hostname;
|
||||
agent.options.checkServerIdentity = (_name, certificate) =>
|
||||
checkServerIdentity(hostname, certificate);
|
||||
const response = await axios.get<T>(root, {
|
||||
...options,
|
||||
params: undefined,
|
||||
transport: {
|
||||
request: (
|
||||
requestOptions: RequestOptions,
|
||||
callback: (response: IncomingMessage) => void
|
||||
) => {
|
||||
// Path is HTTP metadata, never parsed as an authority.
|
||||
// Even //host/path remains a path on the pinned connection.
|
||||
const request = (secure ? httpsRequest : httpRequest)(
|
||||
{
|
||||
...requestOptions,
|
||||
hostname: 'provider.invalid',
|
||||
port,
|
||||
path: target.pathname + target.search,
|
||||
},
|
||||
callback
|
||||
);
|
||||
retainHeaderTimeout(request, options.timeout);
|
||||
return request;
|
||||
},
|
||||
},
|
||||
headers: { ...options.headers, Host: target.host },
|
||||
});
|
||||
return {
|
||||
...response,
|
||||
headers: {
|
||||
location:
|
||||
typeof response.headers['location'] === 'string'
|
||||
? response.headers['location']
|
||||
: undefined,
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/** Axios identifies native transports by identity; a custom one needs this timer. */
|
||||
function retainHeaderTimeout(request: ClientRequest, timeout?: number): void {
|
||||
if (!timeout) return;
|
||||
const timer = setTimeout(
|
||||
() =>
|
||||
request.destroy(
|
||||
Object.assign(
|
||||
new Error('Provider response headers timed out'),
|
||||
{ code: 'ECONNABORTED' }
|
||||
)
|
||||
),
|
||||
timeout
|
||||
);
|
||||
timer.unref();
|
||||
const clear = () => {
|
||||
clearTimeout(timer);
|
||||
request.removeListener('response', clear);
|
||||
request.removeListener('error', clear);
|
||||
request.removeListener('close', clear);
|
||||
};
|
||||
request.once('response', clear);
|
||||
request.once('error', clear);
|
||||
request.once('close', clear);
|
||||
}
|
||||
@@ -9,6 +9,9 @@
|
||||
* routinely holds Xtream credentials.
|
||||
*/
|
||||
|
||||
import { ProviderRequestError } from './provider-request-error';
|
||||
import { providerUrlErrorBody } from './provider-url-policy';
|
||||
|
||||
export interface ProviderError extends Error {
|
||||
readonly code?: unknown;
|
||||
readonly cause?: unknown;
|
||||
@@ -78,6 +81,10 @@ function visitErrorNode(
|
||||
export function normalizeProviderError(
|
||||
error: unknown
|
||||
): NormalizedProviderError {
|
||||
if (error instanceof ProviderRequestError) {
|
||||
if (error.policyError) return providerUrlErrorBody(error.policyError);
|
||||
return normalizeProviderError(error.cause);
|
||||
}
|
||||
const providerError = error as ProviderError | null | undefined;
|
||||
const response = providerError?.response;
|
||||
if (
|
||||
@@ -111,6 +118,10 @@ export function logProviderRequestFailure(options: {
|
||||
}
|
||||
|
||||
function describeProviderFailure(error: unknown): string {
|
||||
if (error instanceof ProviderRequestError)
|
||||
return error.policyError
|
||||
? `URL policy ${error.policyError.status}`
|
||||
: describeProviderFailure(error.cause);
|
||||
const providerError = error as ProviderError | null | undefined;
|
||||
const response = providerError?.response;
|
||||
if (
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
import type { ProviderUrlError } from './provider-url-policy';
|
||||
|
||||
/** Internal chain evidence, separate from the deliberately small public body. */
|
||||
export class ProviderRequestError extends Error {
|
||||
constructor(
|
||||
readonly initialResponded: boolean,
|
||||
readonly cause: unknown,
|
||||
readonly policyError?: ProviderUrlError
|
||||
) {
|
||||
super('Provider request failed');
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
import { Socket } from 'node:net';
|
||||
import { Readable } from 'node:stream';
|
||||
|
||||
/** Redirect bodies are irrelevant, including invalid gzip and endless bodies. */
|
||||
export function discardProviderBody(data: unknown): void {
|
||||
if (data instanceof Readable) data.destroy();
|
||||
}
|
||||
|
||||
export function discardProviderErrorBody(error: unknown): void {
|
||||
const response = (error as { response?: { data?: unknown } } | null)
|
||||
?.response;
|
||||
discardProviderBody(response?.data);
|
||||
}
|
||||
|
||||
/** Match axios's buffered default and arraybuffer modes for the final hop. */
|
||||
export async function readProviderBody<T>(
|
||||
data: T,
|
||||
arraybuffer: boolean,
|
||||
timeout?: number,
|
||||
socket?: Socket
|
||||
): Promise<T> {
|
||||
if (!(data instanceof Readable)) return data;
|
||||
const chunks: Buffer[] = [];
|
||||
// Axios resolves stream responses at headers and then ignores its own
|
||||
// request timeout callback. Keep the native socket's inactivity timer
|
||||
// alive until the body finishes; arriving wire bytes reset it, even if
|
||||
// decompression has not emitted another decoded chunk yet.
|
||||
const onTimeout = () =>
|
||||
data.destroy(
|
||||
Object.assign(new Error('Provider response body timed out'), {
|
||||
code: 'ECONNABORTED',
|
||||
})
|
||||
);
|
||||
if (timeout && socket) socket.setTimeout(timeout, onTimeout);
|
||||
try {
|
||||
for await (const chunk of data) chunks.push(Buffer.from(chunk));
|
||||
} finally {
|
||||
if (timeout && socket) {
|
||||
socket.removeListener('timeout', onTimeout);
|
||||
socket.setTimeout(0);
|
||||
}
|
||||
}
|
||||
const buffer = Buffer.concat(chunks);
|
||||
if (arraybuffer) return buffer as T;
|
||||
const text = buffer.toString('utf8').replace(/^\uFEFF/, '');
|
||||
try {
|
||||
return JSON.parse(text) as T;
|
||||
} catch {
|
||||
return text as T;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
import { validateProviderUrl } from './provider-url-policy';
|
||||
|
||||
const publicV4 = '93.184.216.34';
|
||||
const publicV6 = '2606:4700:4700::1111';
|
||||
const policy = {
|
||||
allowPrivateNetworkTargets: false,
|
||||
resolveHostname: async () => [publicV4],
|
||||
};
|
||||
const blocked = [
|
||||
'0.0.0.0',
|
||||
'10.1.2.3',
|
||||
'100.64.0.1',
|
||||
'127.0.0.1',
|
||||
'169.254.169.254',
|
||||
'172.16.0.1',
|
||||
'192.168.0.1',
|
||||
'192.0.2.1',
|
||||
'198.18.0.1',
|
||||
'198.51.100.1',
|
||||
'203.0.113.1',
|
||||
'224.0.0.1',
|
||||
'255.255.255.255',
|
||||
'::',
|
||||
'::1',
|
||||
'fc00::1',
|
||||
'fd12::1',
|
||||
'fe80::1',
|
||||
'febf::1',
|
||||
'ff02::1',
|
||||
'::ffff:127.0.0.1',
|
||||
'::ffff:7f00:1',
|
||||
'::ffff:a00:1',
|
||||
'0:0:0:0:0:ffff:a9fe:a9fe',
|
||||
'64:ff9b::a00:1',
|
||||
'2001:db8::1',
|
||||
'2001::1',
|
||||
'2002:7f00:1::',
|
||||
'3fff::1',
|
||||
];
|
||||
|
||||
describe('provider URL policy', () => {
|
||||
it.each(blocked)('rejects literal and DNS answer %s', async (address) => {
|
||||
const url = `http://${address.includes(':') ? `[${address}]` : address}/`;
|
||||
await expect(validateProviderUrl(url, policy)).resolves.toMatchObject({
|
||||
status: 400,
|
||||
});
|
||||
await expect(
|
||||
validateProviderUrl('https://provider.example', {
|
||||
...policy,
|
||||
resolveHostname: async () => [publicV4, address],
|
||||
})
|
||||
).resolves.toMatchObject({ status: 400 });
|
||||
});
|
||||
it.each([
|
||||
'ftp://host/',
|
||||
'file:///etc/passwd',
|
||||
'data:text/plain,hello',
|
||||
'http://user:secret@host',
|
||||
'invalid',
|
||||
'http://localhost.',
|
||||
'http://sub.localhost',
|
||||
])('rejects %s', async (url) => {
|
||||
await expect(validateProviderUrl(url, policy)).resolves.toMatchObject({
|
||||
status: 400,
|
||||
});
|
||||
});
|
||||
it.each([
|
||||
[],
|
||||
['bad-address'],
|
||||
[publicV4, 'host.example'],
|
||||
['127.0.0.1%zone'],
|
||||
])(
|
||||
'rejects malformed DNS records %j even with LAN opt-in',
|
||||
async (...addresses) => {
|
||||
// Jest spreads array table rows; collect the records back into a list.
|
||||
for (const allowPrivateNetworkTargets of [false, true]) {
|
||||
await expect(
|
||||
validateProviderUrl('https://provider.example', {
|
||||
allowPrivateNetworkTargets,
|
||||
resolveHostname: async () => addresses as string[],
|
||||
})
|
||||
).resolves.toMatchObject({ status: 400 });
|
||||
}
|
||||
}
|
||||
);
|
||||
it.each([publicV4, publicV6, '::ffff:5db8:d822'])(
|
||||
'accepts public address %s and retains the hostname',
|
||||
async (address) => {
|
||||
const result = await validateProviderUrl(
|
||||
'https://provider.example/path',
|
||||
{
|
||||
...policy,
|
||||
resolveHostname: async () => [address],
|
||||
}
|
||||
);
|
||||
expect(result).toEqual({
|
||||
url: new URL('https://provider.example/path'),
|
||||
addresses: [address],
|
||||
});
|
||||
}
|
||||
);
|
||||
it('resolves and pins trusted LAN hosts while still rejecting schemes and credentials', async () => {
|
||||
const trusted = {
|
||||
allowPrivateNetworkTargets: true,
|
||||
resolveHostname: async () => ['127.0.0.1', '::1'],
|
||||
};
|
||||
await expect(
|
||||
validateProviderUrl('https://lan.example', trusted)
|
||||
).resolves.toMatchObject({ addresses: ['127.0.0.1', '::1'] });
|
||||
await expect(
|
||||
validateProviderUrl('https://user:secret@lan.example', trusted)
|
||||
).resolves.toMatchObject({ status: 400 });
|
||||
await expect(
|
||||
validateProviderUrl('file:///tmp/test', trusted)
|
||||
).resolves.toMatchObject({ status: 400 });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,140 @@
|
||||
import { lookup } from 'node:dns/promises';
|
||||
import { BlockList, isIP } from 'node:net';
|
||||
|
||||
export interface ProviderUrlPolicy {
|
||||
readonly allowPrivateNetworkTargets: boolean;
|
||||
readonly resolveHostname: (hostname: string) => Promise<readonly string[]>;
|
||||
}
|
||||
|
||||
export interface ProviderUrlError {
|
||||
readonly message: string;
|
||||
readonly status: number;
|
||||
readonly lookupError?: unknown;
|
||||
}
|
||||
|
||||
export interface ValidatedProviderTarget {
|
||||
readonly url: URL;
|
||||
readonly addresses: readonly string[];
|
||||
}
|
||||
|
||||
export function providerUrlErrorBody(error: ProviderUrlError): {
|
||||
message: string;
|
||||
status: number;
|
||||
} {
|
||||
return { message: error.message, status: error.status };
|
||||
}
|
||||
|
||||
export async function resolveHostname(
|
||||
hostname: string
|
||||
): Promise<readonly string[]> {
|
||||
const records = await lookup(hostname, { all: true, verbatim: true });
|
||||
return records.map((record) => record.address);
|
||||
}
|
||||
|
||||
export async function validateProviderUrl(
|
||||
rawUrl: string,
|
||||
policy: ProviderUrlPolicy
|
||||
): Promise<ValidatedProviderTarget | ProviderUrlError> {
|
||||
let url: URL;
|
||||
try {
|
||||
url = new URL(rawUrl);
|
||||
} catch {
|
||||
return { message: 'Provider URL is not a valid URL', status: 400 };
|
||||
}
|
||||
if (url.protocol !== 'http:' && url.protocol !== 'https:') {
|
||||
return {
|
||||
message: 'Only http and https provider URLs are supported',
|
||||
status: 400,
|
||||
};
|
||||
}
|
||||
if (url.username || url.password) {
|
||||
return {
|
||||
message: 'Provider URL credentials are not supported',
|
||||
status: 400,
|
||||
};
|
||||
}
|
||||
const hostname = url.hostname.replace(/^\[|\]$/g, '').toLowerCase();
|
||||
const local = hostname.replace(/\.$/, '');
|
||||
if (
|
||||
!policy.allowPrivateNetworkTargets &&
|
||||
(local === 'localhost' ||
|
||||
local.endsWith('.localhost') ||
|
||||
(isIP(hostname) !== 0 && !isPublicAddress(hostname)))
|
||||
) {
|
||||
return privateAddressError();
|
||||
}
|
||||
let addresses: readonly string[];
|
||||
try {
|
||||
addresses = isIP(hostname)
|
||||
? [hostname]
|
||||
: await policy.resolveHostname(hostname);
|
||||
} catch (lookupError) {
|
||||
return {
|
||||
message: 'Provider URL host could not be resolved',
|
||||
status: 400,
|
||||
lookupError,
|
||||
};
|
||||
}
|
||||
// Validate every answer; never let a malformed record trigger a second DNS
|
||||
// lookup in the transport. Even trusted LAN mode requires concrete IPs.
|
||||
if (!addresses.length || addresses.some((address) => !isIP(address))) {
|
||||
return {
|
||||
message: 'Provider URL host could not be resolved',
|
||||
status: 400,
|
||||
};
|
||||
}
|
||||
if (
|
||||
!policy.allowPrivateNetworkTargets &&
|
||||
addresses.some((address) => !isPublicAddress(address))
|
||||
) {
|
||||
return privateAddressError();
|
||||
}
|
||||
return { url, addresses: [...addresses] };
|
||||
}
|
||||
|
||||
function privateAddressError(): ProviderUrlError {
|
||||
return {
|
||||
message: 'Provider URL points to a private or local network address',
|
||||
status: 400,
|
||||
};
|
||||
}
|
||||
|
||||
const blockedV4 = new BlockList();
|
||||
for (const [network, prefix] of [
|
||||
['0.0.0.0', 8],
|
||||
['10.0.0.0', 8],
|
||||
['100.64.0.0', 10],
|
||||
['127.0.0.0', 8],
|
||||
['169.254.0.0', 16],
|
||||
['172.16.0.0', 12],
|
||||
['192.0.0.0', 24],
|
||||
['192.0.2.0', 24],
|
||||
['192.88.99.0', 24],
|
||||
['192.168.0.0', 16],
|
||||
['198.18.0.0', 15],
|
||||
['198.51.100.0', 24],
|
||||
['203.0.113.0', 24],
|
||||
['224.0.0.0', 3],
|
||||
] as const)
|
||||
blockedV4.addSubnet(network, prefix, 'ipv4');
|
||||
|
||||
const globalV6 = new BlockList();
|
||||
globalV6.addSubnet('2000::', 3, 'ipv6');
|
||||
const blockedV6 = new BlockList();
|
||||
for (const [network, prefix] of [
|
||||
['2001::', 23], // Protocol assignments (including Teredo / benchmarking).
|
||||
['2001:db8::', 32],
|
||||
['2002::', 16], // Documentation / 6to4.
|
||||
['3fff::', 20], // Documentation.
|
||||
] as const)
|
||||
blockedV6.addSubnet(network, prefix, 'ipv6');
|
||||
const mappedV4 = new BlockList();
|
||||
mappedV4.addSubnet('::ffff:0:0', 96, 'ipv6');
|
||||
|
||||
function isPublicAddress(address: string): boolean {
|
||||
if (isIP(address) === 4) return !blockedV4.check(address, 'ipv4');
|
||||
// Node BlockList handles IPv4-mapped IPv6 in both dotted and hex forms.
|
||||
if (mappedV4.check(address, 'ipv6'))
|
||||
return !blockedV4.check(address, 'ipv6');
|
||||
return globalV6.check(address, 'ipv6') && !blockedV6.check(address, 'ipv6');
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
-----BEGIN PRIVATE KEY-----
|
||||
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC+U4KN4bcXoh+W
|
||||
TMyRIBLi2jMWmqv8ZB5rcyiVvcqp4Z8aJOOuiSZjMJJxxsVwu6qKLZwm8KThYo4M
|
||||
m5x5tXMOvUKhqxDoXhQcql5dAiydoFXCJD56cQnRpS/BT5PtqtSBG9VVo3v1YF13
|
||||
rKBF6ETcIiHFT7+jlSzPUP3lnyB7QL+yDFfaqX68TGXRkFG3thl3YMF+JhFILe7V
|
||||
s8H2yZAiI3HuIeBUdTHOqnqVFW2vH1Hl4soXOsEQlQF5VyAFbCJ44duDfCe5uSbX
|
||||
8lA+1VNJSRLKaQUdqvqgjF4UKmnk6a7xMAW0WTiC7RFoIUkH1WosprwEygYohhiF
|
||||
XoG6QbKhAgMBAAECggEAA/H6NlOz9mbzbauo3+dAzPgF8BWDtCclJEgOUtBM16mo
|
||||
ISQbnh4UsCCtIHOk2xngxp18a6g4Wr2uwR8mprU2rdsJew1vO8nbc96qNxZY82mD
|
||||
7ZLPwrz+HZzleQXbxKTyY7y+dth9NNBrD5SB/AD9EG0asxrcl5j7hU6h/LUIONXN
|
||||
GmOwEs7N7IsZgQtfItX8Zgo8+PUfQMp2QUawM4BAXwOSW3rpyioNmMQCa0Y/uKAf
|
||||
8CKvNcP2XcbKucZNc9KHpNtaAjq7PDJ+7L/mSP3MGGNlYRjaasPiIemo2C6Pg1Pb
|
||||
pr+yhwt28PFTqf2LLZcuivBtcPRlMmawcO9KFLjRewKBgQDm28ikYl8bqrROEgNQ
|
||||
QL5rY/DzaaFjyEwvj+qWXWzy6cykQpw4vdLIfe3Kn9Evc4dU1o//RYnoqIWi7EJ0
|
||||
Ma6OBp4lMocHajklZdCG+ArQENcPcdRgVWGf6LZtEbPGaTk88mDOEMm50/FfnKGW
|
||||
7FWsmpmcKqZduq2NX2BvzBDmtwKBgQDTDbTGIGelbe1Cl4ex/zBbcFA9CUm/4583
|
||||
1NMpfZNJaDbi9E3rOZK4ryCHAy4B9GQnmVW+A7ChzqMz2y9jKKM0RHlBXzvkroas
|
||||
IQpRip11Vb/6ZXH3rM+7zgRYHi61+L7de2YbtCUfEOxQ+DcE5yNg1WzgRlYh0yfQ
|
||||
SJ9f2XsZZwKBgEsVYn1sbShvbbMSkrdQR15gI+bXDSGJ7JVvhkmfWybqOZ+W9n5R
|
||||
5rNEmclUD1ISjgpeuni44jCkVsp1cuudmPsiVd8dPuN/fdSW96peFA412+xvBjbK
|
||||
rjS3GFYC8uhuIqqa3jdHKITi1NdW9wtCFF9N7PXovTEw3O9k/NV/lmOjAoGAWsDZ
|
||||
DB0hFHS5gloQYozeKWOZTTWyPc5OR76/cmbqL7WdbGgrHUvreHjt3sCSRwrlClYY
|
||||
FZYWnO1zJjhJHzV5QF91WJPv+DzH8jpe6oNVg//0hmKa6CqqRRKosY+A/ITS5gBK
|
||||
/vyuvbYUOBkT54rQnrIHmEUGgpL+2sRvq9Kj6V8CgYEAxJ46AcgAW0SriYSn+oMX
|
||||
KNMuAeFFS87fMEc+L3e/kH8ctWOGF8WAJew1v7TV+hHJvycMWqBbuvq7N1ArFBH4
|
||||
Hxq76pRoo6DMz5PxIVajSxj9+LuNWn9onuX9Ni0w2p9clH5nD1+Arnb2G91kZby8
|
||||
CqB+d4aWPHBq/0GQDp7xDR8=
|
||||
-----END PRIVATE KEY-----
|
||||
@@ -0,0 +1,20 @@
|
||||
-----BEGIN CERTIFICATE-----
|
||||
MIIDNDCCAhygAwIBAgIUL/DZdKkHvo0op+ZSNbwEJ8/q98UwDQYJKoZIhvcNAQEL
|
||||
BQAwGzEZMBcGA1UEAwwQcHJvdmlkZXIuZXhhbXBsZTAeFw0yNjA5MDYwNTU4Mjda
|
||||
Fw0zNjA5MDMwNTU4MjdaMBsxGTAXBgNVBAMMEHByb3ZpZGVyLmV4YW1wbGUwggEi
|
||||
MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC+U4KN4bcXoh+WTMyRIBLi2jMW
|
||||
mqv8ZB5rcyiVvcqp4Z8aJOOuiSZjMJJxxsVwu6qKLZwm8KThYo4Mm5x5tXMOvUKh
|
||||
qxDoXhQcql5dAiydoFXCJD56cQnRpS/BT5PtqtSBG9VVo3v1YF13rKBF6ETcIiHF
|
||||
T7+jlSzPUP3lnyB7QL+yDFfaqX68TGXRkFG3thl3YMF+JhFILe7Vs8H2yZAiI3Hu
|
||||
IeBUdTHOqnqVFW2vH1Hl4soXOsEQlQF5VyAFbCJ44duDfCe5uSbX8lA+1VNJSRLK
|
||||
aQUdqvqgjF4UKmnk6a7xMAW0WTiC7RFoIUkH1WosprwEygYohhiFXoG6QbKhAgMB
|
||||
AAGjcDBuMB0GA1UdDgQWBBSiXrRLEeZImTh1I0IjfqRQoHIUfzAfBgNVHSMEGDAW
|
||||
gBSiXrRLEeZImTh1I0IjfqRQoHIUfzAPBgNVHRMBAf8EBTADAQH/MBsGA1UdEQQU
|
||||
MBKCEHByb3ZpZGVyLmV4YW1wbGUwDQYJKoZIhvcNAQELBQADggEBAFtT545lqFbw
|
||||
3pgHsnFI2kpXOso1WhjJa44cliFt0L2rg9JgZCih27ba1OZmr55Y6X2lG2xmQtqF
|
||||
Gzm00pyC8EW/Yjwe2xcf0Tk2pGOTQ8MJlXhKOvwcxL6c6d/mc7Uvy+B/cwWbn9F9
|
||||
IeOgDiaPGTRWE/UgiVqq1ZARI3ExRN5G0QWejdw0Q3VYWh2L5lM9kKeRomudaX0T
|
||||
YKQFuuuZji9/RQTN9ZRfjVG0imJOMfqi0Nrl4eva4kHP1FWzH7yR5ybjsuG9WiXx
|
||||
6o7Ktfva+sRr//utJjluL8RvrxJ0d7Dd+BIKQQxBP3yL3YokweL1xzS2WdQM66QW
|
||||
N+E360kOO9k=
|
||||
-----END CERTIFICATE-----
|
||||
@@ -0,0 +1,323 @@
|
||||
import { classifyHostRequestFailure } from '@iptvnator/shared/host-health';
|
||||
import { ProviderRequestError } from './provider-request-error';
|
||||
import { ProviderAxiosTransport } from './provider-axios-transport';
|
||||
import express from 'express';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { createServer, Agent as HttpsAgent } from 'node:https';
|
||||
import { Server } from 'node:http';
|
||||
import { AddressInfo } from 'node:net';
|
||||
import { TLSSocket } from 'node:tls';
|
||||
import { gzipSync } from 'node:zlib';
|
||||
import {
|
||||
ValidatedHttpClient,
|
||||
WebBackendHttpClient,
|
||||
} from './validated-http-client';
|
||||
import { withServer } from './web-backend-app.spec-helpers';
|
||||
|
||||
const lan = {
|
||||
allowPrivateNetworkTargets: true,
|
||||
resolveHostname: async () => ['127.0.0.1'],
|
||||
};
|
||||
|
||||
async function withListeningServer<T>(
|
||||
server: Server,
|
||||
hostname: string,
|
||||
run: (port: number) => Promise<T>
|
||||
): Promise<T> {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
server.once('error', reject);
|
||||
server.listen(0, hostname, resolve);
|
||||
});
|
||||
try {
|
||||
return await run((server.address() as AddressInfo).port);
|
||||
} finally {
|
||||
server.closeAllConnections();
|
||||
await new Promise<void>((resolve, reject) =>
|
||||
server.close((error) => (error ? reject(error) : resolve()))
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
describe('real axios validated transport', () => {
|
||||
it('uses exactly the resolved address at connect time, preserves Host, and bypasses ambient proxy settings', async () => {
|
||||
const proxy = express().use((_req, res) =>
|
||||
res.status(500).send('unexpected proxy')
|
||||
);
|
||||
await withServer(proxy, async (proxyUrl) => {
|
||||
const provider = express().use((req, res) =>
|
||||
res.json({ host: req.headers.host })
|
||||
);
|
||||
await withServer(provider, async (providerUrl) => {
|
||||
const { port } = new URL(providerUrl);
|
||||
const resolveHostname = jest
|
||||
.fn()
|
||||
.mockResolvedValueOnce(['127.0.0.1'])
|
||||
.mockResolvedValue(['127.0.0.2']);
|
||||
const previous = process.env['http_proxy'];
|
||||
process.env['http_proxy'] = proxyUrl;
|
||||
try {
|
||||
const response = await new ValidatedHttpClient({
|
||||
...lan,
|
||||
resolveHostname,
|
||||
}).get(`http://provider.example:${port}/`);
|
||||
expect(response.data).toEqual({
|
||||
host: `provider.example:${port}`,
|
||||
});
|
||||
expect(resolveHostname).toHaveBeenCalledTimes(1);
|
||||
} finally {
|
||||
if (previous === undefined)
|
||||
delete process.env['http_proxy'];
|
||||
else process.env['http_proxy'] = previous;
|
||||
}
|
||||
});
|
||||
});
|
||||
});
|
||||
it('uses the new validated DNS answer on each same-host redirect instead of a pooled socket', async () => {
|
||||
const provider = express().use((_req, res) => res.redirect('/next'));
|
||||
await withServer(provider, async (providerUrl) => {
|
||||
const { port } = new URL(providerUrl);
|
||||
const resolveHostname = jest
|
||||
.fn()
|
||||
.mockResolvedValueOnce(['127.0.0.1'])
|
||||
.mockResolvedValueOnce(['127.0.0.2']);
|
||||
await expect(
|
||||
new ValidatedHttpClient({ ...lan, resolveHostname }).get(
|
||||
`http://provider.example:${port}/`,
|
||||
{ timeout: 500 }
|
||||
)
|
||||
).rejects.toMatchObject({
|
||||
initialResponded: true,
|
||||
cause: {
|
||||
code: expect.stringMatching(
|
||||
/^(ECONNREFUSED|ECONNABORTED)$/
|
||||
),
|
||||
},
|
||||
});
|
||||
expect(resolveHostname).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
});
|
||||
it.each(['invalid-gzip', 'unfinished'])(
|
||||
'discards the %s redirect body immediately',
|
||||
async (kind) => {
|
||||
const provider = express();
|
||||
provider.get('/start', (_req, res) => {
|
||||
res.status(302).set('Location', '/final');
|
||||
if (kind === 'invalid-gzip')
|
||||
res.set('Content-Encoding', 'gzip').end('not gzip');
|
||||
else res.write('a body that never ends');
|
||||
});
|
||||
provider.get('/final', (_req, res) => res.json({ ok: true }));
|
||||
await withServer(provider, async (url) => {
|
||||
await expect(
|
||||
new ValidatedHttpClient(lan).get(`${url}/start`, {
|
||||
timeout: 500,
|
||||
})
|
||||
).resolves.toMatchObject({ data: { ok: true } });
|
||||
});
|
||||
}
|
||||
);
|
||||
it('preserves double-slash paths and escaped segments without changing connection authority', async () => {
|
||||
const provider = express().use((req, res) =>
|
||||
res.json({ path: req.url, host: req.headers.host })
|
||||
);
|
||||
await withServer(provider, async (url) => {
|
||||
const target = new URL(url);
|
||||
await expect(
|
||||
new ValidatedHttpClient(lan).get(
|
||||
`${url}//other.example/a%2Fb?token=a%2Bb`
|
||||
)
|
||||
).resolves.toMatchObject({
|
||||
data: {
|
||||
path: '//other.example/a%2Fb?token=a%2Bb',
|
||||
host: target.host,
|
||||
},
|
||||
});
|
||||
});
|
||||
});
|
||||
it('keeps query serialization and sends only Location query on a real redirect', async () => {
|
||||
const requests: string[] = [];
|
||||
const provider = express().use((req, res) => {
|
||||
requests.push(req.url);
|
||||
if (requests.length === 1) res.redirect('/final?ticket=issued');
|
||||
else res.send('done');
|
||||
});
|
||||
await withServer(provider, async (url) => {
|
||||
await new ValidatedHttpClient(lan).get(`${url}/start?fixed=yes`, {
|
||||
params: { password: 'a b+c' },
|
||||
});
|
||||
expect(requests).toEqual([
|
||||
'/start?fixed=yes&password=a+b%2Bc',
|
||||
'/final?ticket=issued',
|
||||
]);
|
||||
});
|
||||
});
|
||||
it('retains final arraybuffer, decompression, BOM text and JSON behavior', async () => {
|
||||
const provider = express();
|
||||
provider.get('/binary', (_req, res) =>
|
||||
res.end(Buffer.from([0, 255, 128]))
|
||||
);
|
||||
provider.get('/compressed', (_req, res) =>
|
||||
res.set('Content-Encoding', 'gzip').end(gzipSync('{"ok":true}'))
|
||||
);
|
||||
provider.get('/text', (_req, res) => res.end('\uFEFFhello'));
|
||||
await withServer(provider, async (url) => {
|
||||
const client = new ValidatedHttpClient(lan);
|
||||
expect(
|
||||
(
|
||||
await client.get(`${url}/binary`, {
|
||||
responseType: 'arraybuffer',
|
||||
})
|
||||
).data
|
||||
).toEqual(Buffer.from([0, 255, 128]));
|
||||
expect((await client.get(`${url}/compressed`)).data).toEqual({
|
||||
ok: true,
|
||||
});
|
||||
expect((await client.get(`${url}/text`)).data).toEqual('hello');
|
||||
});
|
||||
});
|
||||
it('cancels an unfinished final body and closes its connection', async () => {
|
||||
const controller = new AbortController();
|
||||
let started!: () => void;
|
||||
const bodyStarted = new Promise<void>((resolve) => (started = resolve));
|
||||
const provider = express().use((_req, res) => {
|
||||
res.write('unfinished');
|
||||
started();
|
||||
});
|
||||
await withServer(provider, async (url) => {
|
||||
const pending = new ValidatedHttpClient(lan).get(url, {
|
||||
signal: controller.signal,
|
||||
});
|
||||
await bodyStarted;
|
||||
controller.abort();
|
||||
await expect(pending).rejects.toMatchObject({
|
||||
cause: { code: 'ERR_CANCELED' },
|
||||
});
|
||||
});
|
||||
});
|
||||
it.each(['truncated', 'invalid-gzip', 'timeout'])(
|
||||
'retains response evidence for a %s final body',
|
||||
async (kind) => {
|
||||
const provider = express().use((_req, res) => {
|
||||
if (kind === 'invalid-gzip') {
|
||||
res.set('Content-Encoding', 'gzip').end('not gzip');
|
||||
return;
|
||||
}
|
||||
res.set('Content-Length', '100').write('short');
|
||||
if (kind === 'truncated') setTimeout(() => res.destroy(), 10);
|
||||
});
|
||||
await withServer(provider, async (url) => {
|
||||
const error = await new ValidatedHttpClient(lan)
|
||||
.get(url, { timeout: 100 })
|
||||
.catch((error: unknown) => error);
|
||||
expect(error).toBeInstanceOf(ProviderRequestError);
|
||||
const failure = error as ProviderRequestError;
|
||||
expect(failure.cause).toMatchObject({
|
||||
response: { status: 200, statusText: 'OK' },
|
||||
});
|
||||
expect(classifyHostRequestFailure(failure.cause)).toBe(
|
||||
'responded'
|
||||
);
|
||||
});
|
||||
}
|
||||
);
|
||||
it('times out before headers on the custom native transport', async () => {
|
||||
const provider = express().use(() => {
|
||||
/* Deliberately silent synthetic provider. */
|
||||
});
|
||||
await withServer(provider, async (url) => {
|
||||
await expect(
|
||||
new ValidatedHttpClient(lan).get(url, { timeout: 100 })
|
||||
).rejects.toMatchObject({
|
||||
initialResponded: false,
|
||||
cause: { code: 'ECONNABORTED' },
|
||||
});
|
||||
});
|
||||
});
|
||||
it('does not cap a healthy trickling body at the inactivity timeout', async () => {
|
||||
const provider = express().use((_req, res) => {
|
||||
let count = 0;
|
||||
res.write('start');
|
||||
const interval = setInterval(() => {
|
||||
res.write('x');
|
||||
if (++count === 8) {
|
||||
clearInterval(interval);
|
||||
res.end();
|
||||
}
|
||||
}, 25);
|
||||
res.on('close', () => clearInterval(interval));
|
||||
});
|
||||
await withServer(provider, async (url) => {
|
||||
await expect(
|
||||
new ValidatedHttpClient(lan).get(url, { timeout: 100 })
|
||||
).resolves.toMatchObject({ data: 'startxxxxxxxx' });
|
||||
});
|
||||
});
|
||||
it('preserves TLS SNI and hostname verification with a pinned address', async () => {
|
||||
// This key/certificate is exclusively a synthetic test fixture.
|
||||
const cert = readFileSync(`${__dirname}/testing/provider-test.pem`);
|
||||
const key = readFileSync(`${__dirname}/testing/provider-test.key`);
|
||||
let requests = 0;
|
||||
const server = createServer({ key, cert }, (req, res) => {
|
||||
requests++;
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
host: req.headers.host,
|
||||
sni: (req.socket as TLSSocket & { servername: string })
|
||||
.servername,
|
||||
})
|
||||
);
|
||||
});
|
||||
const trustedTransport: WebBackendHttpClient = {
|
||||
get: (url, options) => {
|
||||
const agent = options?.httpsAgent as HttpsAgent & {
|
||||
options: { ca?: Buffer };
|
||||
};
|
||||
// Supply the local test CA, retaining the production lookup,
|
||||
// SNI and rejectUnauthorized defaults on the actual agent.
|
||||
agent.options.ca = cert;
|
||||
return new ProviderAxiosTransport().get(url, options);
|
||||
},
|
||||
};
|
||||
await withListeningServer(server, '127.0.0.1', async (port) => {
|
||||
await expect(
|
||||
new ValidatedHttpClient(lan).get(
|
||||
`https://provider.example:${port}`
|
||||
)
|
||||
).rejects.toMatchObject({
|
||||
cause: { code: 'DEPTH_ZERO_SELF_SIGNED_CERT' },
|
||||
});
|
||||
expect(requests).toBe(0);
|
||||
const client = new ValidatedHttpClient(lan, trustedTransport);
|
||||
await expect(
|
||||
client.get(`https://provider.example:${port}`)
|
||||
).resolves.toMatchObject({
|
||||
data: {
|
||||
host: `provider.example:${port}`,
|
||||
sni: 'provider.example',
|
||||
},
|
||||
});
|
||||
await expect(
|
||||
client.get(`https://wrong.example:${port}`)
|
||||
).rejects.toMatchObject({
|
||||
cause: { code: 'ERR_TLS_CERT_ALTNAME_INVALID' },
|
||||
});
|
||||
expect(requests).toBe(1);
|
||||
});
|
||||
});
|
||||
it('connects to a pinned IPv6 address and an IPv6 literal in trusted LAN mode', async () => {
|
||||
const server = new Server((_req, res) => res.end('ipv6'));
|
||||
await withListeningServer(server, '::1', async (port) => {
|
||||
const client = new ValidatedHttpClient({
|
||||
...lan,
|
||||
resolveHostname: async () => ['::1'],
|
||||
});
|
||||
expect(
|
||||
(await client.get(`http://provider.example:${port}/`)).data
|
||||
).toBe('ipv6');
|
||||
expect((await client.get(`http://[::1]:${port}/`)).data).toBe(
|
||||
'ipv6'
|
||||
);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,198 @@
|
||||
import { Agent } from 'node:http';
|
||||
import { LookupFunction } from 'node:net';
|
||||
import { ValidatedHttpClient } from './validated-http-client';
|
||||
import {
|
||||
resolvePublicHost,
|
||||
StubHttpClient,
|
||||
} from './web-backend-app.spec-helpers';
|
||||
|
||||
const policy = {
|
||||
allowPrivateNetworkTargets: false,
|
||||
resolveHostname: resolvePublicHost,
|
||||
};
|
||||
|
||||
describe('validated HTTP redirect chain', () => {
|
||||
it.each([301, 302, 303, 307, 308])(
|
||||
'follows relative Location for %s, without replaying original params',
|
||||
async (status) => {
|
||||
const transport = new StubHttpClient();
|
||||
transport.queueRedirect('../next?ticket=provider', status);
|
||||
transport.queueResponse('done');
|
||||
const result = await new ValidatedHttpClient(policy, transport).get(
|
||||
'https://provider.example/base/start',
|
||||
{
|
||||
params: { username: 'demo', password: 'secret' },
|
||||
timeout: 1234,
|
||||
}
|
||||
);
|
||||
expect(result.data).toBe('done');
|
||||
expect(transport.requests[0].params).toEqual({
|
||||
username: 'demo',
|
||||
password: 'secret',
|
||||
});
|
||||
expect(transport.requests[1]).toMatchObject({
|
||||
url: 'https://provider.example/next?ticket=provider',
|
||||
timeout: 1234,
|
||||
});
|
||||
expect(transport.requests[1].params).toBeUndefined();
|
||||
}
|
||||
);
|
||||
it('resolves fragment-only Location against the sent query and detects the cycle', async () => {
|
||||
const transport = new StubHttpClient();
|
||||
transport.queueRedirect('#fragment');
|
||||
await expect(
|
||||
new ValidatedHttpClient(policy, transport).get(
|
||||
'https://provider.example/start',
|
||||
{
|
||||
params: { token: 'secret' },
|
||||
}
|
||||
)
|
||||
).rejects.toMatchObject({
|
||||
policyError: { status: 502, message: 'Redirect cycle detected' },
|
||||
});
|
||||
expect(transport.requests).toHaveLength(1);
|
||||
});
|
||||
it.each([
|
||||
['https://provider.example/next', true],
|
||||
['https://provider.example:8443/next', false],
|
||||
['http://provider.example/next', false],
|
||||
['https://other.example/next', false],
|
||||
])(
|
||||
'scopes sensitive headers on redirect to %s',
|
||||
async (location, retained) => {
|
||||
const transport = new StubHttpClient();
|
||||
transport.queueRedirect(location as string);
|
||||
transport.queueResponse('done');
|
||||
const headers = {
|
||||
Authorization: 'Bearer secret',
|
||||
cOoKiE: 'mac=secret',
|
||||
'Proxy-Authorization': 'secret',
|
||||
sN: 'serial',
|
||||
'User-Agent': 'player',
|
||||
};
|
||||
await new ValidatedHttpClient(policy, transport).get(
|
||||
'https://provider.example/start',
|
||||
{ headers }
|
||||
);
|
||||
expect(transport.requests[1].headers).toEqual(
|
||||
retained ? headers : { 'User-Agent': 'player' }
|
||||
);
|
||||
expect(headers.Authorization).toBe('Bearer secret');
|
||||
}
|
||||
);
|
||||
it.each([
|
||||
'http://127.0.0.1/',
|
||||
'https://user:secret@provider.example/',
|
||||
'file:///tmp/test',
|
||||
'http://[invalid',
|
||||
])(
|
||||
'refuses unsafe Location %s without another transport call',
|
||||
async (location) => {
|
||||
const transport = new StubHttpClient();
|
||||
transport.queueRedirect(location);
|
||||
await expect(
|
||||
new ValidatedHttpClient(policy, transport).get(
|
||||
'https://provider.example/start'
|
||||
)
|
||||
).rejects.toMatchObject({ initialResponded: true });
|
||||
expect(transport.requests).toHaveLength(1);
|
||||
}
|
||||
);
|
||||
it('accepts five redirects and rejects a sixth before its destination', async () => {
|
||||
for (const count of [5, 6]) {
|
||||
const transport = new StubHttpClient();
|
||||
for (let i = 0; i < count; i++) transport.queueRedirect(`/hop${i}`);
|
||||
transport.queueResponse('done');
|
||||
const request = new ValidatedHttpClient(policy, transport).get(
|
||||
'https://provider.example/start'
|
||||
);
|
||||
if (count === 5)
|
||||
await expect(request).resolves.toMatchObject({ data: 'done' });
|
||||
else
|
||||
await expect(request).rejects.toMatchObject({
|
||||
policyError: { status: 502, message: 'Too many redirects' },
|
||||
});
|
||||
expect(transport.requests).toHaveLength(6);
|
||||
}
|
||||
});
|
||||
it('rejects a redirect without Location', async () => {
|
||||
const transport = new StubHttpClient();
|
||||
transport.queueRedirect('');
|
||||
await expect(
|
||||
new ValidatedHttpClient(policy, transport).get(
|
||||
'https://provider.example'
|
||||
)
|
||||
).rejects.toMatchObject({ policyError: { status: 502 } });
|
||||
});
|
||||
it('revalidates DNS on same-host hops and blocks a changed answer before connect', async () => {
|
||||
const transport = new StubHttpClient();
|
||||
transport.queueRedirect('/next');
|
||||
const resolveHostname = jest
|
||||
.fn()
|
||||
.mockResolvedValueOnce(['93.184.216.34'])
|
||||
.mockResolvedValueOnce(['127.0.0.1']);
|
||||
await expect(
|
||||
new ValidatedHttpClient(
|
||||
{ ...policy, resolveHostname },
|
||||
transport
|
||||
).get('https://provider.example/start')
|
||||
).rejects.toMatchObject({
|
||||
policyError: { status: 400 },
|
||||
initialResponded: true,
|
||||
});
|
||||
expect(resolveHostname).toHaveBeenCalledTimes(2);
|
||||
expect(transport.requests).toHaveLength(1);
|
||||
});
|
||||
it('pins all validated IPv4/IPv6 records without a second DNS lookup', async () => {
|
||||
const transport = new StubHttpClient();
|
||||
transport.queueResponse('done');
|
||||
const get = jest.spyOn(transport, 'get');
|
||||
const resolveHostname = jest
|
||||
.fn()
|
||||
.mockResolvedValue(['93.184.216.34', '2606:4700:4700::1111']);
|
||||
await new ValidatedHttpClient(
|
||||
{ ...policy, resolveHostname },
|
||||
transport
|
||||
).get('https://provider.example/start');
|
||||
const options = get.mock.calls[0][1];
|
||||
if (!options) throw new Error('Missing transport options');
|
||||
expect(options).toMatchObject({
|
||||
maxRedirects: 0,
|
||||
proxy: false,
|
||||
adapter: 'http',
|
||||
});
|
||||
const agent = options.httpsAgent as Agent & {
|
||||
options: { lookup: LookupFunction; proxyEnv: object };
|
||||
};
|
||||
const lookup = agent.options.lookup as LookupFunction;
|
||||
const callback = jest.fn();
|
||||
lookup('provider.example', { all: true }, callback);
|
||||
expect(callback).toHaveBeenLastCalledWith(null, [
|
||||
{ address: '93.184.216.34', family: 4 },
|
||||
{ address: '2606:4700:4700::1111', family: 6 },
|
||||
]);
|
||||
lookup('provider.example', { family: 6 }, callback);
|
||||
expect(callback).toHaveBeenLastCalledWith(
|
||||
null,
|
||||
'2606:4700:4700::1111',
|
||||
6
|
||||
);
|
||||
expect(resolveHostname).toHaveBeenCalledTimes(1);
|
||||
expect(agent.options).toMatchObject({ proxyEnv: {} });
|
||||
});
|
||||
it('does not send a hop when canceled during its DNS validation', async () => {
|
||||
const controller = new AbortController();
|
||||
const transport = new StubHttpClient();
|
||||
const resolveHostname = async () => {
|
||||
controller.abort();
|
||||
return ['93.184.216.34'];
|
||||
};
|
||||
await expect(
|
||||
new ValidatedHttpClient(
|
||||
{ ...policy, resolveHostname },
|
||||
transport
|
||||
).get('https://provider.example', { signal: controller.signal })
|
||||
).rejects.toMatchObject({ initialResponded: false });
|
||||
expect(transport.requests).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,216 @@
|
||||
import { ProviderAxiosTransport } from './provider-axios-transport';
|
||||
import {
|
||||
discardProviderBody,
|
||||
discardProviderErrorBody,
|
||||
readProviderBody,
|
||||
} from './provider-response';
|
||||
import axios, { AxiosRequestConfig } from 'axios';
|
||||
import { Agent as HttpAgent, ClientRequest } from 'node:http';
|
||||
import { Agent as HttpsAgent } from 'node:https';
|
||||
import { isIP, LookupFunction } from 'node:net';
|
||||
import { ProviderRequestError } from './provider-request-error';
|
||||
import { ProviderUrlPolicy, validateProviderUrl } from './provider-url-policy';
|
||||
|
||||
export interface WebBackendHttpGetOptions {
|
||||
readonly headers?: Record<string, string>;
|
||||
readonly params?: Record<string, string>;
|
||||
readonly responseType?: 'arraybuffer';
|
||||
readonly timeout?: number;
|
||||
readonly signal?: AbortSignal;
|
||||
}
|
||||
|
||||
export type ProviderTransportOptions = Omit<
|
||||
WebBackendHttpGetOptions,
|
||||
'responseType'
|
||||
> &
|
||||
Pick<
|
||||
AxiosRequestConfig,
|
||||
| 'responseType'
|
||||
| 'maxRedirects'
|
||||
| 'validateStatus'
|
||||
| 'httpAgent'
|
||||
| 'httpsAgent'
|
||||
| 'proxy'
|
||||
| 'adapter'
|
||||
>;
|
||||
|
||||
export interface WebBackendHttpResponse<T> {
|
||||
readonly data: T;
|
||||
readonly status: number;
|
||||
readonly statusText?: string;
|
||||
readonly request?: ClientRequest;
|
||||
readonly headers: { readonly location?: string };
|
||||
}
|
||||
|
||||
/** Injected transports must honor the same options and status contract as axios. */
|
||||
export interface WebBackendHttpClient {
|
||||
get<T>(
|
||||
url: string,
|
||||
options?: ProviderTransportOptions
|
||||
): Promise<WebBackendHttpResponse<T>>;
|
||||
}
|
||||
|
||||
const REDIRECT_STATUSES = new Set([301, 302, 303, 307, 308]);
|
||||
const SENSITIVE_HEADERS = new Set([
|
||||
'authorization',
|
||||
'cookie',
|
||||
'proxy-authorization',
|
||||
'sn',
|
||||
]);
|
||||
|
||||
export class ValidatedHttpClient implements WebBackendHttpClient {
|
||||
constructor(
|
||||
private readonly policy: ProviderUrlPolicy,
|
||||
private readonly transport: WebBackendHttpClient = new ProviderAxiosTransport()
|
||||
) {}
|
||||
|
||||
async get<T>(
|
||||
rawUrl: string,
|
||||
options: WebBackendHttpGetOptions = {}
|
||||
): Promise<WebBackendHttpResponse<T>> {
|
||||
let currentUrl = rawUrl;
|
||||
let params = options.params;
|
||||
let headers = options.headers;
|
||||
let initialResponded = false;
|
||||
const visited = new Set<string>();
|
||||
try {
|
||||
for (let redirects = 0; ; redirects++) {
|
||||
options.signal?.throwIfAborted();
|
||||
const target = await validateProviderUrl(
|
||||
currentUrl,
|
||||
this.policy
|
||||
);
|
||||
options.signal?.throwIfAborted();
|
||||
if ('message' in target) {
|
||||
throw new ProviderRequestError(
|
||||
initialResponded,
|
||||
target.lookupError,
|
||||
target
|
||||
);
|
||||
}
|
||||
// Match axios serialization once. Location is resolved against
|
||||
// the URL actually sent; original params are never replayed.
|
||||
const sentUrl = new URL(
|
||||
axios.getUri({ url: target.url.href, params })
|
||||
);
|
||||
sentUrl.hash = '';
|
||||
if (visited.has(sentUrl.href))
|
||||
throw redirectError('Redirect cycle detected');
|
||||
visited.add(sentUrl.href);
|
||||
const lookup = pinnedLookup(target.addresses);
|
||||
// Fresh agents prevent socket-pool reuse across validations.
|
||||
// Empty proxyEnv also disables Node's native env proxy support.
|
||||
const agentOptions = { lookup, proxyEnv: {} };
|
||||
const httpAgent = new HttpAgent(agentOptions);
|
||||
const httpsAgent = new HttpsAgent(agentOptions);
|
||||
let response: WebBackendHttpResponse<T>;
|
||||
try {
|
||||
response = await this.transport.get<T>(target.url.href, {
|
||||
...options,
|
||||
headers,
|
||||
params,
|
||||
adapter: 'http',
|
||||
proxy: false,
|
||||
maxRedirects: 0,
|
||||
responseType: 'stream',
|
||||
httpAgent,
|
||||
httpsAgent,
|
||||
validateStatus: (status) =>
|
||||
REDIRECT_STATUSES.has(status) ||
|
||||
(status >= 200 && status < 300),
|
||||
});
|
||||
if (!REDIRECT_STATUSES.has(response.status)) {
|
||||
try {
|
||||
return {
|
||||
...response,
|
||||
data: await readProviderBody(
|
||||
response.data,
|
||||
options.responseType === 'arraybuffer',
|
||||
options.timeout,
|
||||
response.request?.socket ?? undefined
|
||||
),
|
||||
};
|
||||
} catch (error) {
|
||||
if (axios.isCancel(error)) throw error;
|
||||
// A broken/stalled body still proves the endpoint
|
||||
// answered. Preserve buffered axios error semantics.
|
||||
throw Object.assign(
|
||||
new Error('Provider response body failed'),
|
||||
{
|
||||
cause: error,
|
||||
response: {
|
||||
status: response.status,
|
||||
statusText: response.statusText,
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
initialResponded = true;
|
||||
discardProviderBody(response.data);
|
||||
} catch (error) {
|
||||
discardProviderErrorBody(error);
|
||||
throw error;
|
||||
} finally {
|
||||
httpAgent.destroy();
|
||||
httpsAgent.destroy();
|
||||
}
|
||||
if (redirects >= 5) throw redirectError('Too many redirects');
|
||||
const location = response.headers.location;
|
||||
if (!location)
|
||||
throw redirectError(
|
||||
'Redirect response did not include a location'
|
||||
);
|
||||
let nextUrl: URL;
|
||||
try {
|
||||
nextUrl = new URL(location, sentUrl);
|
||||
} catch {
|
||||
throw redirectError('Redirect location is not a valid URL');
|
||||
}
|
||||
if (nextUrl.origin !== sentUrl.origin) {
|
||||
headers = Object.fromEntries(
|
||||
Object.entries(headers ?? {}).filter(
|
||||
([name]) =>
|
||||
!SENSITIVE_HEADERS.has(name.toLowerCase()) &&
|
||||
name.toLowerCase() !== 'host'
|
||||
)
|
||||
);
|
||||
}
|
||||
params = undefined;
|
||||
currentUrl = nextUrl.href;
|
||||
}
|
||||
} catch (error) {
|
||||
if (error instanceof ProviderRequestError) throw error;
|
||||
throw new ProviderRequestError(initialResponded, error);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function redirectError(message: string): ProviderRequestError {
|
||||
return new ProviderRequestError(true, undefined, { message, status: 502 });
|
||||
}
|
||||
|
||||
function pinnedLookup(addresses: readonly string[]): LookupFunction {
|
||||
const records = addresses.map((address) => ({
|
||||
address,
|
||||
family: isIP(address),
|
||||
}));
|
||||
return (_hostname, options, callback) => {
|
||||
const eligible = options.family
|
||||
? records.filter((record) => record.family === options.family)
|
||||
: records;
|
||||
if (!eligible.length) {
|
||||
callback(
|
||||
Object.assign(
|
||||
new Error('No validated address for the requested family'),
|
||||
{ code: 'ENOTFOUND' }
|
||||
),
|
||||
[]
|
||||
);
|
||||
} else if (options.all) {
|
||||
callback(null, eligible);
|
||||
} else {
|
||||
callback(null, eligible[0].address, eligible[0].family);
|
||||
}
|
||||
};
|
||||
}
|
||||
@@ -418,7 +418,7 @@ describe('web backend host connectivity guard', () => {
|
||||
// not consume the single trial the breaker allows.
|
||||
resolvable = false;
|
||||
const refusedByPolicy = await call();
|
||||
expect(refusedByPolicy.status).toBe(400);
|
||||
expect(refusedByPolicy.status).toBe(200);
|
||||
|
||||
resolvable = true;
|
||||
const trial = await call();
|
||||
@@ -469,7 +469,7 @@ describe('web backend host connectivity guard', () => {
|
||||
);
|
||||
|
||||
const first = await call();
|
||||
expect(first.status).toBe(400);
|
||||
expect(first.status).toBe(200);
|
||||
// The DNS error is internal: the client sees what it always saw.
|
||||
await expect(first.json()).resolves.toEqual({
|
||||
message: 'Provider URL host could not be resolved',
|
||||
@@ -492,21 +492,11 @@ describe('web backend host connectivity guard', () => {
|
||||
});
|
||||
|
||||
it('does not fast-fail a provider whose redirect destination is dead', async () => {
|
||||
// The shape this route actually produces, verified against axios
|
||||
// 1.19.0: follow-redirects walks the chain inside one `get()`, so
|
||||
// `config` still holds the URL we asked for and only
|
||||
// `request._currentUrl` names the hop that failed. Reading `config.url`
|
||||
// alone would compare the original URL with itself, find no redirect,
|
||||
// and charge the dead destination to the provider that answered.
|
||||
const redirectedFailure = () =>
|
||||
Object.assign(new Error('connect ECONNREFUSED'), {
|
||||
code: 'ECONNREFUSED',
|
||||
config: { url: 'http://xtream.example/player_api.php' },
|
||||
request: { _currentUrl: 'http://cdn.dead.example/stream' },
|
||||
});
|
||||
const httpClient = new StubHttpClient();
|
||||
httpClient.queueNetworkError(redirectedFailure());
|
||||
httpClient.queueNetworkError(redirectedFailure());
|
||||
for (let i = 0; i < 2; i++) {
|
||||
httpClient.queueRedirect('http://cdn.dead.example/stream');
|
||||
httpClient.queueNetworkError(hostLevelFailure());
|
||||
}
|
||||
httpClient.queueResponse({ user_info: { username: 'demo' } });
|
||||
const { guard } = createTestGuard();
|
||||
|
||||
@@ -534,7 +524,7 @@ describe('web backend host connectivity guard', () => {
|
||||
action: 'get_account_info',
|
||||
payload: { user_info: { username: 'demo' } },
|
||||
});
|
||||
expect(httpClient.requests).toHaveLength(3);
|
||||
expect(httpClient.requests).toHaveLength(5);
|
||||
}
|
||||
);
|
||||
});
|
||||
@@ -689,31 +679,32 @@ describe('web backend host connectivity guard', () => {
|
||||
const started = new Promise<void>((resolve) => {
|
||||
arrived = resolve;
|
||||
});
|
||||
const transport = jest
|
||||
.spyOn(httpClient, 'get')
|
||||
.mockImplementationOnce(async () => {
|
||||
arrived();
|
||||
await pending;
|
||||
if (outcome !== 'success') {
|
||||
throw Object.assign(
|
||||
hostLevelFailure(
|
||||
outcome === 'cancel'
|
||||
? 'ERR_CANCELED'
|
||||
: 'ETIMEDOUT'
|
||||
),
|
||||
{
|
||||
request: {
|
||||
_currentUrl:
|
||||
outcome ===
|
||||
'redirect-failure'
|
||||
? 'http://cdn.example/slow'
|
||||
: `http://portal.example/${route === 'xtream' ? 'player_api.php' : ''}`,
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
return { data: [] as never };
|
||||
});
|
||||
const transport = jest.spyOn(httpClient, 'get');
|
||||
if (outcome === 'redirect-failure') {
|
||||
transport.mockImplementationOnce(async () => ({
|
||||
data: '' as never,
|
||||
status: 302,
|
||||
headers: {
|
||||
location: 'http://cdn.example/slow',
|
||||
},
|
||||
}));
|
||||
}
|
||||
transport.mockImplementationOnce(async () => {
|
||||
arrived();
|
||||
await pending;
|
||||
if (outcome !== 'success') {
|
||||
throw hostLevelFailure(
|
||||
outcome === 'cancel'
|
||||
? 'ERR_CANCELED'
|
||||
: 'ETIMEDOUT'
|
||||
);
|
||||
}
|
||||
return {
|
||||
data: [] as never,
|
||||
status: 200,
|
||||
headers: {},
|
||||
};
|
||||
});
|
||||
const trial = call();
|
||||
try {
|
||||
await started;
|
||||
@@ -729,7 +720,9 @@ describe('web backend host connectivity guard', () => {
|
||||
).message
|
||||
)
|
||||
).toBe(true);
|
||||
expect(transport).toHaveBeenCalledTimes(1);
|
||||
expect(transport).toHaveBeenCalledTimes(
|
||||
outcome === 'redirect-failure' ? 2 : 1
|
||||
);
|
||||
} finally {
|
||||
settle();
|
||||
await trial;
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
import {
|
||||
HostConnectivityGuard,
|
||||
OPEN_DURATION_MS,
|
||||
} from '@iptvnator/shared/host-health';
|
||||
import { isHostConnectivityFastFailMessage } from '@iptvnator/shared/interfaces';
|
||||
import axios from 'axios';
|
||||
import express from 'express';
|
||||
import { createWebBackendApp, WebBackendHttpClient } from './web-backend-app';
|
||||
import {
|
||||
registerProviderTarget,
|
||||
resolvePublicHost,
|
||||
StubHttpClient,
|
||||
withServer,
|
||||
} from './web-backend-app.spec-helpers';
|
||||
|
||||
describe('provider proxy redirect boundary', () => {
|
||||
it.each(['/xtream', '/stalker', '/parse', '/parse-xml'])(
|
||||
'%s refuses a private redirect before contacting its destination',
|
||||
async (route) => {
|
||||
let destinationRequests = 0;
|
||||
const destination = express().get('/private', (_req, res) => {
|
||||
destinationRequests++;
|
||||
res.send('synthetic private response');
|
||||
});
|
||||
await withServer(destination, async (destinationUrl) => {
|
||||
const provider = express().use((_req, res) => {
|
||||
res.redirect(`${destinationUrl}/private`);
|
||||
});
|
||||
await withServer(provider, async (providerUrl) => {
|
||||
// Only the first public endpoint is mapped to a synthetic
|
||||
// local provider. Axios and its redirect transport are real.
|
||||
const httpClient: WebBackendHttpClient = {
|
||||
get: (url, options) =>
|
||||
axios.get(
|
||||
new URL(url).hostname === 'provider.example'
|
||||
? providerUrl
|
||||
: url,
|
||||
options
|
||||
),
|
||||
};
|
||||
await withServer(
|
||||
createWebBackendApp({
|
||||
httpClient,
|
||||
resolveHostname: resolvePublicHost,
|
||||
allowPrivateNetworkTargets: false,
|
||||
}),
|
||||
async (backend) => {
|
||||
const id = await registerProviderTarget(
|
||||
backend,
|
||||
'http://provider.example'
|
||||
);
|
||||
const response = await fetch(
|
||||
`${backend}${route}?targetId=${id}`
|
||||
);
|
||||
const body = await response.json();
|
||||
expect(destinationRequests).toBe(0);
|
||||
expect(response.status).toBe(
|
||||
route.startsWith('/parse') ? 400 : 200
|
||||
);
|
||||
expect(body).toEqual({
|
||||
status: 400,
|
||||
message:
|
||||
'Provider URL points to a private or local network address',
|
||||
});
|
||||
}
|
||||
);
|
||||
});
|
||||
});
|
||||
}
|
||||
);
|
||||
});
|
||||
|
||||
describe('redirect routes and admission lifecycle', () => {
|
||||
it.each(['/xtream', '/stalker', '/parse', '/parse-xml'])(
|
||||
'%s follows an allowed local redirect and parses the final body',
|
||||
async (route) => {
|
||||
const provider = express().use((req, res) => {
|
||||
if (req.path !== '/final') {
|
||||
res.redirect('/final');
|
||||
return;
|
||||
}
|
||||
if (route === '/parse')
|
||||
res.send(
|
||||
'#EXTM3U\n#EXTINF:-1,Test\nhttps://example.com/test.ts'
|
||||
);
|
||||
else if (route === '/parse-xml')
|
||||
res.type('xml').send(
|
||||
'<?xml version="1.0"?><tv><channel id="test"><display-name>Test</display-name></channel></tv>'
|
||||
);
|
||||
else res.json({ ok: true });
|
||||
});
|
||||
await withServer(provider, async (url) => {
|
||||
await withServer(
|
||||
createWebBackendApp({ allowPrivateNetworkTargets: true }),
|
||||
async (backend) => {
|
||||
const id = await registerProviderTarget(backend, url);
|
||||
const response = await fetch(
|
||||
`${backend}${route}?targetId=${id}&action=test`
|
||||
);
|
||||
const body = await response.json();
|
||||
expect(response.status).toBe(200);
|
||||
if (route === '/parse')
|
||||
expect(body).toMatchObject({ count: 1 });
|
||||
else if (route === '/parse-xml')
|
||||
expect(body).toHaveProperty('channels');
|
||||
else
|
||||
expect(body).toEqual({
|
||||
action: 'test',
|
||||
payload: { ok: true },
|
||||
});
|
||||
}
|
||||
);
|
||||
});
|
||||
}
|
||||
);
|
||||
it.each(['/xtream', '/stalker', '/parse', '/parse-xml'])(
|
||||
'%s rechecks a registered hostname before its initial connection',
|
||||
async (route) => {
|
||||
const transport = new StubHttpClient();
|
||||
const resolveHostname = jest
|
||||
.fn()
|
||||
.mockResolvedValueOnce(['93.184.216.34'])
|
||||
.mockResolvedValue(['127.0.0.1']);
|
||||
await withServer(
|
||||
createWebBackendApp({ httpClient: transport, resolveHostname }),
|
||||
async (backend) => {
|
||||
const id = await registerProviderTarget(
|
||||
backend,
|
||||
'https://provider.example'
|
||||
);
|
||||
const response = await fetch(
|
||||
`${backend}${route}?targetId=${id}`
|
||||
);
|
||||
expect(response.status).toBe(
|
||||
route.startsWith('/parse') ? 400 : 200
|
||||
);
|
||||
expect(await response.json()).toMatchObject({
|
||||
status: 400,
|
||||
});
|
||||
expect(transport.requests).toHaveLength(0);
|
||||
}
|
||||
);
|
||||
}
|
||||
);
|
||||
it.each(['/xtream', '/stalker'])(
|
||||
'%s keeps query-only redirect failures off the initial endpoint record',
|
||||
async (route) => {
|
||||
const transport = new StubHttpClient();
|
||||
const networkFailure = () =>
|
||||
Object.assign(
|
||||
new Error('secret http://user:password@provider.example'),
|
||||
{ code: 'ENOTFOUND' }
|
||||
);
|
||||
transport.queueNetworkError(networkFailure());
|
||||
transport.queueRedirect('?next=1');
|
||||
transport.queueNetworkError(networkFailure());
|
||||
transport.queueNetworkError(networkFailure());
|
||||
transport.queueResponse({ ok: true });
|
||||
const guard = new HostConnectivityGuard();
|
||||
await withServer(
|
||||
createWebBackendApp({
|
||||
httpClient: transport,
|
||||
hostGuard: guard,
|
||||
resolveHostname: resolvePublicHost,
|
||||
}),
|
||||
async (backend) => {
|
||||
const id = await registerProviderTarget(
|
||||
backend,
|
||||
'https://provider.example'
|
||||
);
|
||||
const call = () =>
|
||||
fetch(`${backend}${route}?targetId=${id}`);
|
||||
await call();
|
||||
const redirected = await call();
|
||||
expect(await redirected.text()).not.toContain('secret');
|
||||
await call();
|
||||
expect(await (await call()).json()).toMatchObject({
|
||||
payload: { ok: true },
|
||||
});
|
||||
expect(transport.requests).toHaveLength(5);
|
||||
}
|
||||
);
|
||||
}
|
||||
);
|
||||
it.each(['/xtream', '/stalker'])(
|
||||
'%s releases a half-open trial after redirect DNS refusal and counts initial DNS failures',
|
||||
async (route) => {
|
||||
let now = 1000;
|
||||
const guard = new HostConnectivityGuard({ now: () => now });
|
||||
const transport = new StubHttpClient();
|
||||
let failDns = false;
|
||||
const resolveHostname = async (hostname: string) => {
|
||||
if (failDns || hostname === 'blocked.example')
|
||||
throw Object.assign(new Error('secret transport details'), {
|
||||
code: 'ENOTFOUND',
|
||||
});
|
||||
return ['93.184.216.34'];
|
||||
};
|
||||
await withServer(
|
||||
createWebBackendApp({
|
||||
httpClient: transport,
|
||||
hostGuard: guard,
|
||||
resolveHostname,
|
||||
}),
|
||||
async (backend) => {
|
||||
const id = await registerProviderTarget(
|
||||
backend,
|
||||
'https://provider.example'
|
||||
);
|
||||
const call = () =>
|
||||
fetch(`${backend}${route}?targetId=${id}`);
|
||||
failDns = true;
|
||||
await call();
|
||||
await call();
|
||||
const refused = (await (await call()).json()) as {
|
||||
message: string;
|
||||
};
|
||||
expect(
|
||||
isHostConnectivityFastFailMessage(refused.message)
|
||||
).toBe(true);
|
||||
expect(transport.requests).toHaveLength(0);
|
||||
now += OPEN_DURATION_MS + 1;
|
||||
failDns = false;
|
||||
transport.queueRedirect('https://blocked.example/next');
|
||||
const trial = await call();
|
||||
expect(await trial.json()).toEqual({
|
||||
status: 400,
|
||||
message: 'Provider URL host could not be resolved',
|
||||
});
|
||||
transport.queueResponse({ ok: true });
|
||||
expect(await (await call()).json()).toMatchObject({
|
||||
payload: { ok: true },
|
||||
});
|
||||
}
|
||||
);
|
||||
}
|
||||
);
|
||||
});
|
||||
@@ -9,11 +9,11 @@
|
||||
import { AddressInfo } from 'node:net';
|
||||
import { Server } from 'node:http';
|
||||
import { STALKER_MAG_USER_AGENT } from '@iptvnator/shared/interfaces';
|
||||
import { createWebBackendApp, WebBackendHttpClient } from './web-backend-app';
|
||||
import {
|
||||
createWebBackendApp,
|
||||
WebBackendHttpClient,
|
||||
WebBackendHttpGetOptions,
|
||||
} from './web-backend-app';
|
||||
ProviderTransportOptions,
|
||||
WebBackendHttpResponse,
|
||||
} from './validated-http-client';
|
||||
|
||||
/** The transport-identity headers every portal-facing Stalker request carries. */
|
||||
export const STALKER_IDENTITY_HEADERS = {
|
||||
@@ -41,12 +41,17 @@ export class StubHttpClient implements WebBackendHttpClient {
|
||||
readonly error?: Error;
|
||||
readonly status?: number;
|
||||
readonly statusText?: string;
|
||||
readonly headers?: { location?: string };
|
||||
}> = [];
|
||||
|
||||
queueResponse(data: unknown): void {
|
||||
this.queuedResponses.push({ data });
|
||||
}
|
||||
|
||||
queueRedirect(location: string, status = 302): void {
|
||||
this.queuedResponses.push({ data: '', status, headers: { location } });
|
||||
}
|
||||
|
||||
queueFailure(status: number, statusText = 'Provider failure'): void {
|
||||
this.queuedResponses.push({ data: null, status, statusText });
|
||||
}
|
||||
@@ -61,8 +66,8 @@ export class StubHttpClient implements WebBackendHttpClient {
|
||||
|
||||
async get<T>(
|
||||
url: string,
|
||||
options: WebBackendHttpGetOptions = {}
|
||||
): Promise<{ data: T }> {
|
||||
options: ProviderTransportOptions = {}
|
||||
): Promise<WebBackendHttpResponse<T>> {
|
||||
this.requests.push({
|
||||
headers: options.headers,
|
||||
params: options.params,
|
||||
@@ -90,7 +95,13 @@ export class StubHttpClient implements WebBackendHttpClient {
|
||||
throw error;
|
||||
}
|
||||
|
||||
if (response.status) {
|
||||
if (
|
||||
response.status &&
|
||||
!(
|
||||
options.validateStatus ??
|
||||
((status) => status >= 200 && status < 300)
|
||||
)(response.status)
|
||||
) {
|
||||
const error = new Error(response.statusText) as Error & {
|
||||
response: { status: number; statusText: string };
|
||||
};
|
||||
@@ -101,7 +112,11 @@ export class StubHttpClient implements WebBackendHttpClient {
|
||||
throw error;
|
||||
}
|
||||
|
||||
return { data: response.data as T };
|
||||
return {
|
||||
data: response.data as T,
|
||||
status: response.status ?? 200,
|
||||
headers: response.headers ?? {},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
@@ -147,6 +162,7 @@ export async function withServer<T>(
|
||||
const address = server.address() as AddressInfo;
|
||||
return await callback(`http://127.0.0.1:${address.port}`);
|
||||
} finally {
|
||||
server.closeAllConnections();
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
server.close((error) => (error ? reject(error) : resolve()));
|
||||
});
|
||||
|
||||
@@ -893,7 +893,7 @@ https://stream.example/live.m3u8`);
|
||||
`${baseUrl}/xtream?targetId=${targetId}&action=get_account_info`
|
||||
);
|
||||
|
||||
expect(response.status).toBe(400);
|
||||
expect(response.status).toBe(200);
|
||||
await expect(response.json()).resolves.toEqual({
|
||||
message:
|
||||
'Provider URL points to a private or local network address',
|
||||
|
||||
@@ -1,10 +1,7 @@
|
||||
import cors from 'cors';
|
||||
import express, { Express, Request, Response } from 'express';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { lookup } from 'node:dns/promises';
|
||||
import { isIP } from 'node:net';
|
||||
import zlib from 'node:zlib';
|
||||
import axios from 'axios';
|
||||
import epgParser from 'epg-parser';
|
||||
import parser from 'iptv-playlist-parser';
|
||||
import {
|
||||
@@ -33,19 +30,21 @@ import {
|
||||
ProviderError,
|
||||
} from './provider-error';
|
||||
|
||||
export interface WebBackendHttpGetOptions {
|
||||
readonly headers?: Record<string, string>;
|
||||
readonly params?: Record<string, string>;
|
||||
readonly responseType?: 'arraybuffer';
|
||||
readonly timeout?: number;
|
||||
}
|
||||
|
||||
export interface WebBackendHttpClient {
|
||||
get<T>(
|
||||
url: string,
|
||||
options?: WebBackendHttpGetOptions
|
||||
): Promise<{ data: T }>;
|
||||
}
|
||||
import {
|
||||
ValidatedHttpClient,
|
||||
WebBackendHttpClient,
|
||||
} from './validated-http-client';
|
||||
import { ProviderRequestError } from './provider-request-error';
|
||||
import {
|
||||
ProviderUrlPolicy,
|
||||
providerUrlErrorBody,
|
||||
resolveHostname,
|
||||
validateProviderUrl,
|
||||
} from './provider-url-policy';
|
||||
export type {
|
||||
WebBackendHttpClient,
|
||||
WebBackendHttpGetOptions,
|
||||
} from './validated-http-client';
|
||||
|
||||
interface PlaylistParseError {
|
||||
readonly message: string;
|
||||
@@ -68,42 +67,12 @@ export interface WebBackendAppOptions {
|
||||
readonly runtimeBackendUrl?: string;
|
||||
}
|
||||
|
||||
interface ProviderUrlPolicy {
|
||||
readonly allowPrivateNetworkTargets: boolean;
|
||||
readonly resolveHostname: (hostname: string) => Promise<readonly string[]>;
|
||||
}
|
||||
|
||||
interface ProviderUrlError {
|
||||
readonly message: string;
|
||||
readonly status: number;
|
||||
/**
|
||||
* The DNS failure behind a "could not be resolved" refusal, when that is
|
||||
* what this is. Internal only — {@link providerUrlErrorBody} strips it,
|
||||
* because the client is told the same thing it always was.
|
||||
*
|
||||
* A name that does not resolve is evidence about reachability, exactly like
|
||||
* the `ENOTFOUND` the transport would have raised a moment later. Without
|
||||
* it, a host whose DNS died is refused by the policy on every request and
|
||||
* the breaker never opens, so each call keeps paying for the lookup.
|
||||
*/
|
||||
readonly lookupError?: unknown;
|
||||
}
|
||||
|
||||
/** The client-facing half of a {@link ProviderUrlError}. */
|
||||
function providerUrlErrorBody(error: ProviderUrlError): {
|
||||
message: string;
|
||||
status: number;
|
||||
} {
|
||||
return { message: error.message, status: error.status };
|
||||
}
|
||||
|
||||
type ProviderTargetRegistry = Map<string, URL>;
|
||||
|
||||
export function createWebBackendApp(
|
||||
options: WebBackendAppOptions = {}
|
||||
): Express {
|
||||
const app = express();
|
||||
const httpClient = (options.httpClient ?? axios) as WebBackendHttpClient;
|
||||
const guid = options.guid ?? createGuid;
|
||||
const now = options.now ?? (() => new Date());
|
||||
const hostGuard =
|
||||
@@ -125,6 +94,10 @@ export function createWebBackendApp(
|
||||
isPrivateNetworkProxyAllowed(),
|
||||
resolveHostname: options.resolveHostname ?? resolveHostname,
|
||||
};
|
||||
const httpClient = new ValidatedHttpClient(
|
||||
providerUrlPolicy,
|
||||
options.httpClient
|
||||
);
|
||||
const providerTargets: ProviderTargetRegistry = new Map();
|
||||
|
||||
const corsMiddleware = cors({
|
||||
@@ -179,8 +152,8 @@ export function createWebBackendApp(
|
||||
return;
|
||||
}
|
||||
|
||||
const targetId = createProviderTargetId(result);
|
||||
providerTargets.set(targetId, result);
|
||||
const targetId = createProviderTargetId(result.url);
|
||||
providerTargets.set(targetId, result.url);
|
||||
res.json({ targetId });
|
||||
}
|
||||
);
|
||||
@@ -303,38 +276,10 @@ export function createWebBackendApp(
|
||||
}
|
||||
guardToken = admission.token;
|
||||
|
||||
const providerUrlError =
|
||||
await normalizeAndValidateXtreamProviderUrl(
|
||||
url,
|
||||
providerUrlPolicy
|
||||
);
|
||||
if (providerUrlError) {
|
||||
// Admitted, then abandoned before any request went out. A
|
||||
// policy refusal — private address, bad scheme — says nothing
|
||||
// about reachability, so it only hands the half-open slot back.
|
||||
// A name that would not resolve is different: that IS the host
|
||||
// failing to answer, and counting it is what lets the breaker
|
||||
// stop paying for the same dead lookup on every request.
|
||||
if (providerUrlError.lookupError !== undefined) {
|
||||
reportProviderRequestFailure(
|
||||
hostGuard,
|
||||
guardToken,
|
||||
providerUrlError.lookupError
|
||||
);
|
||||
} else {
|
||||
releaseProviderRequest(hostGuard, guardToken);
|
||||
}
|
||||
guardToken = null;
|
||||
res.status(providerUrlError.status).json(
|
||||
providerUrlErrorBody(providerUrlError)
|
||||
);
|
||||
return;
|
||||
}
|
||||
url.href = normalizeXtreamServerUrl(url.href);
|
||||
|
||||
requestUrl = appendPathSegment(url, 'player_api.php');
|
||||
|
||||
// Provider URLs are validated by /provider-targets before they enter the registry.
|
||||
// codeql[js/request-forgery]
|
||||
const response = await httpClient.get(requestUrl, {
|
||||
params: getProxyParams(req, ['targetId']),
|
||||
timeout: PROVIDER_REQUEST_TIMEOUT_MS.xtream,
|
||||
@@ -433,8 +378,6 @@ export function createWebBackendApp(
|
||||
guardToken = observeProviderRequest(hostGuard, requestUrl);
|
||||
}
|
||||
|
||||
// Provider URLs are validated by /provider-targets before they enter the registry.
|
||||
// codeql[js/request-forgery]
|
||||
const response = await httpClient.get(requestUrl, {
|
||||
headers,
|
||||
// `create_link` gets the longer budget: the portal mints a
|
||||
@@ -492,78 +435,6 @@ function getRegisteredProviderUrl(
|
||||
return targetUrl;
|
||||
}
|
||||
|
||||
async function validateProviderUrl(
|
||||
rawUrl: string,
|
||||
policy: ProviderUrlPolicy
|
||||
): Promise<URL | ProviderUrlError> {
|
||||
let url: URL;
|
||||
try {
|
||||
url = new URL(rawUrl);
|
||||
} catch {
|
||||
return { message: 'Provider URL is not a valid URL', status: 400 };
|
||||
}
|
||||
|
||||
if (url.protocol !== 'http:' && url.protocol !== 'https:') {
|
||||
return {
|
||||
message: 'Only http and https provider URLs are supported',
|
||||
status: 400,
|
||||
};
|
||||
}
|
||||
|
||||
if (url.username || url.password) {
|
||||
return {
|
||||
message: 'Provider URL credentials are not supported',
|
||||
status: 400,
|
||||
};
|
||||
}
|
||||
|
||||
if (policy.allowPrivateNetworkTargets) {
|
||||
return url;
|
||||
}
|
||||
|
||||
const hostname = normalizeHostname(url.hostname);
|
||||
if (isLocalHostname(hostname) || isPrivateOrReservedIp(hostname)) {
|
||||
return {
|
||||
message:
|
||||
'Provider URL points to a private or local network address',
|
||||
status: 400,
|
||||
};
|
||||
}
|
||||
|
||||
if (isIP(hostname) === 0) {
|
||||
let addresses: readonly string[];
|
||||
try {
|
||||
addresses = await policy.resolveHostname(hostname);
|
||||
} catch (lookupError) {
|
||||
return {
|
||||
message: 'Provider URL host could not be resolved',
|
||||
status: 400,
|
||||
lookupError,
|
||||
};
|
||||
}
|
||||
|
||||
if (
|
||||
addresses.length === 0 ||
|
||||
addresses.some((address) =>
|
||||
isPrivateOrReservedIp(normalizeHostname(address))
|
||||
)
|
||||
) {
|
||||
return {
|
||||
message:
|
||||
'Provider URL points to a private or local network address',
|
||||
status: 400,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return url;
|
||||
}
|
||||
|
||||
async function resolveHostname(hostname: string): Promise<readonly string[]> {
|
||||
const records = await lookup(hostname, { all: true, verbatim: true });
|
||||
return records.map((record) => record.address);
|
||||
}
|
||||
|
||||
function createProviderTargetId(url: URL): string {
|
||||
return createHash('sha256').update(url.href).digest('hex');
|
||||
}
|
||||
@@ -636,29 +507,6 @@ function appendPathSegment(url: URL, segment: string): string {
|
||||
return nextUrl.href;
|
||||
}
|
||||
|
||||
async function normalizeAndValidateXtreamProviderUrl(
|
||||
url: URL,
|
||||
policy: ProviderUrlPolicy
|
||||
): Promise<ProviderUrlError | null> {
|
||||
let normalizedUrl: URL;
|
||||
try {
|
||||
normalizedUrl = new URL(normalizeXtreamServerUrl(url.href));
|
||||
} catch {
|
||||
return { message: 'Provider URL is not a valid URL', status: 400 };
|
||||
}
|
||||
|
||||
const validatedUrl = await validateProviderUrl(
|
||||
appendPathSegment(normalizedUrl, 'player_api.php'),
|
||||
policy
|
||||
);
|
||||
if ('message' in validatedUrl) {
|
||||
return validatedUrl;
|
||||
}
|
||||
|
||||
url.href = normalizedUrl.href;
|
||||
return null;
|
||||
}
|
||||
|
||||
async function handlePlaylistParse(options: {
|
||||
readonly guid: () => string;
|
||||
readonly httpClient: WebBackendHttpClient;
|
||||
@@ -667,8 +515,6 @@ async function handlePlaylistParse(options: {
|
||||
readonly userAgent?: string;
|
||||
}): Promise<Record<string, unknown> | PlaylistParseError> {
|
||||
try {
|
||||
// Provider URLs are validated by /provider-targets before playlist parsing.
|
||||
// codeql[js/request-forgery]
|
||||
const response = await options.httpClient.get<string>(options.url, {
|
||||
timeout: PROVIDER_REQUEST_TIMEOUT_MS.playlist,
|
||||
...(options.userAgent
|
||||
@@ -689,14 +535,19 @@ async function handlePlaylistParse(options: {
|
||||
};
|
||||
} catch (error) {
|
||||
logProviderRequestFailure({ error, route: '/parse', url: options.url });
|
||||
const providerError = error as ProviderError;
|
||||
if (error instanceof ProviderRequestError && error.policyError) {
|
||||
return providerUrlErrorBody(error.policyError);
|
||||
}
|
||||
const providerError = (
|
||||
error instanceof ProviderRequestError ? error.cause : error
|
||||
) as ProviderError;
|
||||
if (providerError?.response?.statusText !== undefined) {
|
||||
return {
|
||||
status: providerError.response.status ?? 500,
|
||||
message: providerError.response.statusText,
|
||||
};
|
||||
}
|
||||
const code = collectProviderErrorCodes(error)[0];
|
||||
const code = collectProviderErrorCodes(providerError)[0];
|
||||
return {
|
||||
status: providerError?.response?.status ?? 500,
|
||||
message: code
|
||||
@@ -712,8 +563,6 @@ async function fetchEpgDataFromUrl(
|
||||
url: URL
|
||||
): Promise<unknown> {
|
||||
const href = url.href;
|
||||
// Provider URLs are validated by /provider-targets before XMLTV parsing.
|
||||
// codeql[js/request-forgery]
|
||||
const response = await httpClient.get<ArrayBuffer | string>(href, {
|
||||
timeout: PROVIDER_REQUEST_TIMEOUT_MS.epg,
|
||||
...(url.pathname.endsWith('.gz')
|
||||
@@ -791,67 +640,3 @@ function getLastUrlSegment(value: string): string {
|
||||
function createGuid(): string {
|
||||
return Math.random().toString(36).slice(2);
|
||||
}
|
||||
|
||||
function normalizeHostname(hostname: string): string {
|
||||
return hostname.trim().replace(/^\[/, '').replace(/\]$/, '').toLowerCase();
|
||||
}
|
||||
|
||||
function isLocalHostname(hostname: string): boolean {
|
||||
return hostname === 'localhost' || hostname.endsWith('.localhost');
|
||||
}
|
||||
|
||||
function isPrivateOrReservedIp(address: string): boolean {
|
||||
const version = isIP(address);
|
||||
if (version === 4) {
|
||||
return isPrivateOrReservedIpv4(address);
|
||||
}
|
||||
|
||||
if (version === 6) {
|
||||
return isPrivateOrReservedIpv6(address);
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
function isPrivateOrReservedIpv4(address: string): boolean {
|
||||
const parts = address.split('.').map((part) => Number(part));
|
||||
if (
|
||||
parts.length !== 4 ||
|
||||
parts.some((part) => !Number.isInteger(part) || part < 0 || part > 255)
|
||||
) {
|
||||
return true;
|
||||
}
|
||||
|
||||
const [first, second, third] = parts;
|
||||
return (
|
||||
first === 0 ||
|
||||
first === 10 ||
|
||||
first === 127 ||
|
||||
(first === 100 && second >= 64 && second <= 127) ||
|
||||
(first === 169 && second === 254) ||
|
||||
(first === 172 && second >= 16 && second <= 31) ||
|
||||
(first === 192 && second === 168) ||
|
||||
(first === 192 && second === 0) ||
|
||||
(first === 192 && second === 0 && third === 2) ||
|
||||
(first === 198 && (second === 18 || second === 19)) ||
|
||||
(first === 198 && second === 51 && third === 100) ||
|
||||
(first === 203 && second === 0 && third === 113) ||
|
||||
first >= 224
|
||||
);
|
||||
}
|
||||
|
||||
function isPrivateOrReservedIpv6(address: string): boolean {
|
||||
const normalized = address.toLowerCase();
|
||||
if (
|
||||
normalized === '::' ||
|
||||
normalized === '::1' ||
|
||||
normalized.startsWith('fc') ||
|
||||
normalized.startsWith('fd') ||
|
||||
normalized.startsWith('fe80:')
|
||||
) {
|
||||
return true;
|
||||
}
|
||||
|
||||
const mappedIpv4 = normalized.match(/::ffff:(\d+\.\d+\.\d+\.\d+)$/)?.[1];
|
||||
return mappedIpv4 ? isPrivateOrReservedIpv4(mappedIpv4) : false;
|
||||
}
|
||||
@@ -5,6 +5,7 @@
|
||||
"sourceRoot": "apps/web-e2e/src",
|
||||
"implicitDependencies": [
|
||||
"web",
|
||||
"web-backend",
|
||||
"stalker-mock-server",
|
||||
"xtream-mock-server"
|
||||
],
|
||||
|
||||
@@ -47,11 +47,15 @@ so a silent host hung on OS-level TCP timeouts. Both runtimes now use the same
|
||||
budgets: Xtream 30 s, Stalker 15 s (30 s for `create_link`, which mints a stream
|
||||
URL before answering), playlist and XMLTV downloads 30 s.
|
||||
|
||||
Those numbers are safe for large downloads. On axios' default
|
||||
(follow-redirects) transport `timeout` is **not** a wall-clock deadline for the
|
||||
Those numbers are safe for large downloads. On axios 1.20's native HTTP
|
||||
transport (`maxRedirects: 0`) each hop's `timeout` is **not** a wall-clock deadline for the
|
||||
whole response: it bounds the time to response headers and then continues as the
|
||||
socket's inactivity timeout for the body. A multi-megabyte XMLTV file that keeps
|
||||
delivering bytes is never cut off mid-transfer — only a stalled one is.
|
||||
delivering bytes is never cut off mid-transfer — only a stalled one is. The web
|
||||
backend explicitly restores socket inactivity handling while reading the final
|
||||
stream response, because axios stream mode stops its timeout handler at headers.
|
||||
Truncated, malformed and timed-out final bodies retain their received response
|
||||
status for reachability classification; cancellation retains its own semantics.
|
||||
|
||||
### Scope: portal calls only
|
||||
|
||||
@@ -124,33 +128,27 @@ Merely declining to count it would leave an earlier direct failure standing, and
|
||||
a single later timeout would then fast-fail an endpoint that answered in between.
|
||||
Every caller passes the URL it asked for as the baseline.
|
||||
|
||||
**Where the failed hop is found depends on the transport, and both are in play.**
|
||||
**Web backend records redirect evidence explicitly.** Its `ValidatedHttpClient`
|
||||
disables automatic redirects and holds one admission for the entire chain.
|
||||
`ProviderRequestError.initialResponded` becomes true only after receiving an
|
||||
actual redirect response. A later transport or DNS failure, URL policy refusal,
|
||||
invalid location, cycle or exhausted budget therefore clears the initial
|
||||
endpoint's failure streak when the chain settles. This also handles redirects
|
||||
that change only the query. It does not release the half-open slot early: the
|
||||
route retains ownership through validation, all hops and the final body, and
|
||||
releases in `finally` even when reporting throws. Before any redirect, initial
|
||||
DNS failures carry their internal resolver cause into normal classification;
|
||||
policy refusals remain inconclusive. Error bodies never serialize that cause.
|
||||
|
||||
| | Electron | Web backend |
|
||||
| --- | --- | --- |
|
||||
| Redirects | followed hop by hop (`maxRedirects: 0`), each its own request | followed inside one request by follow-redirects |
|
||||
| Failed hop is in | `error.config.url` | `error.request._currentUrl` |
|
||||
|
||||
`failedRequestUrlOf` reads `_currentUrl` first and falls back to `config.url`,
|
||||
which is correct for both: a per-hop request exposes no `_currentUrl`, and on the
|
||||
following transport `config` is built once and keeps the URL we asked for — so
|
||||
reading `config.url` there would compare a URL with itself, find no redirect, and
|
||||
charge a dead destination to the provider that answered. Anything added here must
|
||||
work on both, because the same helper serves both.
|
||||
|
||||
**The comparison is origin + path, not the whole URL.** A same-origin redirect
|
||||
(`/player_api.php` → `/slow/player_api.php`) proves the endpoint answered just as
|
||||
much as a cross-origin one, and charging it would fast-fail every OTHER call to a
|
||||
portal that answers — so the path has to be part of it. The query must NOT be:
|
||||
the web backend passes Xtream credentials through axios' `params`, so the sent
|
||||
URL always carries a query the baseline does not, and comparing whole URLs made
|
||||
every ordinary failure look like a redirect and stopped the breaker from ever
|
||||
opening. What that gives up is a redirect that changes nothing but the query,
|
||||
which is then counted as an ordinary failure — the safe direction.
|
||||
|
||||
It requires positive evidence — anything unparseable or unknown counts the
|
||||
failure as usual, because guessing "redirect" here would stop the guard from ever
|
||||
tripping — and a failure that names no URL at all is still counted.
|
||||
**Electron retains URL-based attribution.** `failedRequestUrlOf` reads
|
||||
`error.request._currentUrl` first (for a following transport) and falls back to
|
||||
`error.config.url` (for a native per-hop transport). The comparison is origin +
|
||||
path, not the query: axios `params` can append credentials absent from the
|
||||
caller's baseline. Unknown/unparseable URLs conservatively count as ordinary
|
||||
failures, and query-only redirects cannot be distinguished by this fallback.
|
||||
Wrapped web-backend requests use their explicit chain evidence and never infer
|
||||
redirects from the transport's fixed logical hostname. The shared helper and
|
||||
Electron behavior are unchanged by the web-backend fix.
|
||||
|
||||
Known gap: the failing hop is not guarded either (it has no token of its own),
|
||||
so a permanently broken redirect chain keeps costing a full timeout.
|
||||
|
||||
@@ -27,6 +27,12 @@ Do not invent a `test`, `build`, or `e2e` target because a similarly named
|
||||
project has one. Run affected lint/test/build targets that exist and the closest
|
||||
available E2E target for the changed behavior.
|
||||
|
||||
E2E applications must declare runtime dependencies even when they use HTTP
|
||||
instead of TypeScript imports. `web-e2e` includes `web-backend` in
|
||||
`implicitDependencies` so provider-proxy changes invalidate cached self-hosted
|
||||
PWA tests; starting the backend through a `serve` dependency alone does not
|
||||
make its source files inputs to the test hash.
|
||||
|
||||
## Nx Dependency Updates
|
||||
|
||||
Keep `nx` and every official `@nx/*` package on the same exact version. Run
|
||||
|
||||
@@ -139,6 +139,63 @@ before any outbound request:
|
||||
- `IPTVNATOR_PROXY_ALLOW_PRIVATE_NETWORKS=1` explicitly enables trusted
|
||||
local/LAN targets for development, mock servers, or private deployments
|
||||
|
||||
### Provider redirects and connection policy
|
||||
|
||||
`ValidatedHttpClient` applies the registration URL policy again before every
|
||||
outbound request on `/xtream`, `/stalker`, `/parse`, and `/parse-xml`, including
|
||||
the initial request. Axios automatic redirects are disabled (`maxRedirects: 0`);
|
||||
301, 302, 303, 307 and 308 are followed manually, with at most five redirects.
|
||||
Relative `Location` values resolve against the URL actually sent, including its
|
||||
query. Missing/malformed locations, repeated URLs (ignoring fragments), and an
|
||||
exhausted redirect budget fail without dispatching another request. Intermediate
|
||||
bodies are destroyed at headers; malformed or endless redirect bodies do not
|
||||
block following an otherwise valid location. Final bodies retain a native socket
|
||||
inactivity timer: axios stream mode stops its own timeout handling at headers,
|
||||
so `provider-response.ts` keeps that timer active until consumption finishes.
|
||||
A body failure preserves received HTTP status evidence; cancellation remains
|
||||
inconclusive before any redirect.
|
||||
|
||||
Every DNS answer must be a valid permitted IP. Mixed public/private answers
|
||||
fail closed. The strict policy rejects private/reserved IPv4, IPv4-mapped private
|
||||
IPv6, and IPv6 outside global unicast `2000::/3`, plus protocol-assignment,
|
||||
documentation and 6to4 ranges within it. These deliberately conservative ranges
|
||||
follow the [IANA IPv6 address space](https://www.iana.org/assignments/ipv6-address-space/)
|
||||
and [special-purpose registry](https://www.iana.org/assignments/iana-ipv6-special-registry/).
|
||||
The LAN opt-in applies to the whole
|
||||
chain; it still requires HTTP(S), no URL userinfo and concrete DNS addresses.
|
||||
|
||||
Fresh HTTP/HTTPS agents pin socket lookup to the exact validated IPv4/IPv6
|
||||
answer set for that hop. No second DNS query or pooled connection may substitute
|
||||
an unvalidated address. The default axios transport uses a fixed logical hostname (`provider.invalid`)
|
||||
so its connection authority cannot come from user-controlled URL metadata;
|
||||
the pinned lookup alone selects an IP. Its native request-options adapter sets
|
||||
`path` separately, so even `//host/path` cannot replace the connection authority.
|
||||
It also owns the deadline from dispatch through response headers, since axios's
|
||||
built-in connection timer only covers its own native transport objects. The original provider host and port are
|
||||
explicitly preserved in Host, TLS SNI and certificate identity checks (literal
|
||||
IPs omit SNI but still verify the original IP). Axios proxies and Node environment proxies
|
||||
are disabled for these requests so a proxy cannot independently resolve the
|
||||
origin. Deployments that require an outbound HTTP proxy must use a different
|
||||
network arrangement; setting `HTTP_PROXY`/`HTTPS_PROXY` does not route provider
|
||||
requests through it. This guarantee concerns address selection in the Node
|
||||
transport, not routing/NAT performed outside the process. Private-network
|
||||
opt-in intentionally relaxes address restrictions.
|
||||
|
||||
Original axios `params` are applied only to the initial request. Subsequent
|
||||
queries come from `Location` resolution, without appending the original
|
||||
credentials again. Authorization, Cookie, Proxy-Authorization and Stalker SN
|
||||
headers are retained only on the same origin; changes of host, port or scheme
|
||||
(including HTTPS downgrades) strip them for the rest of the chain. User-Agent
|
||||
and other non-secret protocol headers survive. Providers can explicitly put
|
||||
query values in `Location`; the backend does not rewrite provider-issued URLs.
|
||||
|
||||
Policy errors contain fixed messages/statuses without URLs, DNS exceptions or
|
||||
transport objects. Portal routes keep HTTP 200 with a `{ message, status }`
|
||||
error envelope; playlist/XMLTV routes use the actual error status. Registration
|
||||
continues to use real HTTP error statuses. See
|
||||
[host connectivity guard](host-connectivity-guard.md) for chain ownership and
|
||||
failure attribution.
|
||||
|
||||
Do not disable TLS certificate validation in the backend proxy. For private
|
||||
certificate authorities, configure Node with `NODE_EXTRA_CA_CERTS`.
|
||||
|
||||
|
||||
Reference in new issue
Block a user