feat(self-hosted): server storage API in web-backend (#1898)

* feat(interfaces): add the server storage HTTP contract

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): read server storage configuration

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): open the server database and track revisions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): playlist and user-state repositories

Playlist documents use the shared desktop row codec, so the server writes
the same rows. keepPayload upserts take the stored playlist/items/header
fields when the client leaves them out. Xtream user data lives in the
server-only server_user_state table, one JSON value per namespace and
playlist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): storage error mapping and access token guard

Storage errors map to JSON status bodies and are logged by name and code
only. An optional bearer token guards every route except /, /health,
/config.js and /v1/instance, provider proxy routes included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): serve the /v1 server storage API

/v1/instance is always public and reports whether server storage is on.
With storage, /v1/playlists serves whole documents with ETags and 304s,
accepts keepPayload upserts, and deletes playlists with their user state;
/v1/user-state and /v1/app-state store the rest. The proxy routes are
unchanged unless an access token is configured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): open server storage on start and close on shutdown

With IPTVNATOR_STORAGE=server, main.ts opens the database before it
listens and exits with a clear message when the data directory is not
writable. SIGTERM and SIGINT close the server and the database. The
native better-sqlite3 module stays outside the esbuild bundle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): match weakened ETags on playlist reads

nginx weakens the ETag of a gzipped response, and clients send the weak
form back in If-None-Match. The route now compares tags weakly, as RFC 9110
requires for GET, so it answers 304 before reading the document.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(docker): persist server storage on a volume

The image builds better-sqlite3 in a native stage per target platform,
enables server storage on the /data volume, and raises the nginx body
limit to 64 MB for /api with gzip for JSON. The compose file mounts a
named volume and documents the optional access token. The PWA does not
use the storage yet; docs say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(web-backend): type the storage route responses

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): harden server storage reads, deletes and app state

- Metadata reads return the stored document without its payload fields,
  so fields that live only in the JSON survive a metadata round-trip.
- A conditional GET checks that the playlist exists before answering 304.
- Deleting a playlist and its user state commits in one transaction; the
  ETag moves only after the commit.
- Browser app state is stored under a client: prefix and the instance id
  is repaired when empty, so a client write cannot stop the backend.
- User-state buckets have no prototype, so ids like __proto__ round-trip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(self-hosted): keep the access token unset until the PWA sends it

Add the release note for the Docker server storage volume.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): bound shutdown and reject padded playlist ids

- SIGTERM closes idle connections at once and cuts the rest after five
  seconds, so a long provider download cannot keep the backend running.
- Playlist ids with surrounding whitespace are rejected: the row codec
  trims ids, so storage and ETags would track different keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): clear optional columns on whole-document upserts

The row codec maps empty values to undefined, which ON CONFLICT DO UPDATE
skips, so a cleared User-Agent, Referrer or credential kept its old column
value and came back on the next read. Server upserts now write NULL to
every replaceable column the document no longer sets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(self-hosted): linear token parsing, no-store app state, longer start

- The access guard reads the Bearer token by string scanning instead of a
  regular expression that CodeQL flagged as polynomial on long whitespace.
- GET /v1/app-state/:key sends Cache-Control: no-store like the other
  state routes, so devices never reuse a stale value.
- The entrypoint waits up to IPTVNATOR_BACKEND_START_TIMEOUT (300 s) for
  /health, because the backend migrates its database before it listens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(self-hosted): metadata reads skip payloads, redact JSON parse warnings

List playlist metadata from a server-only payload-free copy of each
document instead of running json_remove over every payload blob, and log
only the error name when a stored JSON value fails to parse, since the
parser's message quotes the stored text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): copy carried-over panel clock into playlist metadata

When an upsert keeps the stored serverTimezone, build the metadata copy
from the saved payload so metadata and full reads agree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
4grayandClaude Opus 5.5 authored and GitHub committed 2026-10-12 00:05:35 +02:00
1 parent 4e9795ac54
commit b88a1c81e5
41 files changed
+2312 -16

No files matched your search

@@ -0,0 +1,7 @@
---
type: feature
area: self-hosted
issues: [305]
---
The Docker image now keeps a server database on a `/data` volume, the groundwork for sharing playlists and watch progress across devices. Mount a volume to keep it; nothing changes in the browser yet.
+2 -1
View File
@@ -17,7 +17,8 @@
"target": "node22",
"bundle": true,
"thirdParty": true,
"generatePackageJson": true
"generatePackageJson": true,
"external": ["better-sqlite3"]
},
"inputs": [
"default",
@@ -0,0 +1,32 @@
import { createServer, type Server } from 'node:http';
import { AddressInfo } from 'node:net';
import { closeGracefully } from './graceful-shutdown';
describe('closeGracefully', () => {
let server: Server;
afterEach(() => {
server?.closeAllConnections();
});
it('stops waiting for a request that never ends after the grace period', async () => {
server = createServer((_req, res) => {
res.writeHead(200);
res.write('streaming');
// Never ends, like a provider download that keeps sending bytes.
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const { port } = server.address() as AddressInfo;
const request = fetch(`http://127.0.0.1:${port}/`).catch(
() => undefined
);
await new Promise((resolve) => setTimeout(resolve, 50));
const started = Date.now();
await closeGracefully(server, 100);
expect(Date.now() - started).toBeLessThan(2_000);
expect(server.listening).toBe(false);
await request;
});
});
@@ -0,0 +1,27 @@
import type { Server } from 'node:http';
/** How long open requests may finish before their connections are cut. */
export const SHUTDOWN_GRACE_MS = 5_000;
/**
* Stops accepting connections, closes idle keep-alive sockets at once and
* cuts the remaining ones after `graceMs`: a provider download that keeps
* sending bytes must not hold the shutdown open until Docker kills it.
*/
export function closeGracefully(
server: Server,
graceMs = SHUTDOWN_GRACE_MS
): Promise<void> {
return new Promise((resolve) => {
const deadline = setTimeout(
() => server.closeAllConnections(),
graceMs
);
deadline.unref();
server.close(() => {
clearTimeout(deadline);
resolve();
});
server.closeIdleConnections();
});
}
@@ -0,0 +1,70 @@
import express from 'express';
import { withServer } from '../web-backend-app.spec-helpers';
import { createAccessGuard } from './access-guard';
function createApp() {
const app = express();
app.use(createAccessGuard('s3cret'));
app.get('/health', (_req, res) => res.json({ ok: true }));
app.get('/v1/instance', (_req, res) => res.json({ ok: true }));
app.get('/v1/playlists', (_req, res) => res.json([]));
return app;
}
describe('createAccessGuard', () => {
it('lets public routes through', () =>
withServer(createApp(), async (baseUrl) => {
expect((await fetch(`${baseUrl}/health`)).status).toBe(200);
expect((await fetch(`${baseUrl}/v1/instance`)).status).toBe(200);
}));
it('rejects a missing or wrong token with a JSON 401', () =>
withServer(createApp(), async (baseUrl) => {
const missing = await fetch(`${baseUrl}/v1/playlists`);
const wrong = await fetch(`${baseUrl}/v1/playlists`, {
headers: { authorization: 'Bearer nope' },
});
expect(missing.status).toBe(401);
await expect(missing.json()).resolves.toMatchObject({
code: 'unauthorized',
});
expect(wrong.status).toBe(401);
}));
it('accepts the right token and lets preflight through', () =>
withServer(createApp(), async (baseUrl) => {
const ok = await fetch(`${baseUrl}/v1/playlists`, {
headers: { authorization: 'Bearer s3cret' },
});
const preflight = await fetch(`${baseUrl}/v1/playlists`, {
method: 'OPTIONS',
});
expect(ok.status).toBe(200);
expect(preflight.status).not.toBe(401);
}));
it('reads the scheme case-insensitively and trims the token', () =>
withServer(createApp(), async (baseUrl) => {
const ok = await fetch(`${baseUrl}/v1/playlists`, {
headers: { authorization: 'bearer s3cret ' },
});
expect(ok.status).toBe(200);
}));
it('rejects a long whitespace header quickly', () =>
withServer(createApp(), async (baseUrl) => {
const started = Date.now();
const response = await fetch(`${baseUrl}/v1/playlists`, {
headers: {
// Under Node's 16 KB header limit, so the guard sees it.
authorization: `bearer ${' '.repeat(10_000)}x`,
},
});
expect(response.status).toBe(401);
expect(Date.now() - started).toBeLessThan(1_000);
}));
});
@@ -0,0 +1,51 @@
import { createHash, timingSafeEqual } from 'node:crypto';
import type { RequestHandler } from 'express';
/** Routes a browser must reach before it has a token. */
const PUBLIC_PATHS = new Set(['/', '/health', '/config.js', '/v1/instance']);
/**
* Optional shared-token protection for every other route, including the
* provider proxy routes. Tokens are compared as SHA-256 digests so the
* comparison is constant-time regardless of length.
*/
export function createAccessGuard(token: string): RequestHandler {
const expected = digest(token);
return (req, res, next) => {
if (req.method === 'OPTIONS' || PUBLIC_PATHS.has(req.path)) {
next();
return;
}
const token = readBearerToken(req.get('authorization'));
if (token && timingSafeEqual(digest(token), expected)) {
next();
return;
}
res.status(401).json({
message: 'This IPTVnator server requires an access token.',
status: 401,
code: 'unauthorized',
});
};
}
/**
* `Bearer <token>`, scheme case-insensitive. Plain string scanning instead
* of a regular expression keeps the cost linear in the header length.
*/
function readBearerToken(header: string | undefined): string | null {
const value = header?.trim() ?? '';
const separator = value.indexOf(' ');
if (separator < 0 || value.slice(0, separator).toLowerCase() !== 'bearer') {
return null;
}
const token = value.slice(separator + 1).trim();
return token ? token : null;
}
function digest(value: string): Buffer {
return createHash('sha256').update(value).digest();
}
@@ -0,0 +1,20 @@
import { matchesEtag } from './etag-match';
describe('matchesEtag', () => {
const etag = '"boot.0.1"';
it('matches the same strong or weakened tag', () => {
expect(matchesEtag(etag, etag)).toBe(true);
expect(matchesEtag(`W/${etag}`, etag)).toBe(true);
});
it('matches one of several listed tags and the wildcard', () => {
expect(matchesEtag(`"other", W/${etag}`, etag)).toBe(true);
expect(matchesEtag('*', etag)).toBe(true);
});
it('does not match a missing or different tag', () => {
expect(matchesEtag(undefined, etag)).toBe(false);
expect(matchesEtag('"boot.0.2"', etag)).toBe(false);
});
});
@@ -0,0 +1,22 @@
/**
* Weak comparison of an `If-None-Match` header with the current ETag, as
* RFC 9110 requires for GET. nginx weakens the ETag of a gzipped response
* (`W/"…"`), so clients send the weak form back.
*/
export function matchesEtag(
ifNoneMatch: string | undefined,
etag: string
): boolean {
if (!ifNoneMatch) {
return false;
}
const current = stripWeak(etag);
return ifNoneMatch
.split(',')
.map((tag) => tag.trim())
.some((tag) => tag === '*' || stripWeak(tag) === current);
}
function stripWeak(tag: string): string {
return tag.startsWith('W/') ? tag.slice(2) : tag;
}
@@ -0,0 +1,46 @@
import { Router } from 'express';
import {
SERVER_STORAGE_API_VERSION,
type ServerInstanceInfo,
type ServerStorageStatusResponse,
} from '@iptvnator/shared/interfaces';
import type { ServerStorage } from './server-storage';
import { StorageRequestError } from './storage-route-errors';
export function createInstanceRouter(deps: {
readonly storage: ServerStorage | null;
readonly accessTokenRequired: boolean;
}): Router {
const router = Router();
router.get('/instance', (_req, res) => {
const body: ServerInstanceInfo = {
service: 'iptvnator',
apiVersion: SERVER_STORAGE_API_VERSION,
instanceId: deps.storage?.database.instanceId ?? null,
storage: deps.storage ? 'server' : 'none',
auth: deps.accessTokenRequired ? 'token' : 'none',
};
res.set('Cache-Control', 'no-store').json(body);
});
router.get('/instance/status', (_req, res) => {
const storage = deps.storage;
if (!storage) {
throw new StorageRequestError(
404,
'storage_disabled',
'Server storage is not enabled.'
);
}
const body: ServerStorageStatusResponse = {
playlists: storage.playlists.count(),
databaseBytes: storage.database.sizeBytes(),
databasePath: storage.database.path,
startedAt: storage.startedAt.toISOString(),
};
res.set('Cache-Control', 'no-store').json(body);
});
return router;
}
@@ -0,0 +1,49 @@
import type { Express, RequestHandler } from 'express';
import { createAccessGuard } from './access-guard';
import { createInstanceRouter } from './instance-router';
import { createPlaylistsRouter } from './playlists-router';
import type { ServerStorage } from './server-storage';
import { storageErrorHandler } from './storage-route-errors';
import { createUserStateRouter } from './user-state-router';
export const DEFAULT_MAX_DOCUMENT_BYTES = 64 * 1024 * 1024;
export interface ServerStorageMountOptions {
readonly corsMiddleware: RequestHandler;
readonly storage: ServerStorage | null;
readonly accessToken: string | null;
readonly maxDocumentBytes: number;
}
/**
* Mounts `/v1` and the optional token guard. Call it before the provider
* proxy routes are registered so the guard also covers them. With a token
* the CORS headers apply to every route, so a cross-origin 401 stays
* readable for the PWA in development.
*/
export function mountServerStorage(
app: Express,
options: ServerStorageMountOptions
): void {
app.use('/v1', options.corsMiddleware);
if (options.accessToken) {
app.use(options.corsMiddleware);
app.use(createAccessGuard(options.accessToken));
}
app.use(
'/v1',
createInstanceRouter({
storage: options.storage,
accessTokenRequired: !!options.accessToken,
})
);
if (options.storage) {
app.use(
'/v1',
createPlaylistsRouter(options.storage, options.maxDocumentBytes)
);
app.use('/v1', createUserStateRouter(options.storage));
}
app.use('/v1', storageErrorHandler());
}
@@ -0,0 +1,29 @@
import { PlaylistRevisions } from './playlist-revisions';
describe('PlaylistRevisions', () => {
it('keeps an ETag until the playlist is written', () => {
const revisions = new PlaylistRevisions('boot');
expect(revisions.etag('a')).toBe('"boot.0.0"');
expect(revisions.etag('a')).toBe('"boot.0.0"');
expect(revisions.bump('a')).toBe('"boot.0.1"');
expect(revisions.etag('a')).toBe('"boot.0.1"');
expect(revisions.etag('b')).toBe('"boot.0.0"');
});
it('invalidates every ETag on reset', () => {
const revisions = new PlaylistRevisions('boot');
revisions.bump('a');
revisions.reset();
expect(revisions.etag('a')).toBe('"boot.1.0"');
expect(revisions.etag('b')).toBe('"boot.1.0"');
});
it('uses a random boot id by default', () => {
expect(new PlaylistRevisions().etag('a')).not.toBe(
new PlaylistRevisions().etag('a')
);
});
});
@@ -0,0 +1,32 @@
import { randomUUID } from 'node:crypto';
/**
* ETags for playlist documents. The server is the only writer of its
* database, so an in-memory counter per playlist is exact. The boot id
* invalidates every client cache after a restart; the generation does so
* after "delete all".
*/
export class PlaylistRevisions {
private readonly revisions = new Map<string, number>();
private generation = 0;
constructor(private readonly bootId = randomUUID().slice(0, 8)) {}
etag(playlistId: string): string {
const revision = this.revisions.get(playlistId) ?? 0;
return `"${this.bootId}.${this.generation}.${revision}"`;
}
bump(playlistId: string): string {
this.revisions.set(
playlistId,
(this.revisions.get(playlistId) ?? 0) + 1
);
return this.etag(playlistId);
}
reset(): void {
this.revisions.clear();
this.generation += 1;
}
}
@@ -0,0 +1,107 @@
import express, { Router, type Response } from 'express';
import { matchesEtag } from './etag-match';
import {
deleteAllPlaylistsWithState,
deletePlaylistWithState,
type ServerStorage,
} from './server-storage';
import { StorageRequestError } from './storage-route-errors';
export function createPlaylistsRouter(
storage: ServerStorage,
maxDocumentBytes: number
): Router {
const router = Router();
const json = express.json({ limit: maxDocumentBytes });
const { playlists, revisions } = storage;
router.get('/playlists', (req, res) => {
res.set('Cache-Control', 'no-store').json(
req.query['view'] === 'meta'
? playlists.listMetas()
: playlists.listAll()
);
});
router.get('/playlists/:id', (req, res) => {
const id = requirePlaylistId(req.params.id);
// A cheap existence check first: a tag must never confirm a
// representation that does not exist.
if (!playlists.exists(id)) {
throw new StorageRequestError(
404,
'not_found',
'Playlist not found.'
);
}
const etag = revisions.etag(id);
withEtag(res, etag);
if (matchesEtag(req.get('if-none-match'), etag)) {
res.status(304).end();
return;
}
const document = playlists.get(id);
if (!document) {
throw new StorageRequestError(
404,
'not_found',
'Playlist not found.'
);
}
res.json(document);
});
router.put('/playlists/:id', json, (req, res) => {
const id = requirePlaylistId(req.params.id);
playlists.upsert(
{ ...requireDocument(req.body), _id: id },
{ keepPayload: req.query['keepPayload'] === '1' }
);
withEtag(res, revisions.bump(id)).json({ success: true });
});
router.delete('/playlists/:id', (req, res) => {
deletePlaylistWithState(storage, requirePlaylistId(req.params.id));
res.json({ success: true });
});
router.delete('/playlists', (_req, res) => {
deleteAllPlaylistsWithState(storage);
res.json({ success: true });
});
return router;
}
function withEtag(res: Response, etag: string): Response {
return res
.set('ETag', etag)
.set('Access-Control-Expose-Headers', 'ETag')
.set('Cache-Control', 'no-cache');
}
/**
* The row codec trims ids, so an id with surrounding whitespace would be
* stored under a different key than the one its ETag tracks.
*/
function requirePlaylistId(value: string): string {
if (!value || value !== value.trim()) {
throw new StorageRequestError(
400,
'invalid_id',
'A playlist id cannot be empty or start or end with whitespace.'
);
}
return value;
}
function requireDocument(value: unknown): Record<string, unknown> {
if (!value || typeof value !== 'object' || Array.isArray(value)) {
throw new StorageRequestError(
400,
'invalid_body',
'Expected a playlist object.'
);
}
return value as Record<string, unknown>;
}
@@ -0,0 +1,93 @@
import { mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { sql } from 'drizzle-orm';
import { IPTVNATOR_DATA_DIR_ENV } from '@iptvnator/shared/database';
import { openServerDatabase } from './server-database';
import { ServerPlaylistRepository } from './server-playlist-repository';
import { m3uDocument } from './server-storage.spec-helpers';
describe('openServerDatabase', () => {
let dataDir: string;
let previous: string | undefined;
beforeEach(() => {
dataDir = mkdtempSync(join(tmpdir(), 'iptvnator-server-db-'));
previous = process.env[IPTVNATOR_DATA_DIR_ENV];
process.env[IPTVNATOR_DATA_DIR_ENV] = dataDir;
});
afterEach(() => {
if (previous === undefined) {
delete process.env[IPTVNATOR_DATA_DIR_ENV];
} else {
process.env[IPTVNATOR_DATA_DIR_ENV] = previous;
}
rmSync(dataDir, { recursive: true, force: true });
});
it('keeps the instance id across restarts', async () => {
const first = await openServerDatabase();
const instanceId = first.instanceId;
first.close();
const second = await openServerDatabase();
expect(second.instanceId).toBe(instanceId);
second.close();
});
it('still opens when the stored instance id is empty', async () => {
const first = await openServerDatabase();
new ServerPlaylistRepository(first.db).setAppState(
'server:instance-id',
''
);
first.close();
const second = await openServerDatabase();
expect(second.instanceId).toMatch(/^[0-9a-f-]{36}$/);
second.close();
});
it('runs several writes in one transaction', async () => {
const database = await openServerDatabase();
const playlists = new ServerPlaylistRepository(database.db);
expect(() =>
database.transaction(() => {
playlists.setAppState('a', '1');
throw new Error('fail after the first write');
})
).toThrow('fail after the first write');
expect(playlists.getAppState('a')).toBeNull();
database.close();
});
it('adds metadata for playlists written without it', async () => {
const first = await openServerDatabase();
new ServerPlaylistRepository(first.db).upsert(
{ ...m3uDocument('p1'), filename: 'family.m3u' },
{ keepPayload: false }
);
new ServerPlaylistRepository(first.db).upsert(m3uDocument('p2'), {
keepPayload: false,
});
// As in a database the desktop app wrote: rows, no server copy.
first.db.run(sql`DELETE FROM server_playlist_meta`);
first.db.run(
sql`UPDATE playlists SET payload = 'not json' WHERE id = 'p2'`
);
first.close();
const second = await openServerDatabase();
const metas = new ServerPlaylistRepository(second.db).listMetas();
expect(metas).toEqual(
expect.arrayContaining([
expect.objectContaining({ _id: 'p1', filename: 'family.m3u' }),
expect.objectContaining({ _id: 'p2', title: 'Family' }),
])
);
expect(metas[0]['playlist']).toBeUndefined();
second.close();
});
});
@@ -0,0 +1,82 @@
import { randomUUID } from 'node:crypto';
import { statSync } from 'node:fs';
import { eq, sql } from 'drizzle-orm';
import {
appState,
closeDatabase,
getDatabasePath,
initDatabase,
type DatabaseInstance,
} from '@iptvnator/shared/database';
import {
SERVER_PLAYLIST_META_BACKFILL_SQL,
SERVER_PLAYLIST_META_TABLE_SQL,
} from './server-playlist-meta.table';
import { SERVER_USER_STATE_TABLE_SQL } from './server-user-state.table';
const INSTANCE_ID_KEY = 'server:instance-id';
export interface ServerDatabase {
readonly db: DatabaseInstance;
readonly instanceId: string;
readonly path: string;
sizeBytes(): number;
/** Runs `work` in one SQLite transaction on the single connection. */
transaction<T>(work: () => T): T;
close(): void;
}
/**
* Opens the shared IPTVnator database (full schema and migrations, the
* same as the desktop app) and adds the server-only tables. The path comes
* from `IPTVNATOR_DATA_DIR`.
*/
export async function openServerDatabase(): Promise<ServerDatabase> {
const db = await initDatabase();
db.run(sql.raw(SERVER_USER_STATE_TABLE_SQL));
db.run(sql.raw(SERVER_PLAYLIST_META_TABLE_SQL));
db.run(sql.raw(SERVER_PLAYLIST_META_BACKFILL_SQL));
const path = getDatabasePath();
return {
db,
instanceId: ensureInstanceId(db),
path,
sizeBytes: () => fileSize(path) + fileSize(`${path}-wal`),
// better-sqlite3 runs every statement synchronously on one
// connection, so writes made inside `work` belong to this
// transaction even when they go through other repositories.
transaction: (work) => db.transaction(() => work()),
close: () => closeDatabase(),
};
}
function ensureInstanceId(db: DatabaseInstance): string {
const existing = db
.select({ value: appState.value })
.from(appState)
.where(eq(appState.key, INSTANCE_ID_KEY))
.get();
if (existing?.value) {
return existing.value;
}
const instanceId = randomUUID();
const updatedAt = new Date().toISOString();
db.insert(appState)
.values({ key: INSTANCE_ID_KEY, value: instanceId, updatedAt })
.onConflictDoUpdate({
target: appState.key,
set: { value: instanceId, updatedAt },
})
.run();
return instanceId;
}
function fileSize(path: string): number {
try {
return statSync(path).size;
} catch {
return 0;
}
}
@@ -0,0 +1,57 @@
import { SERVER_PAYLOAD_FIELDS } from '@iptvnator/shared/interfaces';
import { sql, type SQL, type SQLWrapper } from 'drizzle-orm';
import { sqliteTable, text } from 'drizzle-orm/sqlite-core';
/**
* Server-only table: each playlist document without its M3U payload
* fields, written with every upsert. Metadata reads take the JSON-only
* fields (filename, hidden groups, Stalker identity) from here, so they
* never walk the payload blobs. Created by `openServerDatabase`, never by
* the desktop app.
*/
export const SERVER_PLAYLIST_META_TABLE_SQL = `CREATE TABLE IF NOT EXISTS server_playlist_meta (
playlist_id TEXT PRIMARY KEY NOT NULL,
document TEXT
)`;
const PAYLOAD_PATHS = SERVER_PAYLOAD_FIELDS.map((field) => `'$.${field}'`).join(
', '
);
/**
* Fills the table for playlists written without it, for example a
* database copied from the desktop app. Reads each such payload once;
* later starts find nothing to do.
*/
export const SERVER_PLAYLIST_META_BACKFILL_SQL = `INSERT INTO server_playlist_meta (playlist_id, document)
SELECT id, CASE
WHEN json_valid(payload) THEN json_remove(payload, ${PAYLOAD_PATHS})
ELSE NULL
END
FROM playlists
WHERE id NOT IN (SELECT playlist_id FROM server_playlist_meta)`;
/** A stored payload without its M3U fields, or NULL when it is not JSON. */
export function payloadMetadata(payload: SQLWrapper): SQL<string | null> {
return sql<string | null>`CASE
WHEN json_valid(${payload})
THEN json_remove(${payload}, ${sql.raw(PAYLOAD_PATHS)})
ELSE NULL
END`;
}
export const serverPlaylistMeta = sqliteTable('server_playlist_meta', {
playlistId: text('playlist_id').primaryKey().notNull(),
document: text('document'),
});
/** The document as metadata reads see it: no payload fields. */
export function metadataDocument(
document: Readonly<Record<string, unknown>>
): string {
const metadata: Record<string, unknown> = { ...document };
for (const field of SERVER_PAYLOAD_FIELDS) {
delete metadata[field];
}
return JSON.stringify(metadata);
}
@@ -0,0 +1,219 @@
import { sql } from 'drizzle-orm';
import { PlaylistPayloadMissingError } from './server-playlist-repository';
import { m3uDocument, withTempStorage } from './server-storage.spec-helpers';
describe('ServerPlaylistRepository', () => {
it('stores and reads a whole playlist document', () =>
withTempStorage(({ playlists }) => {
playlists.upsert(m3uDocument('p1'), { keepPayload: false });
expect(playlists.get('p1')).toMatchObject({
_id: 'p1',
title: 'Family',
playlist: m3uDocument('p1').playlist,
});
expect(playlists.count()).toBe(1);
expect(playlists.get('missing')).toBeNull();
}));
it('lists metadata without the payload', () =>
withTempStorage(({ playlists }) => {
playlists.upsert(m3uDocument('p1'), { keepPayload: false });
const [meta] = playlists.listMetas();
expect(meta).toMatchObject({ _id: 'p1', title: 'Family' });
expect(meta['playlist']).toBeUndefined();
expect(playlists.listAll()[0]['playlist']).toEqual(
m3uDocument('p1').playlist
);
}));
it('keeps the stored payload when asked to', () =>
withTempStorage(({ playlists }) => {
playlists.upsert(m3uDocument('p1'), { keepPayload: false });
const meta: Record<string, unknown> = { ...m3uDocument('p1') };
delete meta['playlist'];
playlists.upsert(
{
...meta,
title: 'Renamed',
favorites: ['https://example.test/a'],
},
{ keepPayload: true }
);
expect(playlists.get('p1')).toMatchObject({
title: 'Renamed',
favorites: ['https://example.test/a'],
playlist: m3uDocument('p1').playlist,
});
}));
it('keeps payload-only metadata in metadata reads and round-trips', () =>
withTempStorage(({ playlists }) => {
playlists.upsert(
{
...m3uDocument('p1'),
filename: 'family.m3u',
hiddenGroupTitles: ['Adult'],
stalkerSerialNumber: 'SN-1',
},
{ keepPayload: false }
);
const [meta] = playlists.listMetas();
expect(meta).toMatchObject({
filename: 'family.m3u',
hiddenGroupTitles: ['Adult'],
stalkerSerialNumber: 'SN-1',
});
expect(meta['playlist']).toBeUndefined();
playlists.upsert(
{ ...meta, title: 'Renamed' },
{ keepPayload: true }
);
expect(playlists.get('p1')).toMatchObject({
title: 'Renamed',
filename: 'family.m3u',
hiddenGroupTitles: ['Adult'],
stalkerSerialNumber: 'SN-1',
playlist: m3uDocument('p1').playlist,
});
}));
it('serves complete metadata without reading the payload', () =>
withTempStorage(({ database, playlists }) => {
playlists.upsert(
{
...m3uDocument('p1'),
filename: 'family.m3u',
hiddenGroupTitles: ['Adult'],
},
{ keepPayload: false }
);
// A metadata read must not depend on the payload blob at all.
database.db.run(
sql`UPDATE playlists SET payload = 'not json' WHERE id = 'p1'`
);
expect(playlists.listMetas()[0]).toMatchObject({
_id: 'p1',
title: 'Family',
filename: 'family.m3u',
hiddenGroupTitles: ['Adult'],
});
}));
it('keeps a carried-over panel clock in metadata reads', () =>
withTempStorage(({ playlists }) => {
const panel = {
_id: 'x1',
title: 'Panel',
serverUrl: 'http://panel.test',
username: 'user',
password: 'secret',
importDate: '2026-10-01T00:00:00.000Z',
};
playlists.upsert(
{ ...panel, serverTimezone: 'Europe/Berlin' },
{ keepPayload: false }
);
// An older snapshot without the clock, same panel account.
playlists.upsert(
{ ...panel, title: 'Renamed' },
{ keepPayload: false }
);
const kept = { title: 'Renamed', serverTimezone: 'Europe/Berlin' };
expect(playlists.get('x1')).toMatchObject(kept);
expect(playlists.listMetas()[0]).toMatchObject(kept);
// A different account drops the clock in both reads.
playlists.upsert(
{ ...panel, password: 'changed' },
{ keepPayload: false }
);
expect(playlists.get('x1')?.['serverTimezone']).toBeUndefined();
expect(playlists.listMetas()[0]['serverTimezone']).toBeUndefined();
}));
it('drops the metadata copy with its playlist', () =>
withTempStorage(({ playlists }) => {
playlists.upsert(m3uDocument('p1'), { keepPayload: false });
playlists.upsert(m3uDocument('p2'), { keepPayload: false });
playlists.delete('p1');
expect(playlists.listMetas().map((p) => p['_id'])).toEqual(['p2']);
playlists.deleteAll();
expect(playlists.listMetas()).toEqual([]);
}));
it('clears optional columns that a replacement leaves empty', () =>
withTempStorage(({ playlists }) => {
playlists.upsert(
{
...m3uDocument('p1'),
userAgent: 'UA/1',
referrer: 'https://ref.test',
origin: 'https://origin.test',
},
{ keepPayload: false }
);
playlists.upsert(
{
...m3uDocument('p1'),
userAgent: '',
referrer: '',
},
{ keepPayload: false }
);
const stored = playlists.get('p1');
expect(stored?.['userAgent']).toBeUndefined();
expect(stored?.['referrer']).toBeUndefined();
expect(stored?.['origin']).toBeUndefined();
expect(playlists.listMetas()[0]['userAgent']).toBeUndefined();
}));
it('tells whether a playlist exists', () =>
withTempStorage(({ playlists }) => {
playlists.upsert(m3uDocument('p1'), { keepPayload: false });
expect(playlists.exists('p1')).toBe(true);
expect(playlists.exists('nope')).toBe(false);
}));
it('refuses keepPayload for an unknown playlist', () =>
withTempStorage(({ playlists }) => {
expect(() =>
playlists.upsert(
{ _id: 'nope', title: 'x' },
{ keepPayload: true }
)
).toThrow(PlaylistPayloadMissingError);
}));
it('deletes one or all playlists', () =>
withTempStorage(({ playlists }) => {
playlists.upsert(m3uDocument('p1'), { keepPayload: false });
playlists.upsert(m3uDocument('p2'), { keepPayload: false });
playlists.delete('p1');
expect(playlists.listMetas().map((p) => p['_id'])).toEqual(['p2']);
playlists.deleteAll();
expect(playlists.count()).toBe(0);
}));
it('reads and writes app state', () =>
withTempStorage(({ playlists }) => {
expect(playlists.getAppState('flag')).toBeNull();
playlists.setAppState('flag', '1');
expect(playlists.getAppState('flag')).toBe('1');
}));
});
@@ -0,0 +1,255 @@
import { eq, getTableColumns } from 'drizzle-orm';
import type { DatabaseInstance } from '@iptvnator/shared/database';
import {
buildPlaylistRow,
parseAppPlaylist,
playlistConflictUpdate,
} from '@iptvnator/shared/database/app-playlist';
import { appState, playlists } from '@iptvnator/shared/database/schema';
import type {
NewPlaylist,
Playlist as PlaylistRow,
} from '@iptvnator/shared/database/schema';
import { SERVER_PAYLOAD_FIELDS } from '@iptvnator/shared/interfaces';
import {
metadataDocument,
payloadMetadata,
serverPlaylistMeta,
} from './server-playlist-meta.table';
export type PlaylistDocument = Record<string, unknown>;
export class PlaylistPayloadMissingError extends Error {
constructor() {
super('No stored payload to keep');
this.name = 'PlaylistPayloadMissingError';
}
}
export class InvalidPlaylistDocumentError extends Error {
constructor() {
super('A playlist document needs an id');
this.name = 'InvalidPlaylistDocumentError';
}
}
/** Every column except the M3U payload, for metadata reads. */
function metadataColumns() {
const { payload, ...columns } = getTableColumns(playlists);
void payload;
return columns;
}
const metaColumns = metadataColumns();
/**
* Columns a whole-document upsert must rewrite even when the document no
* longer has a value: the codec maps empty values to `undefined`, which an
* `ON CONFLICT DO UPDATE` skips, so a cleared field would keep its old
* column value and `parseAppPlaylist` would bring it back.
*/
const REPLACEABLE_COLUMNS = [
'serverUrl',
'username',
'password',
'lastUpdated',
'userAgent',
'origin',
'referrer',
'filePath',
'epgUrls',
'detectedEpgUrls',
'manualEpgUrls',
'disabledEpgUrls',
'macAddress',
'url',
'portalUrl',
'count',
'importDate',
'updateDate',
'position',
'favorites',
'recentlyViewed',
] as const satisfies readonly (keyof NewPlaylist)[];
function clearedColumns(row: NewPlaylist): Partial<NewPlaylist> {
const cleared: Partial<Record<keyof NewPlaylist, null>> = {};
for (const column of REPLACEABLE_COLUMNS) {
if (row[column] === undefined) {
cleared[column] = null;
}
}
return cleared as Partial<NewPlaylist>;
}
/** Playlist documents in the shared `playlists` table, same rows as desktop. */
export class ServerPlaylistRepository {
constructor(private readonly db: DatabaseInstance) {}
count(): number {
return this.db.select({ id: playlists.id }).from(playlists).all()
.length;
}
/**
* Columns plus the payload-free copy of each document, so metadata
* reads keep fields that live only in the JSON (filename, hidden
* groups, Stalker identity) and a metadata round-trip with keepPayload
* loses nothing, without reading a payload blob.
*/
listMetas(): PlaylistDocument[] {
return this.db
.select({ ...metaColumns, payload: serverPlaylistMeta.document })
.from(playlists)
.leftJoin(
serverPlaylistMeta,
eq(serverPlaylistMeta.playlistId, playlists.id)
)
.all()
.map((row) => parseAppPlaylist(row as PlaylistRow));
}
exists(playlistId: string): boolean {
return !!this.db
.select({ id: playlists.id })
.from(playlists)
.where(eq(playlists.id, playlistId))
.get();
}
listAll(): PlaylistDocument[] {
return this.db
.select()
.from(playlists)
.all()
.map((row) => parseAppPlaylist(row));
}
get(playlistId: string): PlaylistDocument | null {
const row = this.db
.select()
.from(playlists)
.where(eq(playlists.id, playlistId))
.get();
return row ? parseAppPlaylist(row) : null;
}
upsert(
document: PlaylistDocument,
options: { readonly keepPayload: boolean }
): void {
const next = options.keepPayload
? this.withStoredPayload(document)
: document;
const row = buildPlaylistRow(next);
if (!row) {
throw new InvalidPlaylistDocumentError();
}
const update = {
...playlistConflictUpdate(row, next),
...clearedColumns(row),
};
this.db.transaction((tx) => {
tx.insert(playlists)
.values(row)
.onConflictDoUpdate({ target: playlists.id, set: update })
.run();
// The conflict update may carry stored fields (the panel clock)
// into the payload; the copy must then match what was saved.
const metadata =
update.payload === row.payload
? metadataDocument(next)
: (tx
.select({
document: payloadMetadata(playlists.payload),
})
.from(playlists)
.where(eq(playlists.id, row.id))
.get()?.document ?? null);
tx.insert(serverPlaylistMeta)
.values({ playlistId: row.id, document: metadata })
.onConflictDoUpdate({
target: serverPlaylistMeta.playlistId,
set: { document: metadata },
})
.run();
});
}
delete(playlistId: string): void {
this.db.transaction((tx) => {
tx.delete(playlists).where(eq(playlists.id, playlistId)).run();
tx.delete(serverPlaylistMeta)
.where(eq(serverPlaylistMeta.playlistId, playlistId))
.run();
});
}
deleteAll(): void {
this.db.transaction((tx) => {
tx.delete(playlists).run();
tx.delete(serverPlaylistMeta).run();
});
}
getAppState(key: string): string | null {
return (
this.db
.select({ value: appState.value })
.from(appState)
.where(eq(appState.key, key))
.get()?.value ?? null
);
}
setAppState(key: string, value: string): void {
const updatedAt = new Date().toISOString();
this.db
.insert(appState)
.values({ key, value, updatedAt })
.onConflictDoUpdate({
target: appState.key,
set: { value, updatedAt },
})
.run();
}
/** Fills the payload fields the client left out from the stored row. */
private withStoredPayload(document: PlaylistDocument): PlaylistDocument {
const id = String(document['_id'] ?? '');
const stored = this.db
.select({ payload: playlists.payload })
.from(playlists)
.where(eq(playlists.id, id))
.get();
const parsed = parseStoredPayload(stored?.payload);
if (!parsed) {
throw new PlaylistPayloadMissingError();
}
const merged: PlaylistDocument = { ...document };
for (const field of SERVER_PAYLOAD_FIELDS) {
if (!(field in merged) && field in parsed) {
merged[field] = parsed[field];
}
}
return merged;
}
}
function parseStoredPayload(
payload: string | null | undefined
): PlaylistDocument | null {
if (!payload) {
return null;
}
try {
const parsed: unknown = JSON.parse(payload);
return parsed && typeof parsed === 'object'
? (parsed as PlaylistDocument)
: null;
} catch {
return null;
}
}
@@ -0,0 +1,45 @@
import { mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { IPTVNATOR_DATA_DIR_ENV } from '@iptvnator/shared/database';
import { openServerStorage, type ServerStorage } from './server-storage';
/** Real SQLite in a throwaway data directory, closed and removed afterwards. */
export async function withTempStorage<T>(
run: (storage: ServerStorage) => Promise<T> | T
): Promise<T> {
const dataDir = mkdtempSync(join(tmpdir(), 'iptvnator-server-storage-'));
const previous = process.env[IPTVNATOR_DATA_DIR_ENV];
process.env[IPTVNATOR_DATA_DIR_ENV] = dataDir;
const storage = await openServerStorage();
try {
return await run(storage);
} finally {
storage.database.close();
if (previous === undefined) {
delete process.env[IPTVNATOR_DATA_DIR_ENV];
} else {
process.env[IPTVNATOR_DATA_DIR_ENV] = previous;
}
rmSync(dataDir, { recursive: true, force: true });
}
}
export const m3uDocument = (id: string, title = 'Family') => ({
_id: id,
title,
url: `https://example.test/${id}.m3u`,
count: 1,
importDate: '2026-10-01T00:00:00.000Z',
lastUsage: '2026-10-01T00:00:00.000Z',
favorites: [],
recentlyViewed: [],
autoRefresh: false,
playlist: {
header: { raw: '#EXTM3U' },
items: [
{ id: 'a', url: 'https://example.test/a', raw: '#EXTINF:-1,A' },
],
},
});
@@ -0,0 +1,44 @@
import { PlaylistRevisions } from './playlist-revisions';
import { openServerDatabase, type ServerDatabase } from './server-database';
import { ServerPlaylistRepository } from './server-playlist-repository';
import { ServerUserStateRepository } from './server-user-state-repository';
export interface ServerStorage {
readonly database: ServerDatabase;
readonly playlists: ServerPlaylistRepository;
readonly userState: ServerUserStateRepository;
readonly revisions: PlaylistRevisions;
readonly startedAt: Date;
}
export async function openServerStorage(): Promise<ServerStorage> {
const database = await openServerDatabase();
return {
database,
playlists: new ServerPlaylistRepository(database.db),
userState: new ServerUserStateRepository(database.db),
revisions: new PlaylistRevisions(),
startedAt: new Date(),
};
}
/** Deletes a playlist and its user state together, then moves its ETag. */
export function deletePlaylistWithState(
storage: ServerStorage,
playlistId: string
): void {
storage.database.transaction(() => {
storage.playlists.delete(playlistId);
storage.userState.deleteForPlaylist(playlistId);
});
storage.revisions.bump(playlistId);
}
/** Deletes every playlist and all user state together. */
export function deleteAllPlaylistsWithState(storage: ServerStorage): void {
storage.database.transaction(() => {
storage.playlists.deleteAll();
storage.userState.deleteAll();
});
storage.revisions.reset();
}
@@ -0,0 +1,54 @@
import { withTempStorage } from './server-storage.spec-helpers';
describe('ServerUserStateRepository', () => {
it('returns every namespace, empty by default', () =>
withTempStorage(({ userState }) => {
expect(userState.all()).toEqual({
'xtream-favorites': {},
'xtream-recent-items': {},
'xtream-collection-items': {},
'xtream-playback-positions': {},
});
}));
it('stores, overwrites and deletes one entry', () =>
withTempStorage(({ userState }) => {
userState.put('xtream-favorites', 'p1', ['live:1']);
userState.put('xtream-favorites', 'p1', ['live:2']);
expect(userState.all()['xtream-favorites']).toEqual({
p1: ['live:2'],
});
userState.delete('xtream-favorites', 'p1');
expect(userState.all()['xtream-favorites']).toEqual({});
}));
it('deletes a playlist across namespaces', () =>
withTempStorage(({ userState }) => {
userState.put('xtream-favorites', 'p1', ['live:1']);
userState.put('xtream-recent-items', 'p1', []);
userState.put('xtream-favorites', 'p2', ['live:3']);
userState.deleteForPlaylist('p1');
const all = userState.all();
expect(all['xtream-favorites']).toEqual({ p2: ['live:3'] });
expect(all['xtream-recent-items']).toEqual({});
}));
it('round-trips a playlist id that names an Object.prototype key', () =>
withTempStorage(({ userState }) => {
userState.put('xtream-favorites', '__proto__', ['live:1']);
userState.put('xtream-favorites', 'constructor', ['live:2']);
const favorites = userState.all()['xtream-favorites'];
expect(Object.getOwnPropertyNames(favorites).sort()).toEqual([
'__proto__',
'constructor',
]);
expect(JSON.parse(JSON.stringify(favorites))).toEqual(
JSON.parse('{"__proto__":["live:1"],"constructor":["live:2"]}')
);
}));
});
@@ -0,0 +1,72 @@
import { and, eq } from 'drizzle-orm';
import type { DatabaseInstance } from '@iptvnator/shared/database';
import {
SERVER_USER_STATE_NAMESPACES,
type ServerUserStateNamespace,
type ServerUserStateResponse,
} from '@iptvnator/shared/interfaces';
import { serverUserState } from './server-user-state.table';
/** Xtream PWA user data, one JSON value per namespace and playlist. */
export class ServerUserStateRepository {
constructor(private readonly db: DatabaseInstance) {}
all(): ServerUserStateResponse['entries'] {
// Prototype-free buckets: a playlist id such as `__proto__` must be
// stored as an own key, not change the bucket's prototype.
const entries = Object.fromEntries(
SERVER_USER_STATE_NAMESPACES.map((namespace) => [
namespace,
Object.create(null) as Record<string, unknown>,
])
) as ServerUserStateResponse['entries'];
for (const row of this.db.select().from(serverUserState).all()) {
const bucket = entries[row.namespace as ServerUserStateNamespace];
if (bucket) {
bucket[row.playlistId] = JSON.parse(row.value);
}
}
return entries;
}
put(
namespace: ServerUserStateNamespace,
playlistId: string,
value: unknown
): void {
const serialized = JSON.stringify(value ?? null);
const updatedAt = new Date().toISOString();
this.db
.insert(serverUserState)
.values({ namespace, playlistId, value: serialized, updatedAt })
.onConflictDoUpdate({
target: [serverUserState.namespace, serverUserState.playlistId],
set: { value: serialized, updatedAt },
})
.run();
}
delete(namespace: ServerUserStateNamespace, playlistId: string): void {
this.db
.delete(serverUserState)
.where(
and(
eq(serverUserState.namespace, namespace),
eq(serverUserState.playlistId, playlistId)
)
)
.run();
}
deleteForPlaylist(playlistId: string): void {
this.db
.delete(serverUserState)
.where(eq(serverUserState.playlistId, playlistId))
.run();
}
deleteAll(): void {
this.db.delete(serverUserState).run();
}
}
@@ -0,0 +1,24 @@
import { primaryKey, sqliteTable, text } from 'drizzle-orm/sqlite-core';
/**
* Server-only table: one JSON value per Xtream user-data namespace and
* playlist. Created by `openServerDatabase`, never by the desktop app.
*/
export const SERVER_USER_STATE_TABLE_SQL = `CREATE TABLE IF NOT EXISTS server_user_state (
namespace TEXT NOT NULL,
playlist_id TEXT NOT NULL,
value TEXT NOT NULL,
updated_at TEXT NOT NULL,
PRIMARY KEY (namespace, playlist_id)
)`;
export const serverUserState = sqliteTable(
'server_user_state',
{
namespace: text('namespace').notNull(),
playlistId: text('playlist_id').notNull(),
value: text('value').notNull(),
updatedAt: text('updated_at').notNull(),
},
(table) => [primaryKey({ columns: [table.namespace, table.playlistId] })]
);
@@ -0,0 +1,45 @@
import { readServerStorageConfig } from './storage-config';
describe('readServerStorageConfig', () => {
it('keeps storage off and the API open by default', () => {
expect(readServerStorageConfig({})).toEqual({
enabled: false,
accessToken: null,
maxDocumentBytes: 64 * 1024 * 1024,
});
});
it('enables storage only for IPTVNATOR_STORAGE=server', () => {
expect(
readServerStorageConfig({ IPTVNATOR_STORAGE: ' Server ' }).enabled
).toBe(true);
expect(
readServerStorageConfig({ IPTVNATOR_STORAGE: 'none' }).enabled
).toBe(false);
expect(
readServerStorageConfig({ IPTVNATOR_STORAGE: '1' }).enabled
).toBe(false);
});
it('reads a trimmed access token and ignores a blank one', () => {
expect(
readServerStorageConfig({ IPTVNATOR_ACCESS_TOKEN: ' s3cret ' })
.accessToken
).toBe('s3cret');
expect(
readServerStorageConfig({ IPTVNATOR_ACCESS_TOKEN: ' ' })
.accessToken
).toBeNull();
});
it('reads a positive document limit in megabytes', () => {
expect(
readServerStorageConfig({ IPTVNATOR_MAX_DOCUMENT_MB: '8' })
.maxDocumentBytes
).toBe(8 * 1024 * 1024);
expect(
readServerStorageConfig({ IPTVNATOR_MAX_DOCUMENT_MB: '-1' })
.maxDocumentBytes
).toBe(64 * 1024 * 1024);
});
});
@@ -0,0 +1,28 @@
export interface ServerStorageConfig {
/** `IPTVNATOR_STORAGE=server`; the Docker image sets it. */
readonly enabled: boolean;
/** `IPTVNATOR_ACCESS_TOKEN`; guards every non-public route when set. */
readonly accessToken: string | null;
/** Largest accepted playlist document (`IPTVNATOR_MAX_DOCUMENT_MB`). */
readonly maxDocumentBytes: number;
}
const DEFAULT_MAX_DOCUMENT_MB = 64;
const BYTES_PER_MB = 1024 * 1024;
export function readServerStorageConfig(
env: Record<string, string | undefined> = process.env
): ServerStorageConfig {
const mode = env['IPTVNATOR_STORAGE']?.trim().toLowerCase();
const token = env['IPTVNATOR_ACCESS_TOKEN']?.trim();
const megabytes = Number(env['IPTVNATOR_MAX_DOCUMENT_MB']);
return {
enabled: mode === 'server',
accessToken: token ? token : null,
maxDocumentBytes:
(Number.isFinite(megabytes) && megabytes > 0
? megabytes
: DEFAULT_MAX_DOCUMENT_MB) * BYTES_PER_MB,
};
}
@@ -0,0 +1,40 @@
import {
InvalidPlaylistDocumentError,
PlaylistPayloadMissingError,
} from './server-playlist-repository';
import {
describeStorageError,
StorageRequestError,
toStorageErrorBody,
} from './storage-route-errors';
describe('toStorageErrorBody', () => {
it.each([
[
new StorageRequestError(404, 'not_found', 'Playlist not found.'),
404,
'not_found',
],
[new PlaylistPayloadMissingError(), 409, 'payload_missing'],
[new InvalidPlaylistDocumentError(), 400, 'invalid_body'],
[{ type: 'entity.too.large' }, 413, 'too_large'],
[{ type: 'entity.parse.failed' }, 400, 'invalid_json'],
[
Object.assign(new Error('busy'), { code: 'SQLITE_BUSY' }),
503,
'storage_busy',
],
[new Error('SELECT secret FROM x'), 500, 'storage_error'],
])('maps %p to %i', (error, status, code) => {
expect(toStorageErrorBody(error)).toMatchObject({ status, code });
});
it('never puts the raw error message into a 500 body or log line', () => {
const error = new Error('password=hunter2');
expect(JSON.stringify(toStorageErrorBody(error))).not.toContain(
'hunter2'
);
expect(describeStorageError(error)).toBe('Error');
});
});
@@ -0,0 +1,95 @@
import type { ErrorRequestHandler } from 'express';
import type { ServerStorageErrorBody } from '@iptvnator/shared/interfaces';
import {
InvalidPlaylistDocumentError,
PlaylistPayloadMissingError,
} from './server-playlist-repository';
export class StorageRequestError extends Error {
constructor(
readonly status: number,
readonly code: string,
message: string
) {
super(message);
this.name = 'StorageRequestError';
}
}
export function toStorageErrorBody(error: unknown): ServerStorageErrorBody {
if (error instanceof StorageRequestError) {
return {
message: error.message,
status: error.status,
code: error.code,
};
}
if (error instanceof PlaylistPayloadMissingError) {
return {
message: 'The server has no stored payload for this playlist.',
status: 409,
code: 'payload_missing',
};
}
if (error instanceof InvalidPlaylistDocumentError) {
return {
message: 'A playlist needs an id.',
status: 400,
code: 'invalid_body',
};
}
const type = (error as { type?: unknown } | null)?.type;
if (type === 'entity.too.large') {
return {
message: 'The request is larger than the server allows.',
status: 413,
code: 'too_large',
};
}
if (type === 'entity.parse.failed') {
return {
message: 'The request body is not valid JSON.',
status: 400,
code: 'invalid_json',
};
}
if ((error as { code?: unknown } | null)?.code === 'SQLITE_BUSY') {
return {
message: 'Storage is busy, try again.',
status: 503,
code: 'storage_busy',
};
}
return {
message: 'Storage request failed.',
status: 500,
code: 'storage_error',
};
}
/**
* Error name and code only: Drizzle and SQLite messages can carry bound
* values such as portal credentials, which must never reach a log.
*/
export function describeStorageError(error: unknown): string {
const name = error instanceof Error ? error.name : typeof error;
const code = (error as { code?: unknown } | null)?.code;
return typeof code === 'string' ? `${name} (${code})` : name;
}
export function storageErrorHandler(
log: (message: string) => void = console.error
): ErrorRequestHandler {
// Express recognizes error middleware by its four parameters.
// eslint-disable-next-line @typescript-eslint/no-unused-vars
return (error, req, res, _next) => {
const body = toStorageErrorBody(error);
if (body.status >= 500) {
const route = `${req.baseUrl}${req.route?.path ?? ''}`;
log(
`[web-backend] storage ${req.method} ${route} failed: ${describeStorageError(error)}`
);
}
res.status(body.status).json(body);
};
}
@@ -0,0 +1,312 @@
import { createWebBackendApp } from '../web-backend-app';
import {
resolvePublicHost,
StubHttpClient,
withServer,
} from '../web-backend-app.spec-helpers';
import { m3uDocument, withTempStorage } from './server-storage.spec-helpers';
import type {
ServerInstanceInfo,
ServerStorageStatusResponse,
ServerUserStateResponse,
} from '@iptvnator/shared/interfaces';
import type { ServerStorage } from './server-storage';
function app(
storage: ServerStorage | null,
extra: { accessToken?: string; maxDocumentBytes?: number } = {}
) {
return createWebBackendApp({
clientOrigins: ['http://localhost:4200'],
httpClient: new StubHttpClient(),
resolveHostname: resolvePublicHost,
storage,
...extra,
});
}
const json = (body: unknown, init: RequestInit = {}): RequestInit => ({
...init,
body: JSON.stringify(body),
headers: {
'content-type': 'application/json',
...((init.headers as Record<string, string> | undefined) ?? {}),
},
});
describe('server storage routes', () => {
it('reports no storage when it is disabled', () =>
withServer(app(null), async (baseUrl) => {
const response = await fetch(`${baseUrl}/v1/instance`);
await expect(response.json()).resolves.toEqual({
service: 'iptvnator',
apiVersion: 1,
instanceId: null,
storage: 'none',
auth: 'none',
});
expect((await fetch(`${baseUrl}/v1/playlists`)).status).toBe(404);
}));
it('reports server storage and its status', () =>
withTempStorage((storage) =>
withServer(app(storage), async (baseUrl) => {
const instance = await (
await fetch(`${baseUrl}/v1/instance`)
).json();
expect(instance).toMatchObject({
storage: 'server',
instanceId: storage.database.instanceId,
});
const status = (await (
await fetch(`${baseUrl}/v1/instance/status`)
).json()) as ServerStorageStatusResponse;
expect(status).toMatchObject({ playlists: 0 });
expect(status.databaseBytes).toBeGreaterThan(0);
})
));
it('stores a document, serves it with an ETag and answers 304', () =>
withTempStorage((storage) =>
withServer(app(storage), async (baseUrl) => {
const put = await fetch(
`${baseUrl}/v1/playlists/p1`,
json(m3uDocument('p1'), { method: 'PUT' })
);
expect(put.status).toBe(200);
const etag = put.headers.get('etag');
expect(etag).toBeTruthy();
const get = await fetch(`${baseUrl}/v1/playlists/p1`);
expect(get.headers.get('etag')).toBe(etag);
await expect(get.json()).resolves.toMatchObject({ _id: 'p1' });
const cached = await fetch(`${baseUrl}/v1/playlists/p1`, {
headers: { 'if-none-match': etag ?? '' },
});
expect(cached.status).toBe(304);
// nginx weakens the ETag of a gzipped response.
const weakened = await fetch(`${baseUrl}/v1/playlists/p1`, {
headers: { 'if-none-match': `W/${etag}` },
});
expect(weakened.status).toBe(304);
})
));
it('answers 404, not 304, for a missing playlist', () =>
withTempStorage((storage) =>
withServer(app(storage), async (baseUrl) => {
const wildcard = await fetch(`${baseUrl}/v1/playlists/nope`, {
headers: { 'if-none-match': '*' },
});
const guessed = await fetch(`${baseUrl}/v1/playlists/nope`, {
headers: {
'if-none-match': storage.revisions.etag('nope'),
},
});
expect(wildcard.status).toBe(404);
expect(guessed.status).toBe(404);
})
));
it('keeps the payload on keepPayload and refuses it for unknown ids', () =>
withTempStorage((storage) =>
withServer(app(storage), async (baseUrl) => {
await fetch(
`${baseUrl}/v1/playlists/p1`,
json(m3uDocument('p1'), { method: 'PUT' })
);
const meta: Record<string, unknown> = { ...m3uDocument('p1') };
delete meta['playlist'];
const keep = await fetch(
`${baseUrl}/v1/playlists/p1?keepPayload=1`,
json({ ...meta, title: 'Renamed' }, { method: 'PUT' })
);
const missing = await fetch(
`${baseUrl}/v1/playlists/zz?keepPayload=1`,
json({ title: 'x' }, { method: 'PUT' })
);
expect(keep.status).toBe(200);
await expect(
(await fetch(`${baseUrl}/v1/playlists/p1`)).json()
).resolves.toMatchObject({
title: 'Renamed',
playlist: m3uDocument('p1').playlist,
});
expect(missing.status).toBe(409);
await expect(missing.json()).resolves.toMatchObject({
code: 'payload_missing',
});
})
));
it('lists metadata and deletes documents with their user state', () =>
withTempStorage((storage) =>
withServer(app(storage), async (baseUrl) => {
await fetch(
`${baseUrl}/v1/playlists/p1`,
json(m3uDocument('p1'), { method: 'PUT' })
);
await fetch(
`${baseUrl}/v1/user-state/xtream-favorites/p1`,
json({ value: ['live:1'] }, { method: 'PUT' })
);
const metas = (await (
await fetch(`${baseUrl}/v1/playlists?view=meta`)
).json()) as Record<string, unknown>[];
expect(metas[0]['playlist']).toBeUndefined();
await fetch(`${baseUrl}/v1/playlists/p1`, {
method: 'DELETE',
});
expect((await fetch(`${baseUrl}/v1/playlists/p1`)).status).toBe(
404
);
const state = (await (
await fetch(`${baseUrl}/v1/user-state`)
).json()) as ServerUserStateResponse;
expect(state.entries['xtream-favorites']).toEqual({});
})
));
it('validates user-state namespaces and stores app state', () =>
withTempStorage((storage) =>
withServer(app(storage), async (baseUrl) => {
const bad = await fetch(
`${baseUrl}/v1/user-state/settings/p1`,
json({ value: 1 }, { method: 'PUT' })
);
expect(bad.status).toBe(400);
await fetch(
`${baseUrl}/v1/app-state/flag`,
json({ value: '1' }, { method: 'PUT' })
);
const read = await fetch(`${baseUrl}/v1/app-state/flag`);
expect(read.headers.get('cache-control')).toBe('no-store');
await expect(read.json()).resolves.toEqual({ value: '1' });
})
));
it('keeps browser app state away from server-owned keys', () =>
withTempStorage((storage) =>
withServer(app(storage), async (baseUrl) => {
await fetch(
`${baseUrl}/v1/app-state/server:instance-id`,
json({ value: '' }, { method: 'PUT' })
);
expect(
storage.playlists.getAppState('server:instance-id')
).toBe(storage.database.instanceId);
await expect(
(
await fetch(
`${baseUrl}/v1/app-state/server:instance-id`
)
).json()
).resolves.toEqual({ value: '' });
})
));
it('keeps the playlist when its user-state cleanup fails', () =>
withTempStorage((storage) =>
withServer(app(storage), async (baseUrl) => {
await fetch(
`${baseUrl}/v1/playlists/p1`,
json(m3uDocument('p1'), { method: 'PUT' })
);
const etag = storage.revisions.etag('p1');
jest.spyOn(
storage.userState,
'deleteForPlaylist'
).mockImplementationOnce(() => {
throw new Error('disk full');
});
const response = await fetch(`${baseUrl}/v1/playlists/p1`, {
method: 'DELETE',
});
expect(response.status).toBe(500);
expect(storage.playlists.exists('p1')).toBe(true);
expect(storage.revisions.etag('p1')).toBe(etag);
})
));
it('rejects playlist ids with surrounding whitespace', () =>
withTempStorage((storage) =>
withServer(app(storage), async (baseUrl) => {
const put = await fetch(
`${baseUrl}/v1/playlists/%20p1%20`,
json(m3uDocument('p1'), { method: 'PUT' })
);
const get = await fetch(`${baseUrl}/v1/playlists/%20p1%20`);
expect(put.status).toBe(400);
await expect(put.json()).resolves.toMatchObject({
code: 'invalid_id',
});
expect(get.status).toBe(400);
expect(storage.playlists.count()).toBe(0);
})
));
it('answers 413 above the document limit', () =>
withTempStorage((storage) =>
withServer(
app(storage, { maxDocumentBytes: 512 }),
async (baseUrl) => {
const response = await fetch(
`${baseUrl}/v1/playlists/p1`,
json(
{ ...m3uDocument('p1'), padding: 'x'.repeat(2048) },
{ method: 'PUT' }
)
);
expect(response.status).toBe(413);
}
)
));
it('requires the token for storage and proxy routes when one is set', () =>
withTempStorage((storage) =>
withServer(
app(storage, { accessToken: 's3cret' }),
async (baseUrl) => {
const instance = (await (
await fetch(`${baseUrl}/v1/instance`)
).json()) as ServerInstanceInfo;
expect(instance.auth).toBe('token');
expect(
(await fetch(`${baseUrl}/v1/playlists`)).status
).toBe(401);
expect(
(
await fetch(
`${baseUrl}/provider-targets`,
json(
{ url: 'http://example.test' },
{ method: 'POST' }
)
)
).status
).toBe(401);
expect(
(
await fetch(`${baseUrl}/v1/playlists`, {
headers: { authorization: 'Bearer s3cret' },
})
).status
).toBe(200);
}
)
));
});
@@ -0,0 +1,87 @@
import express, { Router } from 'express';
import {
isServerUserStateNamespace,
type ServerUserStateNamespace,
} from '@iptvnator/shared/interfaces';
import type { ServerStorage } from './server-storage';
import { StorageRequestError } from './storage-route-errors';
const USER_STATE_BODY_LIMIT = '8mb';
/**
* Browser app state lives under its own prefix in `app_state`, so a client
* key can never overwrite a server-owned one such as the instance id.
*/
const CLIENT_APP_STATE_PREFIX = 'client:';
export function createUserStateRouter(storage: ServerStorage): Router {
const router = Router();
const json = express.json({ limit: USER_STATE_BODY_LIMIT });
router.get('/user-state', (_req, res) => {
res.set('Cache-Control', 'no-store').json({
entries: storage.userState.all(),
});
});
router.put('/user-state/:namespace/:playlistId', json, (req, res) => {
const body = req.body as { value?: unknown } | undefined;
if (!body || typeof body !== 'object' || !('value' in body)) {
throw new StorageRequestError(
400,
'invalid_body',
'Expected { value }.'
);
}
storage.userState.put(
requireNamespace(req.params.namespace),
req.params.playlistId,
body.value
);
res.json({ success: true });
});
router.delete('/user-state/:namespace/:playlistId', (req, res) => {
storage.userState.delete(
requireNamespace(req.params.namespace),
req.params.playlistId
);
res.json({ success: true });
});
router.get('/app-state/:key', (req, res) => {
res.set('Cache-Control', 'no-store').json({
value: storage.playlists.getAppState(
CLIENT_APP_STATE_PREFIX + req.params.key
),
});
});
router.put('/app-state/:key', json, (req, res) => {
const value = (req.body as { value?: unknown } | undefined)?.value;
if (typeof value !== 'string') {
throw new StorageRequestError(
400,
'invalid_body',
'Expected { value: string }.'
);
}
storage.playlists.setAppState(
CLIENT_APP_STATE_PREFIX + req.params.key,
value
);
res.json({ success: true });
});
return router;
}
function requireNamespace(value: string): ServerUserStateNamespace {
if (!isServerUserStateNamespace(value)) {
throw new StorageRequestError(
400,
'invalid_namespace',
'Unknown user-state namespace.'
);
}
return value;
}
@@ -43,6 +43,11 @@ import {
resolveHostname,
validateProviderUrl,
} from './provider-url-policy';
import {
DEFAULT_MAX_DOCUMENT_BYTES,
mountServerStorage,
} from './storage/mount-server-storage';
import type { ServerStorage } from './storage/server-storage';
export type {
WebBackendHttpClient,
WebBackendHttpGetOptions,
@@ -67,6 +72,12 @@ export interface WebBackendAppOptions {
readonly now?: () => Date;
readonly resolveHostname?: (hostname: string) => Promise<readonly string[]>;
readonly runtimeBackendUrl?: string;
/** Optional bearer token for every non-public route. */
readonly accessToken?: string | null;
/** Largest accepted playlist document in bytes. */
readonly maxDocumentBytes?: number;
/** Server storage; `null` or absent keeps the backend stateless. */
readonly storage?: ServerStorage | null;
}
type ProviderTargetRegistry = Map<string, URL>;
@@ -129,6 +140,14 @@ export function createWebBackendApp(
);
});
mountServerStorage(app, {
corsMiddleware,
storage: options.storage ?? null,
accessToken: options.accessToken ?? null,
maxDocumentBytes:
options.maxDocumentBytes ?? DEFAULT_MAX_DOCUMENT_BYTES,
});
app.options('/provider-targets', corsMiddleware);
app.post(
'/provider-targets',
+52 -11
View File
@@ -2,18 +2,59 @@ import {
applyDefaultAutoSelectFamilyAttemptTimeout,
AUTO_SELECT_FAMILY_ATTEMPT_TIMEOUT_FLAG,
} from '@iptvnator/shared/host-health';
import { closeGracefully } from './app/graceful-shutdown';
import { createWebBackendApp } from './app/web-backend-app';
import { readServerStorageConfig } from './app/storage/storage-config';
import {
openServerStorage,
type ServerStorage,
} from './app/storage/server-storage';
import { describeStorageError } from './app/storage/storage-route-errors';
const attemptTimeout = applyDefaultAutoSelectFamilyAttemptTimeout();
if (attemptTimeout !== null) {
console.log(
`[web-backend] connection attempt timeout set to ${attemptTimeout} ms for IPv6->IPv4 fallback; pass ${AUTO_SELECT_FAMILY_ATTEMPT_TIMEOUT_FLAG} via NODE_OPTIONS to override`
);
async function start(): Promise<void> {
const attemptTimeout = applyDefaultAutoSelectFamilyAttemptTimeout();
if (attemptTimeout !== null) {
console.log(
`[web-backend] connection attempt timeout set to ${attemptTimeout} ms for IPv6->IPv4 fallback; pass ${AUTO_SELECT_FAMILY_ATTEMPT_TIMEOUT_FLAG} via NODE_OPTIONS to override`
);
}
const config = readServerStorageConfig();
let storage: ServerStorage | null = null;
if (config.enabled) {
try {
storage = await openServerStorage();
console.log(
`[web-backend] server storage at ${storage.database.path}`
);
} catch (error) {
console.error(
`[web-backend] server storage could not open: ${describeStorageError(error)}. Check that IPTVNATOR_DATA_DIR is a writable volume.`
);
process.exit(1);
}
}
const port = Number(process.env['PORT'] ?? 3000);
const app = createWebBackendApp({
storage,
accessToken: config.accessToken,
maxDocumentBytes: config.maxDocumentBytes,
});
const server = app.listen(port, () => {
console.log(
`IPTVnator web backend listening on http://localhost:${port}`
);
});
const shutdown = () => {
void closeGracefully(server).then(() => {
storage?.database.close();
process.exit(0);
});
};
process.once('SIGTERM', shutdown);
process.once('SIGINT', shutdown);
}
const port = Number(process.env['PORT'] ?? 3000);
const app = createWebBackendApp();
app.listen(port, () => {
console.log(`IPTVnator web backend listening on http://localhost:${port}`);
});
void start();
+22 -1
View File
@@ -28,6 +28,22 @@ RUN node tools/build/inject-build-commit.mjs
RUN pnpm nx build web --configuration=pwa
RUN pnpm nx build web-backend
# better-sqlite3 is a native addon. This stage is NOT pinned to
# $BUILDPLATFORM, so multi-arch builds produce it for each target platform.
# prebuild-install downloads a musl binary when one exists and otherwise
# compiles with the toolchain below.
FROM node:24-alpine AS native
RUN apk add --no-cache python3 make g++
WORKDIR /opt/native
COPY package.json ./root-package.json
RUN BETTER_SQLITE3_VERSION="$(node -p 'require("./root-package.json").dependencies["better-sqlite3"]')" \
&& npm init -y >/dev/null \
&& npm install --omit=dev --no-audit --no-fund "better-sqlite3@${BETTER_SQLITE3_VERSION}"
FROM node:24-alpine
RUN apk add --no-cache gettext nginx
@@ -37,13 +53,18 @@ WORKDIR /opt/iptvnator
ENV PORT=3000
ENV BACKEND_URL=/api
ENV CLIENT_URL=http://localhost:4333
ENV IPTVNATOR_STORAGE=server
ENV IPTVNATOR_DATA_DIR=/data
COPY --from=build /usr/src/app/dist/apps/web /usr/share/nginx/html
COPY --from=build /usr/src/app/dist/apps/web-backend ./web-backend
COPY --from=native /opt/native/node_modules ./web-backend/node_modules
COPY docker/nginx.conf /etc/nginx/http.d/default.conf.template
COPY docker/docker-entrypoint.sh /usr/local/bin/iptvnator-entrypoint
RUN chmod +x /usr/local/bin/iptvnator-entrypoint
RUN chmod +x /usr/local/bin/iptvnator-entrypoint && mkdir -p /data
VOLUME ["/data"]
EXPOSE 80
+21
View File
@@ -98,6 +98,11 @@ safe local defaults shown below.
| `PORT` | `3000` | Internal Express backend port. nginx proxy config is rendered from the template to match it at startup. |
| `IPTVNATOR_PROXY_ALLOW_PRIVATE_NETWORKS` | `0` | Set to `1` or `true` only for trusted local/LAN deployments that intentionally proxy private network IPTV or mock endpoints. |
| `NODE_EXTRA_CA_CERTS` | unset | Optional Node.js CA bundle path for providers using private certificate authorities. Mount the CA file into the container and set this path. |
| `IPTVNATOR_STORAGE` | `server` in the image | `server` opens the server storage database under `IPTVNATOR_DATA_DIR` and its `/api/v1` routes; `none` keeps the backend stateless. |
| `IPTVNATOR_DATA_DIR` | `/data` in the image | Data root; the database is `<root>/databases/iptvnator.db`. Mount a volume here. |
| `IPTVNATOR_ACCESS_TOKEN` | unset | When set, every route except `/`, `/health`, `/config.js` and `/v1/instance` requires `Authorization: Bearer <token>`, proxy routes included. The browser app cannot send a token yet: leave it unset until it can. |
| `IPTVNATOR_MAX_DOCUMENT_MB` | `64` | Largest playlist document the backend accepts. Keep the nginx `client_max_body_size` in step. |
| `IPTVNATOR_BACKEND_START_TIMEOUT` | `300` | Seconds the entrypoint waits for the backend's `/health` before giving up. Database migrations run before the backend listens. |
The web backend proxy accepts only `http` and `https` provider URLs. The PWA
first registers provider URLs through `/provider-targets`, then uses the
@@ -127,6 +132,22 @@ define a health check against `/api/health`. If nginx or the backend exits
after startup, the entrypoint exits the container so Docker Compose can apply
the `restart: unless-stopped` policy.
## Server Storage
The image opens a server storage database on the `/data` volume
(`IPTVNATOR_STORAGE=server`, `IPTVNATOR_DATA_DIR=/data`) and serves its API
under `/api/v1`. It is the backend half of keeping playlists, favorites,
history and playback positions on the server for every device; the PWA
starts using it in a follow-up change. Until then the volume holds an empty
database and each browser keeps its own data.
Provider credentials will be stored unencrypted in that database, as the
desktop app does. Do not expose an instance with server storage to the
internet without authentication in your reverse proxy. The backend already
supports `IPTVNATOR_ACCESS_TOKEN`, but the browser app cannot send the
token yet, so setting it now makes the app unusable. Back up the volume like any database: stop the container, or copy
`databases/iptvnator.db` together with its `-wal` file.
## IPv6, VPNs, And Connection Fallback
Node races IPv6 and IPv4 addresses when a provider hostname has both ("happy
+8
View File
@@ -13,9 +13,17 @@ services:
CLIENT_URL: http://localhost:4333
PORT: "3000"
IPTVNATOR_PROXY_ALLOW_PRIVATE_NETWORKS: "0"
# Server storage database on the /data volume. Set to "none" to keep
# the backend stateless.
IPTVNATOR_STORAGE: server
volumes:
- iptvnator-data:/data
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1/api/health >/dev/null || exit 1"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
volumes:
iptvnator-data: {}
+6 -2
View File
@@ -60,8 +60,12 @@ trap handle_signal INT TERM
trap cleanup EXIT
backend_ready=0
# With server storage the backend migrates its database before it listens,
# which can take a while on a large volume. The loop still exits at once if
# the backend process dies.
BACKEND_START_TIMEOUT="${IPTVNATOR_BACKEND_START_TIMEOUT:-300}"
for _ in $(seq 1 30); do
for _ in $(seq 1 "$BACKEND_START_TIMEOUT"); do
if wget -qO- "http://127.0.0.1:${PORT}/health" >/dev/null 2>&1; then
backend_ready=1
break
@@ -79,7 +83,7 @@ for _ in $(seq 1 30); do
done
if [ "$backend_ready" -ne 1 ]; then
echo "IPTVnator web backend did not become healthy on port ${PORT}."
echo "IPTVnator web backend did not become healthy on port ${PORT} within ${BACKEND_START_TIMEOUT} seconds (IPTVNATOR_BACKEND_START_TIMEOUT)."
exit 1
fi
+8
View File
@@ -13,6 +13,14 @@ server {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Playlist documents carry whole M3U payloads (default backend
# limit IPTVNATOR_MAX_DOCUMENT_MB=64).
client_max_body_size 64m;
proxy_read_timeout 120s;
gzip on;
gzip_proxied any;
gzip_types application/json;
gzip_min_length 1024;
}
location /assets/app-config.js {
+44
View File
@@ -73,6 +73,40 @@ The current self-hosted PWA uses these `apps/web-backend` routes:
- `GET /xtream?targetId=<id>&username=<u>&password=<p>&action=<action>`
- `GET /stalker?targetId=<id>&macAddress=<mac>&action=<action>`
Server storage adds versioned routes under `/v1`; the PWA does not call them
yet:
- `GET /v1/instance` (always public): `{ service, apiVersion, instanceId,
storage: 'server' | 'none', auth: 'none' | 'token' }`
- `GET /v1/instance/status`: playlist count, database size and path
- `GET /v1/playlists[?view=meta]`, `GET|PUT|DELETE /v1/playlists/:id`,
`DELETE /v1/playlists`: whole playlist documents; `GET` sends an `ETag`
and answers `304` to a matching `If-None-Match`, and
`PUT ...?keepPayload=1` keeps the stored `playlist`, `items` and
`header` fields (`409 payload_missing` without a stored row)
- `GET /v1/user-state`, `PUT|DELETE /v1/user-state/:namespace/:playlistId`:
Xtream favorites, recently viewed, collection snapshots and playback
positions, one JSON value per namespace and playlist
- `GET|PUT /v1/app-state/:key`
Server storage lives in `apps/web-backend/src/app/storage/`. With
`IPTVNATOR_STORAGE=server`, `main.ts` opens the shared database through
`initDatabase()` (path from `IPTVNATOR_DATA_DIR`), so the server has the same
schema and migrations as the desktop app and writes playlists with the
shared row codec. It adds two server-only tables: `server_user_state` for
Xtream user data, and `server_playlist_meta`, a copy of each document
without its M3U payload fields that metadata reads use instead of walking
the payload blobs (filled once on open for rows written without it). ETags
come from an in-memory revision counter; the server is the only writer of
its database. Storage errors are logged by error name and code only, because
Drizzle messages can carry bound values such as portal credentials. With
`IPTVNATOR_ACCESS_TOKEN`, every route except `/`, `/health`, `/config.js`
and `/v1/instance` requires `Authorization: Bearer <token>`, the proxy
routes included. The PWA cannot send that token yet, so the variable must
stay unset until the browser side lands. Browser app state written through
`/v1/app-state` is stored under a `client:` key prefix, apart from
server-owned keys such as the instance id.
`pnpm nx run web-backend:serve` starts the backend by hand on port 3333 with
`TSX_TSCONFIG_PATH=tsconfig.base.json`, which `tsx` needs for the
`@iptvnator/*` path aliases. `apps/web-e2e/playwright.config.ts` does not use
@@ -274,6 +308,15 @@ Default runtime values:
- `PORT=3000`
- `IPTVNATOR_PROXY_ALLOW_PRIVATE_NETWORKS=0`
- `NODE_EXTRA_CA_CERTS` unset
- `IPTVNATOR_STORAGE=server`, `IPTVNATOR_DATA_DIR=/data` (a `VOLUME`)
- `IPTVNATOR_ACCESS_TOKEN` unset, `IPTVNATOR_MAX_DOCUMENT_MB=64`
- `IPTVNATOR_BACKEND_START_TIMEOUT=300`: seconds the entrypoint waits for
`/health`; the backend migrates its database before it listens
better-sqlite3 is a native addon, so the Dockerfile builds it in a separate
`native` stage that is not pinned to `$BUILDPLATFORM`; multi-arch builds
produce it for each target platform. nginx raises `client_max_body_size` to
64 MB for `/api/` and gzips JSON responses.
When hosting behind another domain, set `CLIENT_URL` to the browser origin and
keep `BACKEND_URL=/api` unless the reverse proxy exposes the backend elsewhere.
@@ -288,6 +331,7 @@ Use the narrow validation ladder for self-hosted changes:
```bash
pnpm nx test web-backend
pnpm nx test web-backend --testPathPatterns=storage
pnpm nx test web --runTestsByPath apps/web/src/app/services/runtime-config.spec.ts
pnpm nx build web --configuration=pwa --skip-nx-cache
pnpm nx build web-backend
@@ -108,4 +108,29 @@ describe('app playlist codec', () => {
it('refuses a document without an id', () => {
expect(buildPlaylistRow({ title: 'No id' })).toBeNull();
});
it('never logs the content of a malformed stored value', () => {
const warn = jest.spyOn(console, 'warn').mockImplementation(() => {
/* silenced */
});
try {
const row = rowOf(m3uDocument);
parseAppPlaylist(
storedRow(row, '{"title":"x","password":hunter2-secret}')
);
const logged = warn.mock.calls
.flat()
.map((arg) =>
arg instanceof Error
? `${arg.name}: ${arg.message} ${arg.stack}`
: String(arg)
)
.join(' ');
expect(warn).toHaveBeenCalled();
expect(logged).not.toContain('hunter2');
} finally {
warn.mockRestore();
}
});
});
@@ -53,7 +53,12 @@ export function parseJsonValue<T>(
try {
return JSON.parse(value) as T;
} catch (error) {
console.warn('Failed to parse JSON value from DB:', error);
// The parser's message quotes the stored text, which can hold
// portal credentials; the error name is enough to diagnose.
console.warn(
'Failed to parse JSON value from DB:',
error instanceof Error ? error.name : typeof error
);
return fallback;
}
}
+1
View File
@@ -112,3 +112,4 @@ export * from './lib/xtream-connection-test';
export * from './lib/portal-request-failure.util';
export * from './lib/source-health';
export * from './lib/server-storage.contract';
@@ -0,0 +1,54 @@
/**
* HTTP contract between the PWA and the self-hosted server storage. Routes
* live under `/v1` of the web backend (`/api/v1` behind the Docker nginx).
*/
export const SERVER_STORAGE_API_VERSION = 1;
/** Xtream PWA user-data maps that the server stores per playlist id. */
export const SERVER_USER_STATE_NAMESPACES = [
'xtream-favorites',
'xtream-recent-items',
'xtream-collection-items',
'xtream-playback-positions',
] as const;
export type ServerUserStateNamespace =
(typeof SERVER_USER_STATE_NAMESPACES)[number];
export function isServerUserStateNamespace(
value: string
): value is ServerUserStateNamespace {
return (SERVER_USER_STATE_NAMESPACES as readonly string[]).includes(value);
}
/**
* Document fields that carry an M3U payload. An upsert with
* `keepPayload=1` omits them and the server keeps the stored ones.
*/
export const SERVER_PAYLOAD_FIELDS = ['playlist', 'items', 'header'] as const;
export interface ServerInstanceInfo {
readonly service: 'iptvnator';
readonly apiVersion: number;
/** Stable per database; `null` without server storage. */
readonly instanceId: string | null;
readonly storage: 'server' | 'none';
readonly auth: 'none' | 'token';
}
export interface ServerStorageStatusResponse {
readonly playlists: number;
readonly databaseBytes: number;
readonly databasePath: string;
readonly startedAt: string;
}
export interface ServerUserStateResponse {
readonly entries: Record<ServerUserStateNamespace, Record<string, unknown>>;
}
export interface ServerStorageErrorBody {
readonly message: string;
readonly status: number;
readonly code?: string;
}