mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-13 12:36:15 -08:00
feat(self-hosted): server storage API in web-backend (#1898)
* feat(interfaces): add the server storage HTTP contract Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web-backend): read server storage configuration Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web-backend): open the server database and track revisions Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web-backend): playlist and user-state repositories Playlist documents use the shared desktop row codec, so the server writes the same rows. keepPayload upserts take the stored playlist/items/header fields when the client leaves them out. Xtream user data lives in the server-only server_user_state table, one JSON value per namespace and playlist. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web-backend): storage error mapping and access token guard Storage errors map to JSON status bodies and are logged by name and code only. An optional bearer token guards every route except /, /health, /config.js and /v1/instance, provider proxy routes included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web-backend): serve the /v1 server storage API /v1/instance is always public and reports whether server storage is on. With storage, /v1/playlists serves whole documents with ETags and 304s, accepts keepPayload upserts, and deletes playlists with their user state; /v1/user-state and /v1/app-state store the rest. The proxy routes are unchanged unless an access token is configured. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web-backend): open server storage on start and close on shutdown With IPTVNATOR_STORAGE=server, main.ts opens the database before it listens and exits with a clear message when the data directory is not writable. SIGTERM and SIGINT close the server and the database. The native better-sqlite3 module stays outside the esbuild bundle. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web-backend): match weakened ETags on playlist reads nginx weakens the ETag of a gzipped response, and clients send the weak form back in If-None-Match. The route now compares tags weakly, as RFC 9110 requires for GET, so it answers 304 before reading the document. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(docker): persist server storage on a volume The image builds better-sqlite3 in a native stage per target platform, enables server storage on the /data volume, and raises the nginx body limit to 64 MB for /api with gzip for JSON. The compose file mounts a named volume and documents the optional access token. The PWA does not use the storage yet; docs say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(web-backend): type the storage route responses Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web-backend): harden server storage reads, deletes and app state - Metadata reads return the stored document without its payload fields, so fields that live only in the JSON survive a metadata round-trip. - A conditional GET checks that the playlist exists before answering 304. - Deleting a playlist and its user state commits in one transaction; the ETag moves only after the commit. - Browser app state is stored under a client: prefix and the instance id is repaired when empty, so a client write cannot stop the backend. - User-state buckets have no prototype, so ids like __proto__ round-trip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(self-hosted): keep the access token unset until the PWA sends it Add the release note for the Docker server storage volume. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web-backend): bound shutdown and reject padded playlist ids - SIGTERM closes idle connections at once and cuts the rest after five seconds, so a long provider download cannot keep the backend running. - Playlist ids with surrounding whitespace are rejected: the row codec trims ids, so storage and ETags would track different keys. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web-backend): clear optional columns on whole-document upserts The row codec maps empty values to undefined, which ON CONFLICT DO UPDATE skips, so a cleared User-Agent, Referrer or credential kept its old column value and came back on the next read. Server upserts now write NULL to every replaceable column the document no longer sets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(self-hosted): linear token parsing, no-store app state, longer start - The access guard reads the Bearer token by string scanning instead of a regular expression that CodeQL flagged as polynomial on long whitespace. - GET /v1/app-state/:key sends Cache-Control: no-store like the other state routes, so devices never reuse a stale value. - The entrypoint waits up to IPTVNATOR_BACKEND_START_TIMEOUT (300 s) for /health, because the backend migrates its database before it listens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(self-hosted): metadata reads skip payloads, redact JSON parse warnings List playlist metadata from a server-only payload-free copy of each document instead of running json_remove over every payload blob, and log only the error name when a stored JSON value fails to parse, since the parser's message quotes the stored text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web-backend): copy carried-over panel clock into playlist metadata When an upsert keeps the stored serverTimezone, build the metadata copy from the saved payload so metadata and full reads agree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
4e9795ac54
commit
b88a1c81e5
41 files changed
+2312
-16
No files matched your search
@@ -0,0 +1,7 @@
|
||||
---
|
||||
type: feature
|
||||
area: self-hosted
|
||||
issues: [305]
|
||||
---
|
||||
|
||||
The Docker image now keeps a server database on a `/data` volume, the groundwork for sharing playlists and watch progress across devices. Mount a volume to keep it; nothing changes in the browser yet.
|
||||
@@ -17,7 +17,8 @@
|
||||
"target": "node22",
|
||||
"bundle": true,
|
||||
"thirdParty": true,
|
||||
"generatePackageJson": true
|
||||
"generatePackageJson": true,
|
||||
"external": ["better-sqlite3"]
|
||||
},
|
||||
"inputs": [
|
||||
"default",
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
import { createServer, type Server } from 'node:http';
|
||||
import { AddressInfo } from 'node:net';
|
||||
import { closeGracefully } from './graceful-shutdown';
|
||||
|
||||
describe('closeGracefully', () => {
|
||||
let server: Server;
|
||||
|
||||
afterEach(() => {
|
||||
server?.closeAllConnections();
|
||||
});
|
||||
|
||||
it('stops waiting for a request that never ends after the grace period', async () => {
|
||||
server = createServer((_req, res) => {
|
||||
res.writeHead(200);
|
||||
res.write('streaming');
|
||||
// Never ends, like a provider download that keeps sending bytes.
|
||||
});
|
||||
await new Promise<void>((resolve) => server.listen(0, resolve));
|
||||
const { port } = server.address() as AddressInfo;
|
||||
const request = fetch(`http://127.0.0.1:${port}/`).catch(
|
||||
() => undefined
|
||||
);
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
|
||||
const started = Date.now();
|
||||
await closeGracefully(server, 100);
|
||||
|
||||
expect(Date.now() - started).toBeLessThan(2_000);
|
||||
expect(server.listening).toBe(false);
|
||||
await request;
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,27 @@
|
||||
import type { Server } from 'node:http';
|
||||
|
||||
/** How long open requests may finish before their connections are cut. */
|
||||
export const SHUTDOWN_GRACE_MS = 5_000;
|
||||
|
||||
/**
|
||||
* Stops accepting connections, closes idle keep-alive sockets at once and
|
||||
* cuts the remaining ones after `graceMs`: a provider download that keeps
|
||||
* sending bytes must not hold the shutdown open until Docker kills it.
|
||||
*/
|
||||
export function closeGracefully(
|
||||
server: Server,
|
||||
graceMs = SHUTDOWN_GRACE_MS
|
||||
): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
const deadline = setTimeout(
|
||||
() => server.closeAllConnections(),
|
||||
graceMs
|
||||
);
|
||||
deadline.unref();
|
||||
server.close(() => {
|
||||
clearTimeout(deadline);
|
||||
resolve();
|
||||
});
|
||||
server.closeIdleConnections();
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
import express from 'express';
|
||||
import { withServer } from '../web-backend-app.spec-helpers';
|
||||
import { createAccessGuard } from './access-guard';
|
||||
|
||||
function createApp() {
|
||||
const app = express();
|
||||
app.use(createAccessGuard('s3cret'));
|
||||
app.get('/health', (_req, res) => res.json({ ok: true }));
|
||||
app.get('/v1/instance', (_req, res) => res.json({ ok: true }));
|
||||
app.get('/v1/playlists', (_req, res) => res.json([]));
|
||||
return app;
|
||||
}
|
||||
|
||||
describe('createAccessGuard', () => {
|
||||
it('lets public routes through', () =>
|
||||
withServer(createApp(), async (baseUrl) => {
|
||||
expect((await fetch(`${baseUrl}/health`)).status).toBe(200);
|
||||
expect((await fetch(`${baseUrl}/v1/instance`)).status).toBe(200);
|
||||
}));
|
||||
|
||||
it('rejects a missing or wrong token with a JSON 401', () =>
|
||||
withServer(createApp(), async (baseUrl) => {
|
||||
const missing = await fetch(`${baseUrl}/v1/playlists`);
|
||||
const wrong = await fetch(`${baseUrl}/v1/playlists`, {
|
||||
headers: { authorization: 'Bearer nope' },
|
||||
});
|
||||
|
||||
expect(missing.status).toBe(401);
|
||||
await expect(missing.json()).resolves.toMatchObject({
|
||||
code: 'unauthorized',
|
||||
});
|
||||
expect(wrong.status).toBe(401);
|
||||
}));
|
||||
|
||||
it('accepts the right token and lets preflight through', () =>
|
||||
withServer(createApp(), async (baseUrl) => {
|
||||
const ok = await fetch(`${baseUrl}/v1/playlists`, {
|
||||
headers: { authorization: 'Bearer s3cret' },
|
||||
});
|
||||
const preflight = await fetch(`${baseUrl}/v1/playlists`, {
|
||||
method: 'OPTIONS',
|
||||
});
|
||||
|
||||
expect(ok.status).toBe(200);
|
||||
expect(preflight.status).not.toBe(401);
|
||||
}));
|
||||
|
||||
it('reads the scheme case-insensitively and trims the token', () =>
|
||||
withServer(createApp(), async (baseUrl) => {
|
||||
const ok = await fetch(`${baseUrl}/v1/playlists`, {
|
||||
headers: { authorization: 'bearer s3cret ' },
|
||||
});
|
||||
|
||||
expect(ok.status).toBe(200);
|
||||
}));
|
||||
|
||||
it('rejects a long whitespace header quickly', () =>
|
||||
withServer(createApp(), async (baseUrl) => {
|
||||
const started = Date.now();
|
||||
const response = await fetch(`${baseUrl}/v1/playlists`, {
|
||||
headers: {
|
||||
// Under Node's 16 KB header limit, so the guard sees it.
|
||||
authorization: `bearer ${' '.repeat(10_000)}x`,
|
||||
},
|
||||
});
|
||||
|
||||
expect(response.status).toBe(401);
|
||||
expect(Date.now() - started).toBeLessThan(1_000);
|
||||
}));
|
||||
});
|
||||
@@ -0,0 +1,51 @@
|
||||
import { createHash, timingSafeEqual } from 'node:crypto';
|
||||
import type { RequestHandler } from 'express';
|
||||
|
||||
/** Routes a browser must reach before it has a token. */
|
||||
const PUBLIC_PATHS = new Set(['/', '/health', '/config.js', '/v1/instance']);
|
||||
|
||||
/**
|
||||
* Optional shared-token protection for every other route, including the
|
||||
* provider proxy routes. Tokens are compared as SHA-256 digests so the
|
||||
* comparison is constant-time regardless of length.
|
||||
*/
|
||||
export function createAccessGuard(token: string): RequestHandler {
|
||||
const expected = digest(token);
|
||||
|
||||
return (req, res, next) => {
|
||||
if (req.method === 'OPTIONS' || PUBLIC_PATHS.has(req.path)) {
|
||||
next();
|
||||
return;
|
||||
}
|
||||
|
||||
const token = readBearerToken(req.get('authorization'));
|
||||
if (token && timingSafeEqual(digest(token), expected)) {
|
||||
next();
|
||||
return;
|
||||
}
|
||||
|
||||
res.status(401).json({
|
||||
message: 'This IPTVnator server requires an access token.',
|
||||
status: 401,
|
||||
code: 'unauthorized',
|
||||
});
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* `Bearer <token>`, scheme case-insensitive. Plain string scanning instead
|
||||
* of a regular expression keeps the cost linear in the header length.
|
||||
*/
|
||||
function readBearerToken(header: string | undefined): string | null {
|
||||
const value = header?.trim() ?? '';
|
||||
const separator = value.indexOf(' ');
|
||||
if (separator < 0 || value.slice(0, separator).toLowerCase() !== 'bearer') {
|
||||
return null;
|
||||
}
|
||||
const token = value.slice(separator + 1).trim();
|
||||
return token ? token : null;
|
||||
}
|
||||
|
||||
function digest(value: string): Buffer {
|
||||
return createHash('sha256').update(value).digest();
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
import { matchesEtag } from './etag-match';
|
||||
|
||||
describe('matchesEtag', () => {
|
||||
const etag = '"boot.0.1"';
|
||||
|
||||
it('matches the same strong or weakened tag', () => {
|
||||
expect(matchesEtag(etag, etag)).toBe(true);
|
||||
expect(matchesEtag(`W/${etag}`, etag)).toBe(true);
|
||||
});
|
||||
|
||||
it('matches one of several listed tags and the wildcard', () => {
|
||||
expect(matchesEtag(`"other", W/${etag}`, etag)).toBe(true);
|
||||
expect(matchesEtag('*', etag)).toBe(true);
|
||||
});
|
||||
|
||||
it('does not match a missing or different tag', () => {
|
||||
expect(matchesEtag(undefined, etag)).toBe(false);
|
||||
expect(matchesEtag('"boot.0.2"', etag)).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,22 @@
|
||||
/**
|
||||
* Weak comparison of an `If-None-Match` header with the current ETag, as
|
||||
* RFC 9110 requires for GET. nginx weakens the ETag of a gzipped response
|
||||
* (`W/"…"`), so clients send the weak form back.
|
||||
*/
|
||||
export function matchesEtag(
|
||||
ifNoneMatch: string | undefined,
|
||||
etag: string
|
||||
): boolean {
|
||||
if (!ifNoneMatch) {
|
||||
return false;
|
||||
}
|
||||
const current = stripWeak(etag);
|
||||
return ifNoneMatch
|
||||
.split(',')
|
||||
.map((tag) => tag.trim())
|
||||
.some((tag) => tag === '*' || stripWeak(tag) === current);
|
||||
}
|
||||
|
||||
function stripWeak(tag: string): string {
|
||||
return tag.startsWith('W/') ? tag.slice(2) : tag;
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
import { Router } from 'express';
|
||||
import {
|
||||
SERVER_STORAGE_API_VERSION,
|
||||
type ServerInstanceInfo,
|
||||
type ServerStorageStatusResponse,
|
||||
} from '@iptvnator/shared/interfaces';
|
||||
import type { ServerStorage } from './server-storage';
|
||||
import { StorageRequestError } from './storage-route-errors';
|
||||
|
||||
export function createInstanceRouter(deps: {
|
||||
readonly storage: ServerStorage | null;
|
||||
readonly accessTokenRequired: boolean;
|
||||
}): Router {
|
||||
const router = Router();
|
||||
|
||||
router.get('/instance', (_req, res) => {
|
||||
const body: ServerInstanceInfo = {
|
||||
service: 'iptvnator',
|
||||
apiVersion: SERVER_STORAGE_API_VERSION,
|
||||
instanceId: deps.storage?.database.instanceId ?? null,
|
||||
storage: deps.storage ? 'server' : 'none',
|
||||
auth: deps.accessTokenRequired ? 'token' : 'none',
|
||||
};
|
||||
res.set('Cache-Control', 'no-store').json(body);
|
||||
});
|
||||
|
||||
router.get('/instance/status', (_req, res) => {
|
||||
const storage = deps.storage;
|
||||
if (!storage) {
|
||||
throw new StorageRequestError(
|
||||
404,
|
||||
'storage_disabled',
|
||||
'Server storage is not enabled.'
|
||||
);
|
||||
}
|
||||
const body: ServerStorageStatusResponse = {
|
||||
playlists: storage.playlists.count(),
|
||||
databaseBytes: storage.database.sizeBytes(),
|
||||
databasePath: storage.database.path,
|
||||
startedAt: storage.startedAt.toISOString(),
|
||||
};
|
||||
res.set('Cache-Control', 'no-store').json(body);
|
||||
});
|
||||
|
||||
return router;
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
import type { Express, RequestHandler } from 'express';
|
||||
import { createAccessGuard } from './access-guard';
|
||||
import { createInstanceRouter } from './instance-router';
|
||||
import { createPlaylistsRouter } from './playlists-router';
|
||||
import type { ServerStorage } from './server-storage';
|
||||
import { storageErrorHandler } from './storage-route-errors';
|
||||
import { createUserStateRouter } from './user-state-router';
|
||||
|
||||
export const DEFAULT_MAX_DOCUMENT_BYTES = 64 * 1024 * 1024;
|
||||
|
||||
export interface ServerStorageMountOptions {
|
||||
readonly corsMiddleware: RequestHandler;
|
||||
readonly storage: ServerStorage | null;
|
||||
readonly accessToken: string | null;
|
||||
readonly maxDocumentBytes: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Mounts `/v1` and the optional token guard. Call it before the provider
|
||||
* proxy routes are registered so the guard also covers them. With a token
|
||||
* the CORS headers apply to every route, so a cross-origin 401 stays
|
||||
* readable for the PWA in development.
|
||||
*/
|
||||
export function mountServerStorage(
|
||||
app: Express,
|
||||
options: ServerStorageMountOptions
|
||||
): void {
|
||||
app.use('/v1', options.corsMiddleware);
|
||||
if (options.accessToken) {
|
||||
app.use(options.corsMiddleware);
|
||||
app.use(createAccessGuard(options.accessToken));
|
||||
}
|
||||
|
||||
app.use(
|
||||
'/v1',
|
||||
createInstanceRouter({
|
||||
storage: options.storage,
|
||||
accessTokenRequired: !!options.accessToken,
|
||||
})
|
||||
);
|
||||
if (options.storage) {
|
||||
app.use(
|
||||
'/v1',
|
||||
createPlaylistsRouter(options.storage, options.maxDocumentBytes)
|
||||
);
|
||||
app.use('/v1', createUserStateRouter(options.storage));
|
||||
}
|
||||
app.use('/v1', storageErrorHandler());
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
import { PlaylistRevisions } from './playlist-revisions';
|
||||
|
||||
describe('PlaylistRevisions', () => {
|
||||
it('keeps an ETag until the playlist is written', () => {
|
||||
const revisions = new PlaylistRevisions('boot');
|
||||
|
||||
expect(revisions.etag('a')).toBe('"boot.0.0"');
|
||||
expect(revisions.etag('a')).toBe('"boot.0.0"');
|
||||
expect(revisions.bump('a')).toBe('"boot.0.1"');
|
||||
expect(revisions.etag('a')).toBe('"boot.0.1"');
|
||||
expect(revisions.etag('b')).toBe('"boot.0.0"');
|
||||
});
|
||||
|
||||
it('invalidates every ETag on reset', () => {
|
||||
const revisions = new PlaylistRevisions('boot');
|
||||
revisions.bump('a');
|
||||
|
||||
revisions.reset();
|
||||
|
||||
expect(revisions.etag('a')).toBe('"boot.1.0"');
|
||||
expect(revisions.etag('b')).toBe('"boot.1.0"');
|
||||
});
|
||||
|
||||
it('uses a random boot id by default', () => {
|
||||
expect(new PlaylistRevisions().etag('a')).not.toBe(
|
||||
new PlaylistRevisions().etag('a')
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,32 @@
|
||||
import { randomUUID } from 'node:crypto';
|
||||
|
||||
/**
|
||||
* ETags for playlist documents. The server is the only writer of its
|
||||
* database, so an in-memory counter per playlist is exact. The boot id
|
||||
* invalidates every client cache after a restart; the generation does so
|
||||
* after "delete all".
|
||||
*/
|
||||
export class PlaylistRevisions {
|
||||
private readonly revisions = new Map<string, number>();
|
||||
private generation = 0;
|
||||
|
||||
constructor(private readonly bootId = randomUUID().slice(0, 8)) {}
|
||||
|
||||
etag(playlistId: string): string {
|
||||
const revision = this.revisions.get(playlistId) ?? 0;
|
||||
return `"${this.bootId}.${this.generation}.${revision}"`;
|
||||
}
|
||||
|
||||
bump(playlistId: string): string {
|
||||
this.revisions.set(
|
||||
playlistId,
|
||||
(this.revisions.get(playlistId) ?? 0) + 1
|
||||
);
|
||||
return this.etag(playlistId);
|
||||
}
|
||||
|
||||
reset(): void {
|
||||
this.revisions.clear();
|
||||
this.generation += 1;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,107 @@
|
||||
import express, { Router, type Response } from 'express';
|
||||
import { matchesEtag } from './etag-match';
|
||||
import {
|
||||
deleteAllPlaylistsWithState,
|
||||
deletePlaylistWithState,
|
||||
type ServerStorage,
|
||||
} from './server-storage';
|
||||
import { StorageRequestError } from './storage-route-errors';
|
||||
|
||||
export function createPlaylistsRouter(
|
||||
storage: ServerStorage,
|
||||
maxDocumentBytes: number
|
||||
): Router {
|
||||
const router = Router();
|
||||
const json = express.json({ limit: maxDocumentBytes });
|
||||
const { playlists, revisions } = storage;
|
||||
|
||||
router.get('/playlists', (req, res) => {
|
||||
res.set('Cache-Control', 'no-store').json(
|
||||
req.query['view'] === 'meta'
|
||||
? playlists.listMetas()
|
||||
: playlists.listAll()
|
||||
);
|
||||
});
|
||||
|
||||
router.get('/playlists/:id', (req, res) => {
|
||||
const id = requirePlaylistId(req.params.id);
|
||||
// A cheap existence check first: a tag must never confirm a
|
||||
// representation that does not exist.
|
||||
if (!playlists.exists(id)) {
|
||||
throw new StorageRequestError(
|
||||
404,
|
||||
'not_found',
|
||||
'Playlist not found.'
|
||||
);
|
||||
}
|
||||
const etag = revisions.etag(id);
|
||||
withEtag(res, etag);
|
||||
if (matchesEtag(req.get('if-none-match'), etag)) {
|
||||
res.status(304).end();
|
||||
return;
|
||||
}
|
||||
const document = playlists.get(id);
|
||||
if (!document) {
|
||||
throw new StorageRequestError(
|
||||
404,
|
||||
'not_found',
|
||||
'Playlist not found.'
|
||||
);
|
||||
}
|
||||
res.json(document);
|
||||
});
|
||||
|
||||
router.put('/playlists/:id', json, (req, res) => {
|
||||
const id = requirePlaylistId(req.params.id);
|
||||
playlists.upsert(
|
||||
{ ...requireDocument(req.body), _id: id },
|
||||
{ keepPayload: req.query['keepPayload'] === '1' }
|
||||
);
|
||||
withEtag(res, revisions.bump(id)).json({ success: true });
|
||||
});
|
||||
|
||||
router.delete('/playlists/:id', (req, res) => {
|
||||
deletePlaylistWithState(storage, requirePlaylistId(req.params.id));
|
||||
res.json({ success: true });
|
||||
});
|
||||
|
||||
router.delete('/playlists', (_req, res) => {
|
||||
deleteAllPlaylistsWithState(storage);
|
||||
res.json({ success: true });
|
||||
});
|
||||
|
||||
return router;
|
||||
}
|
||||
|
||||
function withEtag(res: Response, etag: string): Response {
|
||||
return res
|
||||
.set('ETag', etag)
|
||||
.set('Access-Control-Expose-Headers', 'ETag')
|
||||
.set('Cache-Control', 'no-cache');
|
||||
}
|
||||
|
||||
/**
|
||||
* The row codec trims ids, so an id with surrounding whitespace would be
|
||||
* stored under a different key than the one its ETag tracks.
|
||||
*/
|
||||
function requirePlaylistId(value: string): string {
|
||||
if (!value || value !== value.trim()) {
|
||||
throw new StorageRequestError(
|
||||
400,
|
||||
'invalid_id',
|
||||
'A playlist id cannot be empty or start or end with whitespace.'
|
||||
);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function requireDocument(value: unknown): Record<string, unknown> {
|
||||
if (!value || typeof value !== 'object' || Array.isArray(value)) {
|
||||
throw new StorageRequestError(
|
||||
400,
|
||||
'invalid_body',
|
||||
'Expected a playlist object.'
|
||||
);
|
||||
}
|
||||
return value as Record<string, unknown>;
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
import { mkdtempSync, rmSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { sql } from 'drizzle-orm';
|
||||
import { IPTVNATOR_DATA_DIR_ENV } from '@iptvnator/shared/database';
|
||||
import { openServerDatabase } from './server-database';
|
||||
import { ServerPlaylistRepository } from './server-playlist-repository';
|
||||
import { m3uDocument } from './server-storage.spec-helpers';
|
||||
|
||||
describe('openServerDatabase', () => {
|
||||
let dataDir: string;
|
||||
let previous: string | undefined;
|
||||
|
||||
beforeEach(() => {
|
||||
dataDir = mkdtempSync(join(tmpdir(), 'iptvnator-server-db-'));
|
||||
previous = process.env[IPTVNATOR_DATA_DIR_ENV];
|
||||
process.env[IPTVNATOR_DATA_DIR_ENV] = dataDir;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (previous === undefined) {
|
||||
delete process.env[IPTVNATOR_DATA_DIR_ENV];
|
||||
} else {
|
||||
process.env[IPTVNATOR_DATA_DIR_ENV] = previous;
|
||||
}
|
||||
rmSync(dataDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('keeps the instance id across restarts', async () => {
|
||||
const first = await openServerDatabase();
|
||||
const instanceId = first.instanceId;
|
||||
first.close();
|
||||
|
||||
const second = await openServerDatabase();
|
||||
expect(second.instanceId).toBe(instanceId);
|
||||
second.close();
|
||||
});
|
||||
|
||||
it('still opens when the stored instance id is empty', async () => {
|
||||
const first = await openServerDatabase();
|
||||
new ServerPlaylistRepository(first.db).setAppState(
|
||||
'server:instance-id',
|
||||
''
|
||||
);
|
||||
first.close();
|
||||
|
||||
const second = await openServerDatabase();
|
||||
expect(second.instanceId).toMatch(/^[0-9a-f-]{36}$/);
|
||||
second.close();
|
||||
});
|
||||
|
||||
it('runs several writes in one transaction', async () => {
|
||||
const database = await openServerDatabase();
|
||||
const playlists = new ServerPlaylistRepository(database.db);
|
||||
|
||||
expect(() =>
|
||||
database.transaction(() => {
|
||||
playlists.setAppState('a', '1');
|
||||
throw new Error('fail after the first write');
|
||||
})
|
||||
).toThrow('fail after the first write');
|
||||
expect(playlists.getAppState('a')).toBeNull();
|
||||
database.close();
|
||||
});
|
||||
|
||||
it('adds metadata for playlists written without it', async () => {
|
||||
const first = await openServerDatabase();
|
||||
new ServerPlaylistRepository(first.db).upsert(
|
||||
{ ...m3uDocument('p1'), filename: 'family.m3u' },
|
||||
{ keepPayload: false }
|
||||
);
|
||||
new ServerPlaylistRepository(first.db).upsert(m3uDocument('p2'), {
|
||||
keepPayload: false,
|
||||
});
|
||||
// As in a database the desktop app wrote: rows, no server copy.
|
||||
first.db.run(sql`DELETE FROM server_playlist_meta`);
|
||||
first.db.run(
|
||||
sql`UPDATE playlists SET payload = 'not json' WHERE id = 'p2'`
|
||||
);
|
||||
first.close();
|
||||
|
||||
const second = await openServerDatabase();
|
||||
const metas = new ServerPlaylistRepository(second.db).listMetas();
|
||||
expect(metas).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({ _id: 'p1', filename: 'family.m3u' }),
|
||||
expect.objectContaining({ _id: 'p2', title: 'Family' }),
|
||||
])
|
||||
);
|
||||
expect(metas[0]['playlist']).toBeUndefined();
|
||||
second.close();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,82 @@
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { statSync } from 'node:fs';
|
||||
import { eq, sql } from 'drizzle-orm';
|
||||
import {
|
||||
appState,
|
||||
closeDatabase,
|
||||
getDatabasePath,
|
||||
initDatabase,
|
||||
type DatabaseInstance,
|
||||
} from '@iptvnator/shared/database';
|
||||
import {
|
||||
SERVER_PLAYLIST_META_BACKFILL_SQL,
|
||||
SERVER_PLAYLIST_META_TABLE_SQL,
|
||||
} from './server-playlist-meta.table';
|
||||
import { SERVER_USER_STATE_TABLE_SQL } from './server-user-state.table';
|
||||
|
||||
const INSTANCE_ID_KEY = 'server:instance-id';
|
||||
|
||||
export interface ServerDatabase {
|
||||
readonly db: DatabaseInstance;
|
||||
readonly instanceId: string;
|
||||
readonly path: string;
|
||||
sizeBytes(): number;
|
||||
/** Runs `work` in one SQLite transaction on the single connection. */
|
||||
transaction<T>(work: () => T): T;
|
||||
close(): void;
|
||||
}
|
||||
|
||||
/**
|
||||
* Opens the shared IPTVnator database (full schema and migrations, the
|
||||
* same as the desktop app) and adds the server-only tables. The path comes
|
||||
* from `IPTVNATOR_DATA_DIR`.
|
||||
*/
|
||||
export async function openServerDatabase(): Promise<ServerDatabase> {
|
||||
const db = await initDatabase();
|
||||
db.run(sql.raw(SERVER_USER_STATE_TABLE_SQL));
|
||||
db.run(sql.raw(SERVER_PLAYLIST_META_TABLE_SQL));
|
||||
db.run(sql.raw(SERVER_PLAYLIST_META_BACKFILL_SQL));
|
||||
const path = getDatabasePath();
|
||||
|
||||
return {
|
||||
db,
|
||||
instanceId: ensureInstanceId(db),
|
||||
path,
|
||||
sizeBytes: () => fileSize(path) + fileSize(`${path}-wal`),
|
||||
// better-sqlite3 runs every statement synchronously on one
|
||||
// connection, so writes made inside `work` belong to this
|
||||
// transaction even when they go through other repositories.
|
||||
transaction: (work) => db.transaction(() => work()),
|
||||
close: () => closeDatabase(),
|
||||
};
|
||||
}
|
||||
|
||||
function ensureInstanceId(db: DatabaseInstance): string {
|
||||
const existing = db
|
||||
.select({ value: appState.value })
|
||||
.from(appState)
|
||||
.where(eq(appState.key, INSTANCE_ID_KEY))
|
||||
.get();
|
||||
if (existing?.value) {
|
||||
return existing.value;
|
||||
}
|
||||
|
||||
const instanceId = randomUUID();
|
||||
const updatedAt = new Date().toISOString();
|
||||
db.insert(appState)
|
||||
.values({ key: INSTANCE_ID_KEY, value: instanceId, updatedAt })
|
||||
.onConflictDoUpdate({
|
||||
target: appState.key,
|
||||
set: { value: instanceId, updatedAt },
|
||||
})
|
||||
.run();
|
||||
return instanceId;
|
||||
}
|
||||
|
||||
function fileSize(path: string): number {
|
||||
try {
|
||||
return statSync(path).size;
|
||||
} catch {
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
import { SERVER_PAYLOAD_FIELDS } from '@iptvnator/shared/interfaces';
|
||||
import { sql, type SQL, type SQLWrapper } from 'drizzle-orm';
|
||||
import { sqliteTable, text } from 'drizzle-orm/sqlite-core';
|
||||
|
||||
/**
|
||||
* Server-only table: each playlist document without its M3U payload
|
||||
* fields, written with every upsert. Metadata reads take the JSON-only
|
||||
* fields (filename, hidden groups, Stalker identity) from here, so they
|
||||
* never walk the payload blobs. Created by `openServerDatabase`, never by
|
||||
* the desktop app.
|
||||
*/
|
||||
export const SERVER_PLAYLIST_META_TABLE_SQL = `CREATE TABLE IF NOT EXISTS server_playlist_meta (
|
||||
playlist_id TEXT PRIMARY KEY NOT NULL,
|
||||
document TEXT
|
||||
)`;
|
||||
|
||||
const PAYLOAD_PATHS = SERVER_PAYLOAD_FIELDS.map((field) => `'$.${field}'`).join(
|
||||
', '
|
||||
);
|
||||
|
||||
/**
|
||||
* Fills the table for playlists written without it, for example a
|
||||
* database copied from the desktop app. Reads each such payload once;
|
||||
* later starts find nothing to do.
|
||||
*/
|
||||
export const SERVER_PLAYLIST_META_BACKFILL_SQL = `INSERT INTO server_playlist_meta (playlist_id, document)
|
||||
SELECT id, CASE
|
||||
WHEN json_valid(payload) THEN json_remove(payload, ${PAYLOAD_PATHS})
|
||||
ELSE NULL
|
||||
END
|
||||
FROM playlists
|
||||
WHERE id NOT IN (SELECT playlist_id FROM server_playlist_meta)`;
|
||||
|
||||
/** A stored payload without its M3U fields, or NULL when it is not JSON. */
|
||||
export function payloadMetadata(payload: SQLWrapper): SQL<string | null> {
|
||||
return sql<string | null>`CASE
|
||||
WHEN json_valid(${payload})
|
||||
THEN json_remove(${payload}, ${sql.raw(PAYLOAD_PATHS)})
|
||||
ELSE NULL
|
||||
END`;
|
||||
}
|
||||
|
||||
export const serverPlaylistMeta = sqliteTable('server_playlist_meta', {
|
||||
playlistId: text('playlist_id').primaryKey().notNull(),
|
||||
document: text('document'),
|
||||
});
|
||||
|
||||
/** The document as metadata reads see it: no payload fields. */
|
||||
export function metadataDocument(
|
||||
document: Readonly<Record<string, unknown>>
|
||||
): string {
|
||||
const metadata: Record<string, unknown> = { ...document };
|
||||
for (const field of SERVER_PAYLOAD_FIELDS) {
|
||||
delete metadata[field];
|
||||
}
|
||||
return JSON.stringify(metadata);
|
||||
}
|
||||
@@ -0,0 +1,219 @@
|
||||
import { sql } from 'drizzle-orm';
|
||||
import { PlaylistPayloadMissingError } from './server-playlist-repository';
|
||||
import { m3uDocument, withTempStorage } from './server-storage.spec-helpers';
|
||||
|
||||
describe('ServerPlaylistRepository', () => {
|
||||
it('stores and reads a whole playlist document', () =>
|
||||
withTempStorage(({ playlists }) => {
|
||||
playlists.upsert(m3uDocument('p1'), { keepPayload: false });
|
||||
|
||||
expect(playlists.get('p1')).toMatchObject({
|
||||
_id: 'p1',
|
||||
title: 'Family',
|
||||
playlist: m3uDocument('p1').playlist,
|
||||
});
|
||||
expect(playlists.count()).toBe(1);
|
||||
expect(playlists.get('missing')).toBeNull();
|
||||
}));
|
||||
|
||||
it('lists metadata without the payload', () =>
|
||||
withTempStorage(({ playlists }) => {
|
||||
playlists.upsert(m3uDocument('p1'), { keepPayload: false });
|
||||
|
||||
const [meta] = playlists.listMetas();
|
||||
|
||||
expect(meta).toMatchObject({ _id: 'p1', title: 'Family' });
|
||||
expect(meta['playlist']).toBeUndefined();
|
||||
expect(playlists.listAll()[0]['playlist']).toEqual(
|
||||
m3uDocument('p1').playlist
|
||||
);
|
||||
}));
|
||||
|
||||
it('keeps the stored payload when asked to', () =>
|
||||
withTempStorage(({ playlists }) => {
|
||||
playlists.upsert(m3uDocument('p1'), { keepPayload: false });
|
||||
const meta: Record<string, unknown> = { ...m3uDocument('p1') };
|
||||
delete meta['playlist'];
|
||||
|
||||
playlists.upsert(
|
||||
{
|
||||
...meta,
|
||||
title: 'Renamed',
|
||||
favorites: ['https://example.test/a'],
|
||||
},
|
||||
{ keepPayload: true }
|
||||
);
|
||||
|
||||
expect(playlists.get('p1')).toMatchObject({
|
||||
title: 'Renamed',
|
||||
favorites: ['https://example.test/a'],
|
||||
playlist: m3uDocument('p1').playlist,
|
||||
});
|
||||
}));
|
||||
|
||||
it('keeps payload-only metadata in metadata reads and round-trips', () =>
|
||||
withTempStorage(({ playlists }) => {
|
||||
playlists.upsert(
|
||||
{
|
||||
...m3uDocument('p1'),
|
||||
filename: 'family.m3u',
|
||||
hiddenGroupTitles: ['Adult'],
|
||||
stalkerSerialNumber: 'SN-1',
|
||||
},
|
||||
{ keepPayload: false }
|
||||
);
|
||||
|
||||
const [meta] = playlists.listMetas();
|
||||
expect(meta).toMatchObject({
|
||||
filename: 'family.m3u',
|
||||
hiddenGroupTitles: ['Adult'],
|
||||
stalkerSerialNumber: 'SN-1',
|
||||
});
|
||||
expect(meta['playlist']).toBeUndefined();
|
||||
|
||||
playlists.upsert(
|
||||
{ ...meta, title: 'Renamed' },
|
||||
{ keepPayload: true }
|
||||
);
|
||||
|
||||
expect(playlists.get('p1')).toMatchObject({
|
||||
title: 'Renamed',
|
||||
filename: 'family.m3u',
|
||||
hiddenGroupTitles: ['Adult'],
|
||||
stalkerSerialNumber: 'SN-1',
|
||||
playlist: m3uDocument('p1').playlist,
|
||||
});
|
||||
}));
|
||||
|
||||
it('serves complete metadata without reading the payload', () =>
|
||||
withTempStorage(({ database, playlists }) => {
|
||||
playlists.upsert(
|
||||
{
|
||||
...m3uDocument('p1'),
|
||||
filename: 'family.m3u',
|
||||
hiddenGroupTitles: ['Adult'],
|
||||
},
|
||||
{ keepPayload: false }
|
||||
);
|
||||
// A metadata read must not depend on the payload blob at all.
|
||||
database.db.run(
|
||||
sql`UPDATE playlists SET payload = 'not json' WHERE id = 'p1'`
|
||||
);
|
||||
|
||||
expect(playlists.listMetas()[0]).toMatchObject({
|
||||
_id: 'p1',
|
||||
title: 'Family',
|
||||
filename: 'family.m3u',
|
||||
hiddenGroupTitles: ['Adult'],
|
||||
});
|
||||
}));
|
||||
|
||||
it('keeps a carried-over panel clock in metadata reads', () =>
|
||||
withTempStorage(({ playlists }) => {
|
||||
const panel = {
|
||||
_id: 'x1',
|
||||
title: 'Panel',
|
||||
serverUrl: 'http://panel.test',
|
||||
username: 'user',
|
||||
password: 'secret',
|
||||
importDate: '2026-10-01T00:00:00.000Z',
|
||||
};
|
||||
playlists.upsert(
|
||||
{ ...panel, serverTimezone: 'Europe/Berlin' },
|
||||
{ keepPayload: false }
|
||||
);
|
||||
|
||||
// An older snapshot without the clock, same panel account.
|
||||
playlists.upsert(
|
||||
{ ...panel, title: 'Renamed' },
|
||||
{ keepPayload: false }
|
||||
);
|
||||
const kept = { title: 'Renamed', serverTimezone: 'Europe/Berlin' };
|
||||
expect(playlists.get('x1')).toMatchObject(kept);
|
||||
expect(playlists.listMetas()[0]).toMatchObject(kept);
|
||||
|
||||
// A different account drops the clock in both reads.
|
||||
playlists.upsert(
|
||||
{ ...panel, password: 'changed' },
|
||||
{ keepPayload: false }
|
||||
);
|
||||
expect(playlists.get('x1')?.['serverTimezone']).toBeUndefined();
|
||||
expect(playlists.listMetas()[0]['serverTimezone']).toBeUndefined();
|
||||
}));
|
||||
|
||||
it('drops the metadata copy with its playlist', () =>
|
||||
withTempStorage(({ playlists }) => {
|
||||
playlists.upsert(m3uDocument('p1'), { keepPayload: false });
|
||||
playlists.upsert(m3uDocument('p2'), { keepPayload: false });
|
||||
|
||||
playlists.delete('p1');
|
||||
expect(playlists.listMetas().map((p) => p['_id'])).toEqual(['p2']);
|
||||
playlists.deleteAll();
|
||||
expect(playlists.listMetas()).toEqual([]);
|
||||
}));
|
||||
|
||||
it('clears optional columns that a replacement leaves empty', () =>
|
||||
withTempStorage(({ playlists }) => {
|
||||
playlists.upsert(
|
||||
{
|
||||
...m3uDocument('p1'),
|
||||
userAgent: 'UA/1',
|
||||
referrer: 'https://ref.test',
|
||||
origin: 'https://origin.test',
|
||||
},
|
||||
{ keepPayload: false }
|
||||
);
|
||||
|
||||
playlists.upsert(
|
||||
{
|
||||
...m3uDocument('p1'),
|
||||
userAgent: '',
|
||||
referrer: '',
|
||||
},
|
||||
{ keepPayload: false }
|
||||
);
|
||||
|
||||
const stored = playlists.get('p1');
|
||||
expect(stored?.['userAgent']).toBeUndefined();
|
||||
expect(stored?.['referrer']).toBeUndefined();
|
||||
expect(stored?.['origin']).toBeUndefined();
|
||||
expect(playlists.listMetas()[0]['userAgent']).toBeUndefined();
|
||||
}));
|
||||
|
||||
it('tells whether a playlist exists', () =>
|
||||
withTempStorage(({ playlists }) => {
|
||||
playlists.upsert(m3uDocument('p1'), { keepPayload: false });
|
||||
|
||||
expect(playlists.exists('p1')).toBe(true);
|
||||
expect(playlists.exists('nope')).toBe(false);
|
||||
}));
|
||||
|
||||
it('refuses keepPayload for an unknown playlist', () =>
|
||||
withTempStorage(({ playlists }) => {
|
||||
expect(() =>
|
||||
playlists.upsert(
|
||||
{ _id: 'nope', title: 'x' },
|
||||
{ keepPayload: true }
|
||||
)
|
||||
).toThrow(PlaylistPayloadMissingError);
|
||||
}));
|
||||
|
||||
it('deletes one or all playlists', () =>
|
||||
withTempStorage(({ playlists }) => {
|
||||
playlists.upsert(m3uDocument('p1'), { keepPayload: false });
|
||||
playlists.upsert(m3uDocument('p2'), { keepPayload: false });
|
||||
|
||||
playlists.delete('p1');
|
||||
expect(playlists.listMetas().map((p) => p['_id'])).toEqual(['p2']);
|
||||
|
||||
playlists.deleteAll();
|
||||
expect(playlists.count()).toBe(0);
|
||||
}));
|
||||
|
||||
it('reads and writes app state', () =>
|
||||
withTempStorage(({ playlists }) => {
|
||||
expect(playlists.getAppState('flag')).toBeNull();
|
||||
playlists.setAppState('flag', '1');
|
||||
expect(playlists.getAppState('flag')).toBe('1');
|
||||
}));
|
||||
});
|
||||
@@ -0,0 +1,255 @@
|
||||
import { eq, getTableColumns } from 'drizzle-orm';
|
||||
import type { DatabaseInstance } from '@iptvnator/shared/database';
|
||||
import {
|
||||
buildPlaylistRow,
|
||||
parseAppPlaylist,
|
||||
playlistConflictUpdate,
|
||||
} from '@iptvnator/shared/database/app-playlist';
|
||||
import { appState, playlists } from '@iptvnator/shared/database/schema';
|
||||
import type {
|
||||
NewPlaylist,
|
||||
Playlist as PlaylistRow,
|
||||
} from '@iptvnator/shared/database/schema';
|
||||
import { SERVER_PAYLOAD_FIELDS } from '@iptvnator/shared/interfaces';
|
||||
import {
|
||||
metadataDocument,
|
||||
payloadMetadata,
|
||||
serverPlaylistMeta,
|
||||
} from './server-playlist-meta.table';
|
||||
|
||||
export type PlaylistDocument = Record<string, unknown>;
|
||||
|
||||
export class PlaylistPayloadMissingError extends Error {
|
||||
constructor() {
|
||||
super('No stored payload to keep');
|
||||
this.name = 'PlaylistPayloadMissingError';
|
||||
}
|
||||
}
|
||||
|
||||
export class InvalidPlaylistDocumentError extends Error {
|
||||
constructor() {
|
||||
super('A playlist document needs an id');
|
||||
this.name = 'InvalidPlaylistDocumentError';
|
||||
}
|
||||
}
|
||||
|
||||
/** Every column except the M3U payload, for metadata reads. */
|
||||
function metadataColumns() {
|
||||
const { payload, ...columns } = getTableColumns(playlists);
|
||||
void payload;
|
||||
return columns;
|
||||
}
|
||||
|
||||
const metaColumns = metadataColumns();
|
||||
|
||||
/**
|
||||
* Columns a whole-document upsert must rewrite even when the document no
|
||||
* longer has a value: the codec maps empty values to `undefined`, which an
|
||||
* `ON CONFLICT DO UPDATE` skips, so a cleared field would keep its old
|
||||
* column value and `parseAppPlaylist` would bring it back.
|
||||
*/
|
||||
const REPLACEABLE_COLUMNS = [
|
||||
'serverUrl',
|
||||
'username',
|
||||
'password',
|
||||
'lastUpdated',
|
||||
'userAgent',
|
||||
'origin',
|
||||
'referrer',
|
||||
'filePath',
|
||||
'epgUrls',
|
||||
'detectedEpgUrls',
|
||||
'manualEpgUrls',
|
||||
'disabledEpgUrls',
|
||||
'macAddress',
|
||||
'url',
|
||||
'portalUrl',
|
||||
'count',
|
||||
'importDate',
|
||||
'updateDate',
|
||||
'position',
|
||||
'favorites',
|
||||
'recentlyViewed',
|
||||
] as const satisfies readonly (keyof NewPlaylist)[];
|
||||
|
||||
function clearedColumns(row: NewPlaylist): Partial<NewPlaylist> {
|
||||
const cleared: Partial<Record<keyof NewPlaylist, null>> = {};
|
||||
for (const column of REPLACEABLE_COLUMNS) {
|
||||
if (row[column] === undefined) {
|
||||
cleared[column] = null;
|
||||
}
|
||||
}
|
||||
return cleared as Partial<NewPlaylist>;
|
||||
}
|
||||
|
||||
/** Playlist documents in the shared `playlists` table, same rows as desktop. */
|
||||
export class ServerPlaylistRepository {
|
||||
constructor(private readonly db: DatabaseInstance) {}
|
||||
|
||||
count(): number {
|
||||
return this.db.select({ id: playlists.id }).from(playlists).all()
|
||||
.length;
|
||||
}
|
||||
|
||||
/**
|
||||
* Columns plus the payload-free copy of each document, so metadata
|
||||
* reads keep fields that live only in the JSON (filename, hidden
|
||||
* groups, Stalker identity) and a metadata round-trip with keepPayload
|
||||
* loses nothing, without reading a payload blob.
|
||||
*/
|
||||
listMetas(): PlaylistDocument[] {
|
||||
return this.db
|
||||
.select({ ...metaColumns, payload: serverPlaylistMeta.document })
|
||||
.from(playlists)
|
||||
.leftJoin(
|
||||
serverPlaylistMeta,
|
||||
eq(serverPlaylistMeta.playlistId, playlists.id)
|
||||
)
|
||||
.all()
|
||||
.map((row) => parseAppPlaylist(row as PlaylistRow));
|
||||
}
|
||||
|
||||
exists(playlistId: string): boolean {
|
||||
return !!this.db
|
||||
.select({ id: playlists.id })
|
||||
.from(playlists)
|
||||
.where(eq(playlists.id, playlistId))
|
||||
.get();
|
||||
}
|
||||
|
||||
listAll(): PlaylistDocument[] {
|
||||
return this.db
|
||||
.select()
|
||||
.from(playlists)
|
||||
.all()
|
||||
.map((row) => parseAppPlaylist(row));
|
||||
}
|
||||
|
||||
get(playlistId: string): PlaylistDocument | null {
|
||||
const row = this.db
|
||||
.select()
|
||||
.from(playlists)
|
||||
.where(eq(playlists.id, playlistId))
|
||||
.get();
|
||||
return row ? parseAppPlaylist(row) : null;
|
||||
}
|
||||
|
||||
upsert(
|
||||
document: PlaylistDocument,
|
||||
options: { readonly keepPayload: boolean }
|
||||
): void {
|
||||
const next = options.keepPayload
|
||||
? this.withStoredPayload(document)
|
||||
: document;
|
||||
const row = buildPlaylistRow(next);
|
||||
if (!row) {
|
||||
throw new InvalidPlaylistDocumentError();
|
||||
}
|
||||
|
||||
const update = {
|
||||
...playlistConflictUpdate(row, next),
|
||||
...clearedColumns(row),
|
||||
};
|
||||
this.db.transaction((tx) => {
|
||||
tx.insert(playlists)
|
||||
.values(row)
|
||||
.onConflictDoUpdate({ target: playlists.id, set: update })
|
||||
.run();
|
||||
// The conflict update may carry stored fields (the panel clock)
|
||||
// into the payload; the copy must then match what was saved.
|
||||
const metadata =
|
||||
update.payload === row.payload
|
||||
? metadataDocument(next)
|
||||
: (tx
|
||||
.select({
|
||||
document: payloadMetadata(playlists.payload),
|
||||
})
|
||||
.from(playlists)
|
||||
.where(eq(playlists.id, row.id))
|
||||
.get()?.document ?? null);
|
||||
tx.insert(serverPlaylistMeta)
|
||||
.values({ playlistId: row.id, document: metadata })
|
||||
.onConflictDoUpdate({
|
||||
target: serverPlaylistMeta.playlistId,
|
||||
set: { document: metadata },
|
||||
})
|
||||
.run();
|
||||
});
|
||||
}
|
||||
|
||||
delete(playlistId: string): void {
|
||||
this.db.transaction((tx) => {
|
||||
tx.delete(playlists).where(eq(playlists.id, playlistId)).run();
|
||||
tx.delete(serverPlaylistMeta)
|
||||
.where(eq(serverPlaylistMeta.playlistId, playlistId))
|
||||
.run();
|
||||
});
|
||||
}
|
||||
|
||||
deleteAll(): void {
|
||||
this.db.transaction((tx) => {
|
||||
tx.delete(playlists).run();
|
||||
tx.delete(serverPlaylistMeta).run();
|
||||
});
|
||||
}
|
||||
|
||||
getAppState(key: string): string | null {
|
||||
return (
|
||||
this.db
|
||||
.select({ value: appState.value })
|
||||
.from(appState)
|
||||
.where(eq(appState.key, key))
|
||||
.get()?.value ?? null
|
||||
);
|
||||
}
|
||||
|
||||
setAppState(key: string, value: string): void {
|
||||
const updatedAt = new Date().toISOString();
|
||||
this.db
|
||||
.insert(appState)
|
||||
.values({ key, value, updatedAt })
|
||||
.onConflictDoUpdate({
|
||||
target: appState.key,
|
||||
set: { value, updatedAt },
|
||||
})
|
||||
.run();
|
||||
}
|
||||
|
||||
/** Fills the payload fields the client left out from the stored row. */
|
||||
private withStoredPayload(document: PlaylistDocument): PlaylistDocument {
|
||||
const id = String(document['_id'] ?? '');
|
||||
const stored = this.db
|
||||
.select({ payload: playlists.payload })
|
||||
.from(playlists)
|
||||
.where(eq(playlists.id, id))
|
||||
.get();
|
||||
const parsed = parseStoredPayload(stored?.payload);
|
||||
if (!parsed) {
|
||||
throw new PlaylistPayloadMissingError();
|
||||
}
|
||||
|
||||
const merged: PlaylistDocument = { ...document };
|
||||
for (const field of SERVER_PAYLOAD_FIELDS) {
|
||||
if (!(field in merged) && field in parsed) {
|
||||
merged[field] = parsed[field];
|
||||
}
|
||||
}
|
||||
return merged;
|
||||
}
|
||||
}
|
||||
|
||||
function parseStoredPayload(
|
||||
payload: string | null | undefined
|
||||
): PlaylistDocument | null {
|
||||
if (!payload) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
const parsed: unknown = JSON.parse(payload);
|
||||
return parsed && typeof parsed === 'object'
|
||||
? (parsed as PlaylistDocument)
|
||||
: null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
import { mkdtempSync, rmSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { IPTVNATOR_DATA_DIR_ENV } from '@iptvnator/shared/database';
|
||||
import { openServerStorage, type ServerStorage } from './server-storage';
|
||||
|
||||
/** Real SQLite in a throwaway data directory, closed and removed afterwards. */
|
||||
export async function withTempStorage<T>(
|
||||
run: (storage: ServerStorage) => Promise<T> | T
|
||||
): Promise<T> {
|
||||
const dataDir = mkdtempSync(join(tmpdir(), 'iptvnator-server-storage-'));
|
||||
const previous = process.env[IPTVNATOR_DATA_DIR_ENV];
|
||||
process.env[IPTVNATOR_DATA_DIR_ENV] = dataDir;
|
||||
const storage = await openServerStorage();
|
||||
|
||||
try {
|
||||
return await run(storage);
|
||||
} finally {
|
||||
storage.database.close();
|
||||
if (previous === undefined) {
|
||||
delete process.env[IPTVNATOR_DATA_DIR_ENV];
|
||||
} else {
|
||||
process.env[IPTVNATOR_DATA_DIR_ENV] = previous;
|
||||
}
|
||||
rmSync(dataDir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
export const m3uDocument = (id: string, title = 'Family') => ({
|
||||
_id: id,
|
||||
title,
|
||||
url: `https://example.test/${id}.m3u`,
|
||||
count: 1,
|
||||
importDate: '2026-10-01T00:00:00.000Z',
|
||||
lastUsage: '2026-10-01T00:00:00.000Z',
|
||||
favorites: [],
|
||||
recentlyViewed: [],
|
||||
autoRefresh: false,
|
||||
playlist: {
|
||||
header: { raw: '#EXTM3U' },
|
||||
items: [
|
||||
{ id: 'a', url: 'https://example.test/a', raw: '#EXTINF:-1,A' },
|
||||
],
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,44 @@
|
||||
import { PlaylistRevisions } from './playlist-revisions';
|
||||
import { openServerDatabase, type ServerDatabase } from './server-database';
|
||||
import { ServerPlaylistRepository } from './server-playlist-repository';
|
||||
import { ServerUserStateRepository } from './server-user-state-repository';
|
||||
|
||||
export interface ServerStorage {
|
||||
readonly database: ServerDatabase;
|
||||
readonly playlists: ServerPlaylistRepository;
|
||||
readonly userState: ServerUserStateRepository;
|
||||
readonly revisions: PlaylistRevisions;
|
||||
readonly startedAt: Date;
|
||||
}
|
||||
|
||||
export async function openServerStorage(): Promise<ServerStorage> {
|
||||
const database = await openServerDatabase();
|
||||
return {
|
||||
database,
|
||||
playlists: new ServerPlaylistRepository(database.db),
|
||||
userState: new ServerUserStateRepository(database.db),
|
||||
revisions: new PlaylistRevisions(),
|
||||
startedAt: new Date(),
|
||||
};
|
||||
}
|
||||
|
||||
/** Deletes a playlist and its user state together, then moves its ETag. */
|
||||
export function deletePlaylistWithState(
|
||||
storage: ServerStorage,
|
||||
playlistId: string
|
||||
): void {
|
||||
storage.database.transaction(() => {
|
||||
storage.playlists.delete(playlistId);
|
||||
storage.userState.deleteForPlaylist(playlistId);
|
||||
});
|
||||
storage.revisions.bump(playlistId);
|
||||
}
|
||||
|
||||
/** Deletes every playlist and all user state together. */
|
||||
export function deleteAllPlaylistsWithState(storage: ServerStorage): void {
|
||||
storage.database.transaction(() => {
|
||||
storage.playlists.deleteAll();
|
||||
storage.userState.deleteAll();
|
||||
});
|
||||
storage.revisions.reset();
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
import { withTempStorage } from './server-storage.spec-helpers';
|
||||
|
||||
describe('ServerUserStateRepository', () => {
|
||||
it('returns every namespace, empty by default', () =>
|
||||
withTempStorage(({ userState }) => {
|
||||
expect(userState.all()).toEqual({
|
||||
'xtream-favorites': {},
|
||||
'xtream-recent-items': {},
|
||||
'xtream-collection-items': {},
|
||||
'xtream-playback-positions': {},
|
||||
});
|
||||
}));
|
||||
|
||||
it('stores, overwrites and deletes one entry', () =>
|
||||
withTempStorage(({ userState }) => {
|
||||
userState.put('xtream-favorites', 'p1', ['live:1']);
|
||||
userState.put('xtream-favorites', 'p1', ['live:2']);
|
||||
expect(userState.all()['xtream-favorites']).toEqual({
|
||||
p1: ['live:2'],
|
||||
});
|
||||
|
||||
userState.delete('xtream-favorites', 'p1');
|
||||
expect(userState.all()['xtream-favorites']).toEqual({});
|
||||
}));
|
||||
|
||||
it('deletes a playlist across namespaces', () =>
|
||||
withTempStorage(({ userState }) => {
|
||||
userState.put('xtream-favorites', 'p1', ['live:1']);
|
||||
userState.put('xtream-recent-items', 'p1', []);
|
||||
userState.put('xtream-favorites', 'p2', ['live:3']);
|
||||
|
||||
userState.deleteForPlaylist('p1');
|
||||
|
||||
const all = userState.all();
|
||||
expect(all['xtream-favorites']).toEqual({ p2: ['live:3'] });
|
||||
expect(all['xtream-recent-items']).toEqual({});
|
||||
}));
|
||||
|
||||
it('round-trips a playlist id that names an Object.prototype key', () =>
|
||||
withTempStorage(({ userState }) => {
|
||||
userState.put('xtream-favorites', '__proto__', ['live:1']);
|
||||
userState.put('xtream-favorites', 'constructor', ['live:2']);
|
||||
|
||||
const favorites = userState.all()['xtream-favorites'];
|
||||
|
||||
expect(Object.getOwnPropertyNames(favorites).sort()).toEqual([
|
||||
'__proto__',
|
||||
'constructor',
|
||||
]);
|
||||
expect(JSON.parse(JSON.stringify(favorites))).toEqual(
|
||||
JSON.parse('{"__proto__":["live:1"],"constructor":["live:2"]}')
|
||||
);
|
||||
}));
|
||||
});
|
||||
@@ -0,0 +1,72 @@
|
||||
import { and, eq } from 'drizzle-orm';
|
||||
import type { DatabaseInstance } from '@iptvnator/shared/database';
|
||||
import {
|
||||
SERVER_USER_STATE_NAMESPACES,
|
||||
type ServerUserStateNamespace,
|
||||
type ServerUserStateResponse,
|
||||
} from '@iptvnator/shared/interfaces';
|
||||
import { serverUserState } from './server-user-state.table';
|
||||
|
||||
/** Xtream PWA user data, one JSON value per namespace and playlist. */
|
||||
export class ServerUserStateRepository {
|
||||
constructor(private readonly db: DatabaseInstance) {}
|
||||
|
||||
all(): ServerUserStateResponse['entries'] {
|
||||
// Prototype-free buckets: a playlist id such as `__proto__` must be
|
||||
// stored as an own key, not change the bucket's prototype.
|
||||
const entries = Object.fromEntries(
|
||||
SERVER_USER_STATE_NAMESPACES.map((namespace) => [
|
||||
namespace,
|
||||
Object.create(null) as Record<string, unknown>,
|
||||
])
|
||||
) as ServerUserStateResponse['entries'];
|
||||
|
||||
for (const row of this.db.select().from(serverUserState).all()) {
|
||||
const bucket = entries[row.namespace as ServerUserStateNamespace];
|
||||
if (bucket) {
|
||||
bucket[row.playlistId] = JSON.parse(row.value);
|
||||
}
|
||||
}
|
||||
return entries;
|
||||
}
|
||||
|
||||
put(
|
||||
namespace: ServerUserStateNamespace,
|
||||
playlistId: string,
|
||||
value: unknown
|
||||
): void {
|
||||
const serialized = JSON.stringify(value ?? null);
|
||||
const updatedAt = new Date().toISOString();
|
||||
this.db
|
||||
.insert(serverUserState)
|
||||
.values({ namespace, playlistId, value: serialized, updatedAt })
|
||||
.onConflictDoUpdate({
|
||||
target: [serverUserState.namespace, serverUserState.playlistId],
|
||||
set: { value: serialized, updatedAt },
|
||||
})
|
||||
.run();
|
||||
}
|
||||
|
||||
delete(namespace: ServerUserStateNamespace, playlistId: string): void {
|
||||
this.db
|
||||
.delete(serverUserState)
|
||||
.where(
|
||||
and(
|
||||
eq(serverUserState.namespace, namespace),
|
||||
eq(serverUserState.playlistId, playlistId)
|
||||
)
|
||||
)
|
||||
.run();
|
||||
}
|
||||
|
||||
deleteForPlaylist(playlistId: string): void {
|
||||
this.db
|
||||
.delete(serverUserState)
|
||||
.where(eq(serverUserState.playlistId, playlistId))
|
||||
.run();
|
||||
}
|
||||
|
||||
deleteAll(): void {
|
||||
this.db.delete(serverUserState).run();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
import { primaryKey, sqliteTable, text } from 'drizzle-orm/sqlite-core';
|
||||
|
||||
/**
|
||||
* Server-only table: one JSON value per Xtream user-data namespace and
|
||||
* playlist. Created by `openServerDatabase`, never by the desktop app.
|
||||
*/
|
||||
export const SERVER_USER_STATE_TABLE_SQL = `CREATE TABLE IF NOT EXISTS server_user_state (
|
||||
namespace TEXT NOT NULL,
|
||||
playlist_id TEXT NOT NULL,
|
||||
value TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL,
|
||||
PRIMARY KEY (namespace, playlist_id)
|
||||
)`;
|
||||
|
||||
export const serverUserState = sqliteTable(
|
||||
'server_user_state',
|
||||
{
|
||||
namespace: text('namespace').notNull(),
|
||||
playlistId: text('playlist_id').notNull(),
|
||||
value: text('value').notNull(),
|
||||
updatedAt: text('updated_at').notNull(),
|
||||
},
|
||||
(table) => [primaryKey({ columns: [table.namespace, table.playlistId] })]
|
||||
);
|
||||
@@ -0,0 +1,45 @@
|
||||
import { readServerStorageConfig } from './storage-config';
|
||||
|
||||
describe('readServerStorageConfig', () => {
|
||||
it('keeps storage off and the API open by default', () => {
|
||||
expect(readServerStorageConfig({})).toEqual({
|
||||
enabled: false,
|
||||
accessToken: null,
|
||||
maxDocumentBytes: 64 * 1024 * 1024,
|
||||
});
|
||||
});
|
||||
|
||||
it('enables storage only for IPTVNATOR_STORAGE=server', () => {
|
||||
expect(
|
||||
readServerStorageConfig({ IPTVNATOR_STORAGE: ' Server ' }).enabled
|
||||
).toBe(true);
|
||||
expect(
|
||||
readServerStorageConfig({ IPTVNATOR_STORAGE: 'none' }).enabled
|
||||
).toBe(false);
|
||||
expect(
|
||||
readServerStorageConfig({ IPTVNATOR_STORAGE: '1' }).enabled
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('reads a trimmed access token and ignores a blank one', () => {
|
||||
expect(
|
||||
readServerStorageConfig({ IPTVNATOR_ACCESS_TOKEN: ' s3cret ' })
|
||||
.accessToken
|
||||
).toBe('s3cret');
|
||||
expect(
|
||||
readServerStorageConfig({ IPTVNATOR_ACCESS_TOKEN: ' ' })
|
||||
.accessToken
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it('reads a positive document limit in megabytes', () => {
|
||||
expect(
|
||||
readServerStorageConfig({ IPTVNATOR_MAX_DOCUMENT_MB: '8' })
|
||||
.maxDocumentBytes
|
||||
).toBe(8 * 1024 * 1024);
|
||||
expect(
|
||||
readServerStorageConfig({ IPTVNATOR_MAX_DOCUMENT_MB: '-1' })
|
||||
.maxDocumentBytes
|
||||
).toBe(64 * 1024 * 1024);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,28 @@
|
||||
export interface ServerStorageConfig {
|
||||
/** `IPTVNATOR_STORAGE=server`; the Docker image sets it. */
|
||||
readonly enabled: boolean;
|
||||
/** `IPTVNATOR_ACCESS_TOKEN`; guards every non-public route when set. */
|
||||
readonly accessToken: string | null;
|
||||
/** Largest accepted playlist document (`IPTVNATOR_MAX_DOCUMENT_MB`). */
|
||||
readonly maxDocumentBytes: number;
|
||||
}
|
||||
|
||||
const DEFAULT_MAX_DOCUMENT_MB = 64;
|
||||
const BYTES_PER_MB = 1024 * 1024;
|
||||
|
||||
export function readServerStorageConfig(
|
||||
env: Record<string, string | undefined> = process.env
|
||||
): ServerStorageConfig {
|
||||
const mode = env['IPTVNATOR_STORAGE']?.trim().toLowerCase();
|
||||
const token = env['IPTVNATOR_ACCESS_TOKEN']?.trim();
|
||||
const megabytes = Number(env['IPTVNATOR_MAX_DOCUMENT_MB']);
|
||||
|
||||
return {
|
||||
enabled: mode === 'server',
|
||||
accessToken: token ? token : null,
|
||||
maxDocumentBytes:
|
||||
(Number.isFinite(megabytes) && megabytes > 0
|
||||
? megabytes
|
||||
: DEFAULT_MAX_DOCUMENT_MB) * BYTES_PER_MB,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
import {
|
||||
InvalidPlaylistDocumentError,
|
||||
PlaylistPayloadMissingError,
|
||||
} from './server-playlist-repository';
|
||||
import {
|
||||
describeStorageError,
|
||||
StorageRequestError,
|
||||
toStorageErrorBody,
|
||||
} from './storage-route-errors';
|
||||
|
||||
describe('toStorageErrorBody', () => {
|
||||
it.each([
|
||||
[
|
||||
new StorageRequestError(404, 'not_found', 'Playlist not found.'),
|
||||
404,
|
||||
'not_found',
|
||||
],
|
||||
[new PlaylistPayloadMissingError(), 409, 'payload_missing'],
|
||||
[new InvalidPlaylistDocumentError(), 400, 'invalid_body'],
|
||||
[{ type: 'entity.too.large' }, 413, 'too_large'],
|
||||
[{ type: 'entity.parse.failed' }, 400, 'invalid_json'],
|
||||
[
|
||||
Object.assign(new Error('busy'), { code: 'SQLITE_BUSY' }),
|
||||
503,
|
||||
'storage_busy',
|
||||
],
|
||||
[new Error('SELECT secret FROM x'), 500, 'storage_error'],
|
||||
])('maps %p to %i', (error, status, code) => {
|
||||
expect(toStorageErrorBody(error)).toMatchObject({ status, code });
|
||||
});
|
||||
|
||||
it('never puts the raw error message into a 500 body or log line', () => {
|
||||
const error = new Error('password=hunter2');
|
||||
|
||||
expect(JSON.stringify(toStorageErrorBody(error))).not.toContain(
|
||||
'hunter2'
|
||||
);
|
||||
expect(describeStorageError(error)).toBe('Error');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,95 @@
|
||||
import type { ErrorRequestHandler } from 'express';
|
||||
import type { ServerStorageErrorBody } from '@iptvnator/shared/interfaces';
|
||||
import {
|
||||
InvalidPlaylistDocumentError,
|
||||
PlaylistPayloadMissingError,
|
||||
} from './server-playlist-repository';
|
||||
|
||||
export class StorageRequestError extends Error {
|
||||
constructor(
|
||||
readonly status: number,
|
||||
readonly code: string,
|
||||
message: string
|
||||
) {
|
||||
super(message);
|
||||
this.name = 'StorageRequestError';
|
||||
}
|
||||
}
|
||||
|
||||
export function toStorageErrorBody(error: unknown): ServerStorageErrorBody {
|
||||
if (error instanceof StorageRequestError) {
|
||||
return {
|
||||
message: error.message,
|
||||
status: error.status,
|
||||
code: error.code,
|
||||
};
|
||||
}
|
||||
if (error instanceof PlaylistPayloadMissingError) {
|
||||
return {
|
||||
message: 'The server has no stored payload for this playlist.',
|
||||
status: 409,
|
||||
code: 'payload_missing',
|
||||
};
|
||||
}
|
||||
if (error instanceof InvalidPlaylistDocumentError) {
|
||||
return {
|
||||
message: 'A playlist needs an id.',
|
||||
status: 400,
|
||||
code: 'invalid_body',
|
||||
};
|
||||
}
|
||||
const type = (error as { type?: unknown } | null)?.type;
|
||||
if (type === 'entity.too.large') {
|
||||
return {
|
||||
message: 'The request is larger than the server allows.',
|
||||
status: 413,
|
||||
code: 'too_large',
|
||||
};
|
||||
}
|
||||
if (type === 'entity.parse.failed') {
|
||||
return {
|
||||
message: 'The request body is not valid JSON.',
|
||||
status: 400,
|
||||
code: 'invalid_json',
|
||||
};
|
||||
}
|
||||
if ((error as { code?: unknown } | null)?.code === 'SQLITE_BUSY') {
|
||||
return {
|
||||
message: 'Storage is busy, try again.',
|
||||
status: 503,
|
||||
code: 'storage_busy',
|
||||
};
|
||||
}
|
||||
return {
|
||||
message: 'Storage request failed.',
|
||||
status: 500,
|
||||
code: 'storage_error',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Error name and code only: Drizzle and SQLite messages can carry bound
|
||||
* values such as portal credentials, which must never reach a log.
|
||||
*/
|
||||
export function describeStorageError(error: unknown): string {
|
||||
const name = error instanceof Error ? error.name : typeof error;
|
||||
const code = (error as { code?: unknown } | null)?.code;
|
||||
return typeof code === 'string' ? `${name} (${code})` : name;
|
||||
}
|
||||
|
||||
export function storageErrorHandler(
|
||||
log: (message: string) => void = console.error
|
||||
): ErrorRequestHandler {
|
||||
// Express recognizes error middleware by its four parameters.
|
||||
// eslint-disable-next-line @typescript-eslint/no-unused-vars
|
||||
return (error, req, res, _next) => {
|
||||
const body = toStorageErrorBody(error);
|
||||
if (body.status >= 500) {
|
||||
const route = `${req.baseUrl}${req.route?.path ?? ''}`;
|
||||
log(
|
||||
`[web-backend] storage ${req.method} ${route} failed: ${describeStorageError(error)}`
|
||||
);
|
||||
}
|
||||
res.status(body.status).json(body);
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,312 @@
|
||||
import { createWebBackendApp } from '../web-backend-app';
|
||||
import {
|
||||
resolvePublicHost,
|
||||
StubHttpClient,
|
||||
withServer,
|
||||
} from '../web-backend-app.spec-helpers';
|
||||
import { m3uDocument, withTempStorage } from './server-storage.spec-helpers';
|
||||
import type {
|
||||
ServerInstanceInfo,
|
||||
ServerStorageStatusResponse,
|
||||
ServerUserStateResponse,
|
||||
} from '@iptvnator/shared/interfaces';
|
||||
import type { ServerStorage } from './server-storage';
|
||||
|
||||
function app(
|
||||
storage: ServerStorage | null,
|
||||
extra: { accessToken?: string; maxDocumentBytes?: number } = {}
|
||||
) {
|
||||
return createWebBackendApp({
|
||||
clientOrigins: ['http://localhost:4200'],
|
||||
httpClient: new StubHttpClient(),
|
||||
resolveHostname: resolvePublicHost,
|
||||
storage,
|
||||
...extra,
|
||||
});
|
||||
}
|
||||
|
||||
const json = (body: unknown, init: RequestInit = {}): RequestInit => ({
|
||||
...init,
|
||||
body: JSON.stringify(body),
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
...((init.headers as Record<string, string> | undefined) ?? {}),
|
||||
},
|
||||
});
|
||||
|
||||
describe('server storage routes', () => {
|
||||
it('reports no storage when it is disabled', () =>
|
||||
withServer(app(null), async (baseUrl) => {
|
||||
const response = await fetch(`${baseUrl}/v1/instance`);
|
||||
await expect(response.json()).resolves.toEqual({
|
||||
service: 'iptvnator',
|
||||
apiVersion: 1,
|
||||
instanceId: null,
|
||||
storage: 'none',
|
||||
auth: 'none',
|
||||
});
|
||||
expect((await fetch(`${baseUrl}/v1/playlists`)).status).toBe(404);
|
||||
}));
|
||||
|
||||
it('reports server storage and its status', () =>
|
||||
withTempStorage((storage) =>
|
||||
withServer(app(storage), async (baseUrl) => {
|
||||
const instance = await (
|
||||
await fetch(`${baseUrl}/v1/instance`)
|
||||
).json();
|
||||
expect(instance).toMatchObject({
|
||||
storage: 'server',
|
||||
instanceId: storage.database.instanceId,
|
||||
});
|
||||
const status = (await (
|
||||
await fetch(`${baseUrl}/v1/instance/status`)
|
||||
).json()) as ServerStorageStatusResponse;
|
||||
expect(status).toMatchObject({ playlists: 0 });
|
||||
expect(status.databaseBytes).toBeGreaterThan(0);
|
||||
})
|
||||
));
|
||||
|
||||
it('stores a document, serves it with an ETag and answers 304', () =>
|
||||
withTempStorage((storage) =>
|
||||
withServer(app(storage), async (baseUrl) => {
|
||||
const put = await fetch(
|
||||
`${baseUrl}/v1/playlists/p1`,
|
||||
json(m3uDocument('p1'), { method: 'PUT' })
|
||||
);
|
||||
expect(put.status).toBe(200);
|
||||
const etag = put.headers.get('etag');
|
||||
expect(etag).toBeTruthy();
|
||||
|
||||
const get = await fetch(`${baseUrl}/v1/playlists/p1`);
|
||||
expect(get.headers.get('etag')).toBe(etag);
|
||||
await expect(get.json()).resolves.toMatchObject({ _id: 'p1' });
|
||||
|
||||
const cached = await fetch(`${baseUrl}/v1/playlists/p1`, {
|
||||
headers: { 'if-none-match': etag ?? '' },
|
||||
});
|
||||
expect(cached.status).toBe(304);
|
||||
|
||||
// nginx weakens the ETag of a gzipped response.
|
||||
const weakened = await fetch(`${baseUrl}/v1/playlists/p1`, {
|
||||
headers: { 'if-none-match': `W/${etag}` },
|
||||
});
|
||||
expect(weakened.status).toBe(304);
|
||||
})
|
||||
));
|
||||
|
||||
it('answers 404, not 304, for a missing playlist', () =>
|
||||
withTempStorage((storage) =>
|
||||
withServer(app(storage), async (baseUrl) => {
|
||||
const wildcard = await fetch(`${baseUrl}/v1/playlists/nope`, {
|
||||
headers: { 'if-none-match': '*' },
|
||||
});
|
||||
const guessed = await fetch(`${baseUrl}/v1/playlists/nope`, {
|
||||
headers: {
|
||||
'if-none-match': storage.revisions.etag('nope'),
|
||||
},
|
||||
});
|
||||
|
||||
expect(wildcard.status).toBe(404);
|
||||
expect(guessed.status).toBe(404);
|
||||
})
|
||||
));
|
||||
|
||||
it('keeps the payload on keepPayload and refuses it for unknown ids', () =>
|
||||
withTempStorage((storage) =>
|
||||
withServer(app(storage), async (baseUrl) => {
|
||||
await fetch(
|
||||
`${baseUrl}/v1/playlists/p1`,
|
||||
json(m3uDocument('p1'), { method: 'PUT' })
|
||||
);
|
||||
const meta: Record<string, unknown> = { ...m3uDocument('p1') };
|
||||
delete meta['playlist'];
|
||||
|
||||
const keep = await fetch(
|
||||
`${baseUrl}/v1/playlists/p1?keepPayload=1`,
|
||||
json({ ...meta, title: 'Renamed' }, { method: 'PUT' })
|
||||
);
|
||||
const missing = await fetch(
|
||||
`${baseUrl}/v1/playlists/zz?keepPayload=1`,
|
||||
json({ title: 'x' }, { method: 'PUT' })
|
||||
);
|
||||
|
||||
expect(keep.status).toBe(200);
|
||||
await expect(
|
||||
(await fetch(`${baseUrl}/v1/playlists/p1`)).json()
|
||||
).resolves.toMatchObject({
|
||||
title: 'Renamed',
|
||||
playlist: m3uDocument('p1').playlist,
|
||||
});
|
||||
expect(missing.status).toBe(409);
|
||||
await expect(missing.json()).resolves.toMatchObject({
|
||||
code: 'payload_missing',
|
||||
});
|
||||
})
|
||||
));
|
||||
|
||||
it('lists metadata and deletes documents with their user state', () =>
|
||||
withTempStorage((storage) =>
|
||||
withServer(app(storage), async (baseUrl) => {
|
||||
await fetch(
|
||||
`${baseUrl}/v1/playlists/p1`,
|
||||
json(m3uDocument('p1'), { method: 'PUT' })
|
||||
);
|
||||
await fetch(
|
||||
`${baseUrl}/v1/user-state/xtream-favorites/p1`,
|
||||
json({ value: ['live:1'] }, { method: 'PUT' })
|
||||
);
|
||||
|
||||
const metas = (await (
|
||||
await fetch(`${baseUrl}/v1/playlists?view=meta`)
|
||||
).json()) as Record<string, unknown>[];
|
||||
expect(metas[0]['playlist']).toBeUndefined();
|
||||
|
||||
await fetch(`${baseUrl}/v1/playlists/p1`, {
|
||||
method: 'DELETE',
|
||||
});
|
||||
|
||||
expect((await fetch(`${baseUrl}/v1/playlists/p1`)).status).toBe(
|
||||
404
|
||||
);
|
||||
const state = (await (
|
||||
await fetch(`${baseUrl}/v1/user-state`)
|
||||
).json()) as ServerUserStateResponse;
|
||||
expect(state.entries['xtream-favorites']).toEqual({});
|
||||
})
|
||||
));
|
||||
|
||||
it('validates user-state namespaces and stores app state', () =>
|
||||
withTempStorage((storage) =>
|
||||
withServer(app(storage), async (baseUrl) => {
|
||||
const bad = await fetch(
|
||||
`${baseUrl}/v1/user-state/settings/p1`,
|
||||
json({ value: 1 }, { method: 'PUT' })
|
||||
);
|
||||
expect(bad.status).toBe(400);
|
||||
|
||||
await fetch(
|
||||
`${baseUrl}/v1/app-state/flag`,
|
||||
json({ value: '1' }, { method: 'PUT' })
|
||||
);
|
||||
const read = await fetch(`${baseUrl}/v1/app-state/flag`);
|
||||
expect(read.headers.get('cache-control')).toBe('no-store');
|
||||
await expect(read.json()).resolves.toEqual({ value: '1' });
|
||||
})
|
||||
));
|
||||
|
||||
it('keeps browser app state away from server-owned keys', () =>
|
||||
withTempStorage((storage) =>
|
||||
withServer(app(storage), async (baseUrl) => {
|
||||
await fetch(
|
||||
`${baseUrl}/v1/app-state/server:instance-id`,
|
||||
json({ value: '' }, { method: 'PUT' })
|
||||
);
|
||||
|
||||
expect(
|
||||
storage.playlists.getAppState('server:instance-id')
|
||||
).toBe(storage.database.instanceId);
|
||||
await expect(
|
||||
(
|
||||
await fetch(
|
||||
`${baseUrl}/v1/app-state/server:instance-id`
|
||||
)
|
||||
).json()
|
||||
).resolves.toEqual({ value: '' });
|
||||
})
|
||||
));
|
||||
|
||||
it('keeps the playlist when its user-state cleanup fails', () =>
|
||||
withTempStorage((storage) =>
|
||||
withServer(app(storage), async (baseUrl) => {
|
||||
await fetch(
|
||||
`${baseUrl}/v1/playlists/p1`,
|
||||
json(m3uDocument('p1'), { method: 'PUT' })
|
||||
);
|
||||
const etag = storage.revisions.etag('p1');
|
||||
jest.spyOn(
|
||||
storage.userState,
|
||||
'deleteForPlaylist'
|
||||
).mockImplementationOnce(() => {
|
||||
throw new Error('disk full');
|
||||
});
|
||||
|
||||
const response = await fetch(`${baseUrl}/v1/playlists/p1`, {
|
||||
method: 'DELETE',
|
||||
});
|
||||
|
||||
expect(response.status).toBe(500);
|
||||
expect(storage.playlists.exists('p1')).toBe(true);
|
||||
expect(storage.revisions.etag('p1')).toBe(etag);
|
||||
})
|
||||
));
|
||||
|
||||
it('rejects playlist ids with surrounding whitespace', () =>
|
||||
withTempStorage((storage) =>
|
||||
withServer(app(storage), async (baseUrl) => {
|
||||
const put = await fetch(
|
||||
`${baseUrl}/v1/playlists/%20p1%20`,
|
||||
json(m3uDocument('p1'), { method: 'PUT' })
|
||||
);
|
||||
const get = await fetch(`${baseUrl}/v1/playlists/%20p1%20`);
|
||||
|
||||
expect(put.status).toBe(400);
|
||||
await expect(put.json()).resolves.toMatchObject({
|
||||
code: 'invalid_id',
|
||||
});
|
||||
expect(get.status).toBe(400);
|
||||
expect(storage.playlists.count()).toBe(0);
|
||||
})
|
||||
));
|
||||
|
||||
it('answers 413 above the document limit', () =>
|
||||
withTempStorage((storage) =>
|
||||
withServer(
|
||||
app(storage, { maxDocumentBytes: 512 }),
|
||||
async (baseUrl) => {
|
||||
const response = await fetch(
|
||||
`${baseUrl}/v1/playlists/p1`,
|
||||
json(
|
||||
{ ...m3uDocument('p1'), padding: 'x'.repeat(2048) },
|
||||
{ method: 'PUT' }
|
||||
)
|
||||
);
|
||||
expect(response.status).toBe(413);
|
||||
}
|
||||
)
|
||||
));
|
||||
|
||||
it('requires the token for storage and proxy routes when one is set', () =>
|
||||
withTempStorage((storage) =>
|
||||
withServer(
|
||||
app(storage, { accessToken: 's3cret' }),
|
||||
async (baseUrl) => {
|
||||
const instance = (await (
|
||||
await fetch(`${baseUrl}/v1/instance`)
|
||||
).json()) as ServerInstanceInfo;
|
||||
expect(instance.auth).toBe('token');
|
||||
|
||||
expect(
|
||||
(await fetch(`${baseUrl}/v1/playlists`)).status
|
||||
).toBe(401);
|
||||
expect(
|
||||
(
|
||||
await fetch(
|
||||
`${baseUrl}/provider-targets`,
|
||||
json(
|
||||
{ url: 'http://example.test' },
|
||||
{ method: 'POST' }
|
||||
)
|
||||
)
|
||||
).status
|
||||
).toBe(401);
|
||||
expect(
|
||||
(
|
||||
await fetch(`${baseUrl}/v1/playlists`, {
|
||||
headers: { authorization: 'Bearer s3cret' },
|
||||
})
|
||||
).status
|
||||
).toBe(200);
|
||||
}
|
||||
)
|
||||
));
|
||||
});
|
||||
@@ -0,0 +1,87 @@
|
||||
import express, { Router } from 'express';
|
||||
import {
|
||||
isServerUserStateNamespace,
|
||||
type ServerUserStateNamespace,
|
||||
} from '@iptvnator/shared/interfaces';
|
||||
import type { ServerStorage } from './server-storage';
|
||||
import { StorageRequestError } from './storage-route-errors';
|
||||
|
||||
const USER_STATE_BODY_LIMIT = '8mb';
|
||||
/**
|
||||
* Browser app state lives under its own prefix in `app_state`, so a client
|
||||
* key can never overwrite a server-owned one such as the instance id.
|
||||
*/
|
||||
const CLIENT_APP_STATE_PREFIX = 'client:';
|
||||
|
||||
export function createUserStateRouter(storage: ServerStorage): Router {
|
||||
const router = Router();
|
||||
const json = express.json({ limit: USER_STATE_BODY_LIMIT });
|
||||
|
||||
router.get('/user-state', (_req, res) => {
|
||||
res.set('Cache-Control', 'no-store').json({
|
||||
entries: storage.userState.all(),
|
||||
});
|
||||
});
|
||||
|
||||
router.put('/user-state/:namespace/:playlistId', json, (req, res) => {
|
||||
const body = req.body as { value?: unknown } | undefined;
|
||||
if (!body || typeof body !== 'object' || !('value' in body)) {
|
||||
throw new StorageRequestError(
|
||||
400,
|
||||
'invalid_body',
|
||||
'Expected { value }.'
|
||||
);
|
||||
}
|
||||
storage.userState.put(
|
||||
requireNamespace(req.params.namespace),
|
||||
req.params.playlistId,
|
||||
body.value
|
||||
);
|
||||
res.json({ success: true });
|
||||
});
|
||||
|
||||
router.delete('/user-state/:namespace/:playlistId', (req, res) => {
|
||||
storage.userState.delete(
|
||||
requireNamespace(req.params.namespace),
|
||||
req.params.playlistId
|
||||
);
|
||||
res.json({ success: true });
|
||||
});
|
||||
|
||||
router.get('/app-state/:key', (req, res) => {
|
||||
res.set('Cache-Control', 'no-store').json({
|
||||
value: storage.playlists.getAppState(
|
||||
CLIENT_APP_STATE_PREFIX + req.params.key
|
||||
),
|
||||
});
|
||||
});
|
||||
|
||||
router.put('/app-state/:key', json, (req, res) => {
|
||||
const value = (req.body as { value?: unknown } | undefined)?.value;
|
||||
if (typeof value !== 'string') {
|
||||
throw new StorageRequestError(
|
||||
400,
|
||||
'invalid_body',
|
||||
'Expected { value: string }.'
|
||||
);
|
||||
}
|
||||
storage.playlists.setAppState(
|
||||
CLIENT_APP_STATE_PREFIX + req.params.key,
|
||||
value
|
||||
);
|
||||
res.json({ success: true });
|
||||
});
|
||||
|
||||
return router;
|
||||
}
|
||||
|
||||
function requireNamespace(value: string): ServerUserStateNamespace {
|
||||
if (!isServerUserStateNamespace(value)) {
|
||||
throw new StorageRequestError(
|
||||
400,
|
||||
'invalid_namespace',
|
||||
'Unknown user-state namespace.'
|
||||
);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
@@ -43,6 +43,11 @@ import {
|
||||
resolveHostname,
|
||||
validateProviderUrl,
|
||||
} from './provider-url-policy';
|
||||
import {
|
||||
DEFAULT_MAX_DOCUMENT_BYTES,
|
||||
mountServerStorage,
|
||||
} from './storage/mount-server-storage';
|
||||
import type { ServerStorage } from './storage/server-storage';
|
||||
export type {
|
||||
WebBackendHttpClient,
|
||||
WebBackendHttpGetOptions,
|
||||
@@ -67,6 +72,12 @@ export interface WebBackendAppOptions {
|
||||
readonly now?: () => Date;
|
||||
readonly resolveHostname?: (hostname: string) => Promise<readonly string[]>;
|
||||
readonly runtimeBackendUrl?: string;
|
||||
/** Optional bearer token for every non-public route. */
|
||||
readonly accessToken?: string | null;
|
||||
/** Largest accepted playlist document in bytes. */
|
||||
readonly maxDocumentBytes?: number;
|
||||
/** Server storage; `null` or absent keeps the backend stateless. */
|
||||
readonly storage?: ServerStorage | null;
|
||||
}
|
||||
|
||||
type ProviderTargetRegistry = Map<string, URL>;
|
||||
@@ -129,6 +140,14 @@ export function createWebBackendApp(
|
||||
);
|
||||
});
|
||||
|
||||
mountServerStorage(app, {
|
||||
corsMiddleware,
|
||||
storage: options.storage ?? null,
|
||||
accessToken: options.accessToken ?? null,
|
||||
maxDocumentBytes:
|
||||
options.maxDocumentBytes ?? DEFAULT_MAX_DOCUMENT_BYTES,
|
||||
});
|
||||
|
||||
app.options('/provider-targets', corsMiddleware);
|
||||
app.post(
|
||||
'/provider-targets',
|
||||
|
||||
@@ -2,18 +2,59 @@ import {
|
||||
applyDefaultAutoSelectFamilyAttemptTimeout,
|
||||
AUTO_SELECT_FAMILY_ATTEMPT_TIMEOUT_FLAG,
|
||||
} from '@iptvnator/shared/host-health';
|
||||
import { closeGracefully } from './app/graceful-shutdown';
|
||||
import { createWebBackendApp } from './app/web-backend-app';
|
||||
import { readServerStorageConfig } from './app/storage/storage-config';
|
||||
import {
|
||||
openServerStorage,
|
||||
type ServerStorage,
|
||||
} from './app/storage/server-storage';
|
||||
import { describeStorageError } from './app/storage/storage-route-errors';
|
||||
|
||||
const attemptTimeout = applyDefaultAutoSelectFamilyAttemptTimeout();
|
||||
if (attemptTimeout !== null) {
|
||||
console.log(
|
||||
`[web-backend] connection attempt timeout set to ${attemptTimeout} ms for IPv6->IPv4 fallback; pass ${AUTO_SELECT_FAMILY_ATTEMPT_TIMEOUT_FLAG} via NODE_OPTIONS to override`
|
||||
);
|
||||
async function start(): Promise<void> {
|
||||
const attemptTimeout = applyDefaultAutoSelectFamilyAttemptTimeout();
|
||||
if (attemptTimeout !== null) {
|
||||
console.log(
|
||||
`[web-backend] connection attempt timeout set to ${attemptTimeout} ms for IPv6->IPv4 fallback; pass ${AUTO_SELECT_FAMILY_ATTEMPT_TIMEOUT_FLAG} via NODE_OPTIONS to override`
|
||||
);
|
||||
}
|
||||
|
||||
const config = readServerStorageConfig();
|
||||
let storage: ServerStorage | null = null;
|
||||
if (config.enabled) {
|
||||
try {
|
||||
storage = await openServerStorage();
|
||||
console.log(
|
||||
`[web-backend] server storage at ${storage.database.path}`
|
||||
);
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`[web-backend] server storage could not open: ${describeStorageError(error)}. Check that IPTVNATOR_DATA_DIR is a writable volume.`
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
const port = Number(process.env['PORT'] ?? 3000);
|
||||
const app = createWebBackendApp({
|
||||
storage,
|
||||
accessToken: config.accessToken,
|
||||
maxDocumentBytes: config.maxDocumentBytes,
|
||||
});
|
||||
const server = app.listen(port, () => {
|
||||
console.log(
|
||||
`IPTVnator web backend listening on http://localhost:${port}`
|
||||
);
|
||||
});
|
||||
|
||||
const shutdown = () => {
|
||||
void closeGracefully(server).then(() => {
|
||||
storage?.database.close();
|
||||
process.exit(0);
|
||||
});
|
||||
};
|
||||
process.once('SIGTERM', shutdown);
|
||||
process.once('SIGINT', shutdown);
|
||||
}
|
||||
|
||||
const port = Number(process.env['PORT'] ?? 3000);
|
||||
const app = createWebBackendApp();
|
||||
|
||||
app.listen(port, () => {
|
||||
console.log(`IPTVnator web backend listening on http://localhost:${port}`);
|
||||
});
|
||||
void start();
|
||||
+22
-1
@@ -28,6 +28,22 @@ RUN node tools/build/inject-build-commit.mjs
|
||||
RUN pnpm nx build web --configuration=pwa
|
||||
RUN pnpm nx build web-backend
|
||||
|
||||
# better-sqlite3 is a native addon. This stage is NOT pinned to
|
||||
# $BUILDPLATFORM, so multi-arch builds produce it for each target platform.
|
||||
# prebuild-install downloads a musl binary when one exists and otherwise
|
||||
# compiles with the toolchain below.
|
||||
FROM node:24-alpine AS native
|
||||
|
||||
RUN apk add --no-cache python3 make g++
|
||||
|
||||
WORKDIR /opt/native
|
||||
|
||||
COPY package.json ./root-package.json
|
||||
|
||||
RUN BETTER_SQLITE3_VERSION="$(node -p 'require("./root-package.json").dependencies["better-sqlite3"]')" \
|
||||
&& npm init -y >/dev/null \
|
||||
&& npm install --omit=dev --no-audit --no-fund "better-sqlite3@${BETTER_SQLITE3_VERSION}"
|
||||
|
||||
FROM node:24-alpine
|
||||
|
||||
RUN apk add --no-cache gettext nginx
|
||||
@@ -37,13 +53,18 @@ WORKDIR /opt/iptvnator
|
||||
ENV PORT=3000
|
||||
ENV BACKEND_URL=/api
|
||||
ENV CLIENT_URL=http://localhost:4333
|
||||
ENV IPTVNATOR_STORAGE=server
|
||||
ENV IPTVNATOR_DATA_DIR=/data
|
||||
|
||||
COPY --from=build /usr/src/app/dist/apps/web /usr/share/nginx/html
|
||||
COPY --from=build /usr/src/app/dist/apps/web-backend ./web-backend
|
||||
COPY --from=native /opt/native/node_modules ./web-backend/node_modules
|
||||
COPY docker/nginx.conf /etc/nginx/http.d/default.conf.template
|
||||
COPY docker/docker-entrypoint.sh /usr/local/bin/iptvnator-entrypoint
|
||||
|
||||
RUN chmod +x /usr/local/bin/iptvnator-entrypoint
|
||||
RUN chmod +x /usr/local/bin/iptvnator-entrypoint && mkdir -p /data
|
||||
|
||||
VOLUME ["/data"]
|
||||
|
||||
EXPOSE 80
|
||||
|
||||
|
||||
@@ -98,6 +98,11 @@ safe local defaults shown below.
|
||||
| `PORT` | `3000` | Internal Express backend port. nginx proxy config is rendered from the template to match it at startup. |
|
||||
| `IPTVNATOR_PROXY_ALLOW_PRIVATE_NETWORKS` | `0` | Set to `1` or `true` only for trusted local/LAN deployments that intentionally proxy private network IPTV or mock endpoints. |
|
||||
| `NODE_EXTRA_CA_CERTS` | unset | Optional Node.js CA bundle path for providers using private certificate authorities. Mount the CA file into the container and set this path. |
|
||||
| `IPTVNATOR_STORAGE` | `server` in the image | `server` opens the server storage database under `IPTVNATOR_DATA_DIR` and its `/api/v1` routes; `none` keeps the backend stateless. |
|
||||
| `IPTVNATOR_DATA_DIR` | `/data` in the image | Data root; the database is `<root>/databases/iptvnator.db`. Mount a volume here. |
|
||||
| `IPTVNATOR_ACCESS_TOKEN` | unset | When set, every route except `/`, `/health`, `/config.js` and `/v1/instance` requires `Authorization: Bearer <token>`, proxy routes included. The browser app cannot send a token yet: leave it unset until it can. |
|
||||
| `IPTVNATOR_MAX_DOCUMENT_MB` | `64` | Largest playlist document the backend accepts. Keep the nginx `client_max_body_size` in step. |
|
||||
| `IPTVNATOR_BACKEND_START_TIMEOUT` | `300` | Seconds the entrypoint waits for the backend's `/health` before giving up. Database migrations run before the backend listens. |
|
||||
|
||||
The web backend proxy accepts only `http` and `https` provider URLs. The PWA
|
||||
first registers provider URLs through `/provider-targets`, then uses the
|
||||
@@ -127,6 +132,22 @@ define a health check against `/api/health`. If nginx or the backend exits
|
||||
after startup, the entrypoint exits the container so Docker Compose can apply
|
||||
the `restart: unless-stopped` policy.
|
||||
|
||||
## Server Storage
|
||||
|
||||
The image opens a server storage database on the `/data` volume
|
||||
(`IPTVNATOR_STORAGE=server`, `IPTVNATOR_DATA_DIR=/data`) and serves its API
|
||||
under `/api/v1`. It is the backend half of keeping playlists, favorites,
|
||||
history and playback positions on the server for every device; the PWA
|
||||
starts using it in a follow-up change. Until then the volume holds an empty
|
||||
database and each browser keeps its own data.
|
||||
|
||||
Provider credentials will be stored unencrypted in that database, as the
|
||||
desktop app does. Do not expose an instance with server storage to the
|
||||
internet without authentication in your reverse proxy. The backend already
|
||||
supports `IPTVNATOR_ACCESS_TOKEN`, but the browser app cannot send the
|
||||
token yet, so setting it now makes the app unusable. Back up the volume like any database: stop the container, or copy
|
||||
`databases/iptvnator.db` together with its `-wal` file.
|
||||
|
||||
## IPv6, VPNs, And Connection Fallback
|
||||
|
||||
Node races IPv6 and IPv4 addresses when a provider hostname has both ("happy
|
||||
|
||||
@@ -13,9 +13,17 @@ services:
|
||||
CLIENT_URL: http://localhost:4333
|
||||
PORT: "3000"
|
||||
IPTVNATOR_PROXY_ALLOW_PRIVATE_NETWORKS: "0"
|
||||
# Server storage database on the /data volume. Set to "none" to keep
|
||||
# the backend stateless.
|
||||
IPTVNATOR_STORAGE: server
|
||||
volumes:
|
||||
- iptvnator-data:/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1/api/health >/dev/null || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
|
||||
volumes:
|
||||
iptvnator-data: {}
|
||||
@@ -60,8 +60,12 @@ trap handle_signal INT TERM
|
||||
trap cleanup EXIT
|
||||
|
||||
backend_ready=0
|
||||
# With server storage the backend migrates its database before it listens,
|
||||
# which can take a while on a large volume. The loop still exits at once if
|
||||
# the backend process dies.
|
||||
BACKEND_START_TIMEOUT="${IPTVNATOR_BACKEND_START_TIMEOUT:-300}"
|
||||
|
||||
for _ in $(seq 1 30); do
|
||||
for _ in $(seq 1 "$BACKEND_START_TIMEOUT"); do
|
||||
if wget -qO- "http://127.0.0.1:${PORT}/health" >/dev/null 2>&1; then
|
||||
backend_ready=1
|
||||
break
|
||||
@@ -79,7 +83,7 @@ for _ in $(seq 1 30); do
|
||||
done
|
||||
|
||||
if [ "$backend_ready" -ne 1 ]; then
|
||||
echo "IPTVnator web backend did not become healthy on port ${PORT}."
|
||||
echo "IPTVnator web backend did not become healthy on port ${PORT} within ${BACKEND_START_TIMEOUT} seconds (IPTVNATOR_BACKEND_START_TIMEOUT)."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
@@ -13,6 +13,14 @@ server {
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
# Playlist documents carry whole M3U payloads (default backend
|
||||
# limit IPTVNATOR_MAX_DOCUMENT_MB=64).
|
||||
client_max_body_size 64m;
|
||||
proxy_read_timeout 120s;
|
||||
gzip on;
|
||||
gzip_proxied any;
|
||||
gzip_types application/json;
|
||||
gzip_min_length 1024;
|
||||
}
|
||||
|
||||
location /assets/app-config.js {
|
||||
|
||||
@@ -73,6 +73,40 @@ The current self-hosted PWA uses these `apps/web-backend` routes:
|
||||
- `GET /xtream?targetId=<id>&username=<u>&password=<p>&action=<action>`
|
||||
- `GET /stalker?targetId=<id>&macAddress=<mac>&action=<action>`
|
||||
|
||||
Server storage adds versioned routes under `/v1`; the PWA does not call them
|
||||
yet:
|
||||
|
||||
- `GET /v1/instance` (always public): `{ service, apiVersion, instanceId,
|
||||
storage: 'server' | 'none', auth: 'none' | 'token' }`
|
||||
- `GET /v1/instance/status`: playlist count, database size and path
|
||||
- `GET /v1/playlists[?view=meta]`, `GET|PUT|DELETE /v1/playlists/:id`,
|
||||
`DELETE /v1/playlists`: whole playlist documents; `GET` sends an `ETag`
|
||||
and answers `304` to a matching `If-None-Match`, and
|
||||
`PUT ...?keepPayload=1` keeps the stored `playlist`, `items` and
|
||||
`header` fields (`409 payload_missing` without a stored row)
|
||||
- `GET /v1/user-state`, `PUT|DELETE /v1/user-state/:namespace/:playlistId`:
|
||||
Xtream favorites, recently viewed, collection snapshots and playback
|
||||
positions, one JSON value per namespace and playlist
|
||||
- `GET|PUT /v1/app-state/:key`
|
||||
|
||||
Server storage lives in `apps/web-backend/src/app/storage/`. With
|
||||
`IPTVNATOR_STORAGE=server`, `main.ts` opens the shared database through
|
||||
`initDatabase()` (path from `IPTVNATOR_DATA_DIR`), so the server has the same
|
||||
schema and migrations as the desktop app and writes playlists with the
|
||||
shared row codec. It adds two server-only tables: `server_user_state` for
|
||||
Xtream user data, and `server_playlist_meta`, a copy of each document
|
||||
without its M3U payload fields that metadata reads use instead of walking
|
||||
the payload blobs (filled once on open for rows written without it). ETags
|
||||
come from an in-memory revision counter; the server is the only writer of
|
||||
its database. Storage errors are logged by error name and code only, because
|
||||
Drizzle messages can carry bound values such as portal credentials. With
|
||||
`IPTVNATOR_ACCESS_TOKEN`, every route except `/`, `/health`, `/config.js`
|
||||
and `/v1/instance` requires `Authorization: Bearer <token>`, the proxy
|
||||
routes included. The PWA cannot send that token yet, so the variable must
|
||||
stay unset until the browser side lands. Browser app state written through
|
||||
`/v1/app-state` is stored under a `client:` key prefix, apart from
|
||||
server-owned keys such as the instance id.
|
||||
|
||||
`pnpm nx run web-backend:serve` starts the backend by hand on port 3333 with
|
||||
`TSX_TSCONFIG_PATH=tsconfig.base.json`, which `tsx` needs for the
|
||||
`@iptvnator/*` path aliases. `apps/web-e2e/playwright.config.ts` does not use
|
||||
@@ -274,6 +308,15 @@ Default runtime values:
|
||||
- `PORT=3000`
|
||||
- `IPTVNATOR_PROXY_ALLOW_PRIVATE_NETWORKS=0`
|
||||
- `NODE_EXTRA_CA_CERTS` unset
|
||||
- `IPTVNATOR_STORAGE=server`, `IPTVNATOR_DATA_DIR=/data` (a `VOLUME`)
|
||||
- `IPTVNATOR_ACCESS_TOKEN` unset, `IPTVNATOR_MAX_DOCUMENT_MB=64`
|
||||
- `IPTVNATOR_BACKEND_START_TIMEOUT=300`: seconds the entrypoint waits for
|
||||
`/health`; the backend migrates its database before it listens
|
||||
|
||||
better-sqlite3 is a native addon, so the Dockerfile builds it in a separate
|
||||
`native` stage that is not pinned to `$BUILDPLATFORM`; multi-arch builds
|
||||
produce it for each target platform. nginx raises `client_max_body_size` to
|
||||
64 MB for `/api/` and gzips JSON responses.
|
||||
|
||||
When hosting behind another domain, set `CLIENT_URL` to the browser origin and
|
||||
keep `BACKEND_URL=/api` unless the reverse proxy exposes the backend elsewhere.
|
||||
@@ -288,6 +331,7 @@ Use the narrow validation ladder for self-hosted changes:
|
||||
|
||||
```bash
|
||||
pnpm nx test web-backend
|
||||
pnpm nx test web-backend --testPathPatterns=storage
|
||||
pnpm nx test web --runTestsByPath apps/web/src/app/services/runtime-config.spec.ts
|
||||
pnpm nx build web --configuration=pwa --skip-nx-cache
|
||||
pnpm nx build web-backend
|
||||
|
||||
@@ -108,4 +108,29 @@ describe('app playlist codec', () => {
|
||||
it('refuses a document without an id', () => {
|
||||
expect(buildPlaylistRow({ title: 'No id' })).toBeNull();
|
||||
});
|
||||
|
||||
it('never logs the content of a malformed stored value', () => {
|
||||
const warn = jest.spyOn(console, 'warn').mockImplementation(() => {
|
||||
/* silenced */
|
||||
});
|
||||
try {
|
||||
const row = rowOf(m3uDocument);
|
||||
parseAppPlaylist(
|
||||
storedRow(row, '{"title":"x","password":hunter2-secret}')
|
||||
);
|
||||
|
||||
const logged = warn.mock.calls
|
||||
.flat()
|
||||
.map((arg) =>
|
||||
arg instanceof Error
|
||||
? `${arg.name}: ${arg.message} ${arg.stack}`
|
||||
: String(arg)
|
||||
)
|
||||
.join(' ');
|
||||
expect(warn).toHaveBeenCalled();
|
||||
expect(logged).not.toContain('hunter2');
|
||||
} finally {
|
||||
warn.mockRestore();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -53,7 +53,12 @@ export function parseJsonValue<T>(
|
||||
try {
|
||||
return JSON.parse(value) as T;
|
||||
} catch (error) {
|
||||
console.warn('Failed to parse JSON value from DB:', error);
|
||||
// The parser's message quotes the stored text, which can hold
|
||||
// portal credentials; the error name is enough to diagnose.
|
||||
console.warn(
|
||||
'Failed to parse JSON value from DB:',
|
||||
error instanceof Error ? error.name : typeof error
|
||||
);
|
||||
return fallback;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -112,3 +112,4 @@ export * from './lib/xtream-connection-test';
|
||||
export * from './lib/portal-request-failure.util';
|
||||
|
||||
export * from './lib/source-health';
|
||||
export * from './lib/server-storage.contract';
|
||||
@@ -0,0 +1,54 @@
|
||||
/**
|
||||
* HTTP contract between the PWA and the self-hosted server storage. Routes
|
||||
* live under `/v1` of the web backend (`/api/v1` behind the Docker nginx).
|
||||
*/
|
||||
export const SERVER_STORAGE_API_VERSION = 1;
|
||||
|
||||
/** Xtream PWA user-data maps that the server stores per playlist id. */
|
||||
export const SERVER_USER_STATE_NAMESPACES = [
|
||||
'xtream-favorites',
|
||||
'xtream-recent-items',
|
||||
'xtream-collection-items',
|
||||
'xtream-playback-positions',
|
||||
] as const;
|
||||
|
||||
export type ServerUserStateNamespace =
|
||||
(typeof SERVER_USER_STATE_NAMESPACES)[number];
|
||||
|
||||
export function isServerUserStateNamespace(
|
||||
value: string
|
||||
): value is ServerUserStateNamespace {
|
||||
return (SERVER_USER_STATE_NAMESPACES as readonly string[]).includes(value);
|
||||
}
|
||||
|
||||
/**
|
||||
* Document fields that carry an M3U payload. An upsert with
|
||||
* `keepPayload=1` omits them and the server keeps the stored ones.
|
||||
*/
|
||||
export const SERVER_PAYLOAD_FIELDS = ['playlist', 'items', 'header'] as const;
|
||||
|
||||
export interface ServerInstanceInfo {
|
||||
readonly service: 'iptvnator';
|
||||
readonly apiVersion: number;
|
||||
/** Stable per database; `null` without server storage. */
|
||||
readonly instanceId: string | null;
|
||||
readonly storage: 'server' | 'none';
|
||||
readonly auth: 'none' | 'token';
|
||||
}
|
||||
|
||||
export interface ServerStorageStatusResponse {
|
||||
readonly playlists: number;
|
||||
readonly databaseBytes: number;
|
||||
readonly databasePath: string;
|
||||
readonly startedAt: string;
|
||||
}
|
||||
|
||||
export interface ServerUserStateResponse {
|
||||
readonly entries: Record<ServerUserStateNamespace, Record<string, unknown>>;
|
||||
}
|
||||
|
||||
export interface ServerStorageErrorBody {
|
||||
readonly message: string;
|
||||
readonly status: number;
|
||||
readonly code?: string;
|
||||
}
|
||||
Reference in new issue
Block a user