Files
iptvnator/libs/shared/database/src/lib/app-playlist-codec.ts
T
4grayandClaude Opus 5.5 b88a1c81e5 feat(self-hosted): server storage API in web-backend (#1898)
* feat(interfaces): add the server storage HTTP contract

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): read server storage configuration

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): open the server database and track revisions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): playlist and user-state repositories

Playlist documents use the shared desktop row codec, so the server writes
the same rows. keepPayload upserts take the stored playlist/items/header
fields when the client leaves them out. Xtream user data lives in the
server-only server_user_state table, one JSON value per namespace and
playlist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): storage error mapping and access token guard

Storage errors map to JSON status bodies and are logged by name and code
only. An optional bearer token guards every route except /, /health,
/config.js and /v1/instance, provider proxy routes included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): serve the /v1 server storage API

/v1/instance is always public and reports whether server storage is on.
With storage, /v1/playlists serves whole documents with ETags and 304s,
accepts keepPayload upserts, and deletes playlists with their user state;
/v1/user-state and /v1/app-state store the rest. The proxy routes are
unchanged unless an access token is configured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): open server storage on start and close on shutdown

With IPTVNATOR_STORAGE=server, main.ts opens the database before it
listens and exits with a clear message when the data directory is not
writable. SIGTERM and SIGINT close the server and the database. The
native better-sqlite3 module stays outside the esbuild bundle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): match weakened ETags on playlist reads

nginx weakens the ETag of a gzipped response, and clients send the weak
form back in If-None-Match. The route now compares tags weakly, as RFC 9110
requires for GET, so it answers 304 before reading the document.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(docker): persist server storage on a volume

The image builds better-sqlite3 in a native stage per target platform,
enables server storage on the /data volume, and raises the nginx body
limit to 64 MB for /api with gzip for JSON. The compose file mounts a
named volume and documents the optional access token. The PWA does not
use the storage yet; docs say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(web-backend): type the storage route responses

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): harden server storage reads, deletes and app state

- Metadata reads return the stored document without its payload fields,
  so fields that live only in the JSON survive a metadata round-trip.
- A conditional GET checks that the playlist exists before answering 304.
- Deleting a playlist and its user state commits in one transaction; the
  ETag moves only after the commit.
- Browser app state is stored under a client: prefix and the instance id
  is repaired when empty, so a client write cannot stop the backend.
- User-state buckets have no prototype, so ids like __proto__ round-trip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(self-hosted): keep the access token unset until the PWA sends it

Add the release note for the Docker server storage volume.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): bound shutdown and reject padded playlist ids

- SIGTERM closes idle connections at once and cuts the rest after five
  seconds, so a long provider download cannot keep the backend running.
- Playlist ids with surrounding whitespace are rejected: the row codec
  trims ids, so storage and ETags would track different keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): clear optional columns on whole-document upserts

The row codec maps empty values to undefined, which ON CONFLICT DO UPDATE
skips, so a cleared User-Agent, Referrer or credential kept its old column
value and came back on the next read. Server upserts now write NULL to
every replaceable column the document no longer sets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(self-hosted): linear token parsing, no-store app state, longer start

- The access guard reads the Bearer token by string scanning instead of a
  regular expression that CodeQL flagged as polynomial on long whitespace.
- GET /v1/app-state/:key sends Cache-Control: no-store like the other
  state routes, so devices never reuse a stale value.
- The entrypoint waits up to IPTVNATOR_BACKEND_START_TIMEOUT (300 s) for
  /health, because the backend migrates its database before it listens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(self-hosted): metadata reads skip payloads, redact JSON parse warnings

List playlist metadata from a server-only payload-free copy of each
document instead of running json_remove over every payload blob, and log
only the error name when a stored JSON value fails to parse, since the
parser's message quotes the stored text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): copy carried-over panel clock into playlist metadata

When an upsert keeps the stored serverTimezone, build the metadata copy
from the saved payload so metadata and full reads agree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:05:35 +02:00

269 lines
8.9 KiB
TypeScript

/**
* Row codec for the `playlists` table. The whole renderer `Playlist`
* document is stored as JSON in `payload`; the fields that metadata reads
* need are projected into columns. The Electron database worker and the
* self-hosted web backend both use this module, so both write the same rows.
*/
import * as schema from './schema';
export const PLAYLIST_TYPES = {
XTREAM: 'xtream',
STALKER: 'stalker',
M3U_FILE: 'm3u-file',
M3U_TEXT: 'm3u-text',
M3U_URL: 'm3u-url',
} as const;
export type AppPlaylistType =
(typeof PLAYLIST_TYPES)[keyof typeof PLAYLIST_TYPES];
const PLAYLIST_TYPE_VALUES = new Set<AppPlaylistType>(
Object.values(PLAYLIST_TYPES)
);
function getStringValue(value: unknown): string | undefined {
if (typeof value !== 'string') {
return undefined;
}
const normalized = value.trim();
return normalized.length > 0 ? normalized : undefined;
}
function getNumericValue(value: unknown): number | undefined {
if (typeof value === 'number' && Number.isFinite(value)) {
return value;
}
if (typeof value === 'string' && value.trim() !== '') {
const parsed = Number(value);
return Number.isFinite(parsed) ? parsed : undefined;
}
return undefined;
}
export function parseJsonValue<T>(
value: string | null | undefined,
fallback: T
): T {
if (!value) {
return fallback;
}
try {
return JSON.parse(value) as T;
} catch (error) {
// The parser's message quotes the stored text, which can hold
// portal credentials; the error name is enough to diagnose.
console.warn(
'Failed to parse JSON value from DB:',
error instanceof Error ? error.name : typeof error
);
return fallback;
}
}
function getStringArrayValue(value: unknown): string[] {
if (!Array.isArray(value)) {
return [];
}
return value
.map((item) => (typeof item === 'string' ? item.trim() : ''))
.filter((item) => item.length > 0);
}
function inferPlaylistType(playlist: Record<string, unknown>): AppPlaylistType {
const explicitType = getStringValue(playlist['type']);
if (
explicitType &&
PLAYLIST_TYPE_VALUES.has(explicitType as AppPlaylistType)
) {
return explicitType as AppPlaylistType;
}
if (getStringValue(playlist['serverUrl'])) {
return PLAYLIST_TYPES.XTREAM;
}
if (getStringValue(playlist['macAddress'])) {
return PLAYLIST_TYPES.STALKER;
}
if (getStringValue(playlist['filePath'])) {
return PLAYLIST_TYPES.M3U_FILE;
}
if (getStringValue(playlist['url'])) {
return PLAYLIST_TYPES.M3U_URL;
}
return PLAYLIST_TYPES.M3U_TEXT;
}
export function buildPlaylistRow(
playlist: Record<string, unknown>
): schema.NewPlaylist | null {
const id =
getStringValue(playlist['_id']) ?? getStringValue(playlist['id']);
if (!id) {
return null;
}
const type = inferPlaylistType(playlist);
const portalUrl = getStringValue(playlist['portalUrl']);
const url = getStringValue(playlist['url']);
const nowIso = new Date().toISOString();
const updateDate = getNumericValue(playlist['updateDate']);
return {
id,
name:
getStringValue(playlist['title']) ??
getStringValue(playlist['name']) ??
id,
serverUrl: getStringValue(playlist['serverUrl']),
username: getStringValue(playlist['username']),
password: getStringValue(playlist['password']),
lastUpdated:
typeof updateDate === 'number'
? new Date(updateDate).toISOString()
: getStringValue(playlist['lastUpdated']),
type,
userAgent: getStringValue(playlist['userAgent']),
origin: getStringValue(playlist['origin']),
referrer: getStringValue(playlist['referrer']),
filePath: getStringValue(playlist['filePath']),
epgUrls:
playlist['epgUrls'] !== undefined
? JSON.stringify(getStringArrayValue(playlist['epgUrls']))
: undefined,
detectedEpgUrls:
playlist['detectedEpgUrls'] !== undefined
? JSON.stringify(
getStringArrayValue(playlist['detectedEpgUrls'])
)
: undefined,
manualEpgUrls:
playlist['manualEpgUrls'] !== undefined
? JSON.stringify(getStringArrayValue(playlist['manualEpgUrls']))
: undefined,
disabledEpgUrls:
playlist['disabledEpgUrls'] !== undefined
? JSON.stringify(
getStringArrayValue(playlist['disabledEpgUrls'])
)
: undefined,
autoRefresh: Boolean(playlist['autoRefresh']),
macAddress: getStringValue(playlist['macAddress']),
url: type === PLAYLIST_TYPES.STALKER ? (portalUrl ?? url) : url,
portalUrl,
count: getNumericValue(playlist['count']),
importDate: getStringValue(playlist['importDate']),
updateDate,
position: getNumericValue(playlist['position']),
favorites:
playlist['favorites'] !== undefined
? JSON.stringify(playlist['favorites'])
: undefined,
recentlyViewed:
playlist['recentlyViewed'] !== undefined
? JSON.stringify(playlist['recentlyViewed'])
: undefined,
payload: JSON.stringify(playlist),
lastUsage: getStringValue(playlist['lastUsage']) ?? nowIso,
};
}
export function getPlaylistItemCount(
playlist: Record<string, unknown>
): number | undefined {
try {
const value = playlist['playlist'];
if (typeof value !== 'object' || value === null) {
return undefined;
}
const items = Reflect.get(value, 'items');
return Array.isArray(items) ? items.length : undefined;
} catch {
return undefined;
}
}
export function parseAppPlaylist(
row: schema.Playlist
): Record<string, unknown> {
const payload = parseJsonValue<Record<string, unknown> | null>(
row.payload,
null
);
const base = payload && typeof payload === 'object' ? payload : {};
const favorites = parseJsonValue<unknown[]>(row.favorites, []);
const recentlyViewed = parseJsonValue<unknown[]>(row.recentlyViewed, []);
const epgUrls = getStringArrayValue(
row.epgUrls
? parseJsonValue<unknown[]>(row.epgUrls, [])
: base['epgUrls']
);
const detectedEpgUrls = getStringArrayValue(
row.detectedEpgUrls
? parseJsonValue<unknown[]>(row.detectedEpgUrls, [])
: (base['detectedEpgUrls'] ?? epgUrls)
);
const manualEpgUrls = getStringArrayValue(
row.manualEpgUrls
? parseJsonValue<unknown[]>(row.manualEpgUrls, [])
: base['manualEpgUrls']
);
const disabledEpgUrls = getStringArrayValue(
row.disabledEpgUrls
? parseJsonValue<unknown[]>(row.disabledEpgUrls, [])
: base['disabledEpgUrls']
);
const importDate =
row.importDate ?? row.dateCreated ?? new Date().toISOString();
const portalUrl =
row.portalUrl ?? (row.type === PLAYLIST_TYPES.STALKER ? row.url : null);
const updateDate =
row.updateDate ??
(row.lastUpdated ? new Date(row.lastUpdated).getTime() : undefined);
return {
...base,
_id: row.id,
title:
getStringValue(base['title']) ??
getStringValue(base['name']) ??
row.name,
count: row.count ?? getNumericValue(base['count']) ?? 0,
importDate: getStringValue(base['importDate']) ?? importDate,
lastUsage:
row.lastUsage ??
getStringValue(base['lastUsage']) ??
getStringValue(base['importDate']) ??
importDate,
favorites,
recentlyViewed,
autoRefresh: row.autoRefresh ?? Boolean(base['autoRefresh']),
url:
row.type === PLAYLIST_TYPES.M3U_URL
? (row.url ?? getStringValue(base['url']))
: getStringValue(base['url']),
filePath: row.filePath ?? getStringValue(base['filePath']),
epgUrls,
detectedEpgUrls,
manualEpgUrls,
disabledEpgUrls,
userAgent: row.userAgent ?? getStringValue(base['userAgent']),
referrer: row.referrer ?? getStringValue(base['referrer']),
origin: row.origin ?? getStringValue(base['origin']),
updateDate,
position: row.position ?? getNumericValue(base['position']),
serverUrl: row.serverUrl ?? getStringValue(base['serverUrl']),
username: row.username ?? getStringValue(base['username']),
password: row.password ?? getStringValue(base['password']),
macAddress: row.macAddress ?? getStringValue(base['macAddress']),
portalUrl: portalUrl ?? getStringValue(base['portalUrl']),
};
}