feat(self-hosted): server storage API in web-backend (#1898)

* feat(interfaces): add the server storage HTTP contract

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): read server storage configuration

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): open the server database and track revisions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): playlist and user-state repositories

Playlist documents use the shared desktop row codec, so the server writes
the same rows. keepPayload upserts take the stored playlist/items/header
fields when the client leaves them out. Xtream user data lives in the
server-only server_user_state table, one JSON value per namespace and
playlist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): storage error mapping and access token guard

Storage errors map to JSON status bodies and are logged by name and code
only. An optional bearer token guards every route except /, /health,
/config.js and /v1/instance, provider proxy routes included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): serve the /v1 server storage API

/v1/instance is always public and reports whether server storage is on.
With storage, /v1/playlists serves whole documents with ETags and 304s,
accepts keepPayload upserts, and deletes playlists with their user state;
/v1/user-state and /v1/app-state store the rest. The proxy routes are
unchanged unless an access token is configured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web-backend): open server storage on start and close on shutdown

With IPTVNATOR_STORAGE=server, main.ts opens the database before it
listens and exits with a clear message when the data directory is not
writable. SIGTERM and SIGINT close the server and the database. The
native better-sqlite3 module stays outside the esbuild bundle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): match weakened ETags on playlist reads

nginx weakens the ETag of a gzipped response, and clients send the weak
form back in If-None-Match. The route now compares tags weakly, as RFC 9110
requires for GET, so it answers 304 before reading the document.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(docker): persist server storage on a volume

The image builds better-sqlite3 in a native stage per target platform,
enables server storage on the /data volume, and raises the nginx body
limit to 64 MB for /api with gzip for JSON. The compose file mounts a
named volume and documents the optional access token. The PWA does not
use the storage yet; docs say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(web-backend): type the storage route responses

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): harden server storage reads, deletes and app state

- Metadata reads return the stored document without its payload fields,
  so fields that live only in the JSON survive a metadata round-trip.
- A conditional GET checks that the playlist exists before answering 304.
- Deleting a playlist and its user state commits in one transaction; the
  ETag moves only after the commit.
- Browser app state is stored under a client: prefix and the instance id
  is repaired when empty, so a client write cannot stop the backend.
- User-state buckets have no prototype, so ids like __proto__ round-trip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(self-hosted): keep the access token unset until the PWA sends it

Add the release note for the Docker server storage volume.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): bound shutdown and reject padded playlist ids

- SIGTERM closes idle connections at once and cuts the rest after five
  seconds, so a long provider download cannot keep the backend running.
- Playlist ids with surrounding whitespace are rejected: the row codec
  trims ids, so storage and ETags would track different keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): clear optional columns on whole-document upserts

The row codec maps empty values to undefined, which ON CONFLICT DO UPDATE
skips, so a cleared User-Agent, Referrer or credential kept its old column
value and came back on the next read. Server upserts now write NULL to
every replaceable column the document no longer sets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(self-hosted): linear token parsing, no-store app state, longer start

- The access guard reads the Bearer token by string scanning instead of a
  regular expression that CodeQL flagged as polynomial on long whitespace.
- GET /v1/app-state/:key sends Cache-Control: no-store like the other
  state routes, so devices never reuse a stale value.
- The entrypoint waits up to IPTVNATOR_BACKEND_START_TIMEOUT (300 s) for
  /health, because the backend migrates its database before it listens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(self-hosted): metadata reads skip payloads, redact JSON parse warnings

List playlist metadata from a server-only payload-free copy of each
document instead of running json_remove over every payload blob, and log
only the error name when a stored JSON value fails to parse, since the
parser's message quotes the stored text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web-backend): copy carried-over panel clock into playlist metadata

When an upsert keeps the stored serverTimezone, build the metadata copy
from the saved payload so metadata and full reads agree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
4grayandClaude Opus 5.5 authored and GitHub committed 2026-10-12 00:05:35 +02:00
1 parent 4e9795ac54
commit b88a1c81e5
41 files changed
+2312 -16

No files matched your search

@@ -108,4 +108,29 @@ describe('app playlist codec', () => {
it('refuses a document without an id', () => {
expect(buildPlaylistRow({ title: 'No id' })).toBeNull();
});
it('never logs the content of a malformed stored value', () => {
const warn = jest.spyOn(console, 'warn').mockImplementation(() => {
/* silenced */
});
try {
const row = rowOf(m3uDocument);
parseAppPlaylist(
storedRow(row, '{"title":"x","password":hunter2-secret}')
);
const logged = warn.mock.calls
.flat()
.map((arg) =>
arg instanceof Error
? `${arg.name}: ${arg.message} ${arg.stack}`
: String(arg)
)
.join(' ');
expect(warn).toHaveBeenCalled();
expect(logged).not.toContain('hunter2');
} finally {
warn.mockRestore();
}
});
});
@@ -53,7 +53,12 @@ export function parseJsonValue<T>(
try {
return JSON.parse(value) as T;
} catch (error) {
console.warn('Failed to parse JSON value from DB:', error);
// The parser's message quotes the stored text, which can hold
// portal credentials; the error name is enough to diagnose.
console.warn(
'Failed to parse JSON value from DB:',
error instanceof Error ? error.name : typeof error
);
return fallback;
}
}