mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-16 22:26:16 -08:00
feat(self-hosted): server storage API in web-backend (#1898)
* feat(interfaces): add the server storage HTTP contract Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web-backend): read server storage configuration Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web-backend): open the server database and track revisions Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web-backend): playlist and user-state repositories Playlist documents use the shared desktop row codec, so the server writes the same rows. keepPayload upserts take the stored playlist/items/header fields when the client leaves them out. Xtream user data lives in the server-only server_user_state table, one JSON value per namespace and playlist. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web-backend): storage error mapping and access token guard Storage errors map to JSON status bodies and are logged by name and code only. An optional bearer token guards every route except /, /health, /config.js and /v1/instance, provider proxy routes included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web-backend): serve the /v1 server storage API /v1/instance is always public and reports whether server storage is on. With storage, /v1/playlists serves whole documents with ETags and 304s, accepts keepPayload upserts, and deletes playlists with their user state; /v1/user-state and /v1/app-state store the rest. The proxy routes are unchanged unless an access token is configured. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web-backend): open server storage on start and close on shutdown With IPTVNATOR_STORAGE=server, main.ts opens the database before it listens and exits with a clear message when the data directory is not writable. SIGTERM and SIGINT close the server and the database. The native better-sqlite3 module stays outside the esbuild bundle. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web-backend): match weakened ETags on playlist reads nginx weakens the ETag of a gzipped response, and clients send the weak form back in If-None-Match. The route now compares tags weakly, as RFC 9110 requires for GET, so it answers 304 before reading the document. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(docker): persist server storage on a volume The image builds better-sqlite3 in a native stage per target platform, enables server storage on the /data volume, and raises the nginx body limit to 64 MB for /api with gzip for JSON. The compose file mounts a named volume and documents the optional access token. The PWA does not use the storage yet; docs say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(web-backend): type the storage route responses Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web-backend): harden server storage reads, deletes and app state - Metadata reads return the stored document without its payload fields, so fields that live only in the JSON survive a metadata round-trip. - A conditional GET checks that the playlist exists before answering 304. - Deleting a playlist and its user state commits in one transaction; the ETag moves only after the commit. - Browser app state is stored under a client: prefix and the instance id is repaired when empty, so a client write cannot stop the backend. - User-state buckets have no prototype, so ids like __proto__ round-trip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(self-hosted): keep the access token unset until the PWA sends it Add the release note for the Docker server storage volume. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web-backend): bound shutdown and reject padded playlist ids - SIGTERM closes idle connections at once and cuts the rest after five seconds, so a long provider download cannot keep the backend running. - Playlist ids with surrounding whitespace are rejected: the row codec trims ids, so storage and ETags would track different keys. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web-backend): clear optional columns on whole-document upserts The row codec maps empty values to undefined, which ON CONFLICT DO UPDATE skips, so a cleared User-Agent, Referrer or credential kept its old column value and came back on the next read. Server upserts now write NULL to every replaceable column the document no longer sets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(self-hosted): linear token parsing, no-store app state, longer start - The access guard reads the Bearer token by string scanning instead of a regular expression that CodeQL flagged as polynomial on long whitespace. - GET /v1/app-state/:key sends Cache-Control: no-store like the other state routes, so devices never reuse a stale value. - The entrypoint waits up to IPTVNATOR_BACKEND_START_TIMEOUT (300 s) for /health, because the backend migrates its database before it listens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(self-hosted): metadata reads skip payloads, redact JSON parse warnings List playlist metadata from a server-only payload-free copy of each document instead of running json_remove over every payload blob, and log only the error name when a stored JSON value fails to parse, since the parser's message quotes the stored text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web-backend): copy carried-over panel clock into playlist metadata When an upsert keeps the stored serverTimezone, build the metadata copy from the saved payload so metadata and full reads agree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
4grayandClaude Opus 5.5 authored and GitHub committed
1 parent
4e9795ac54
commit
b88a1c81e5
41 files changed
+2312
-16
No files matched your search
+22
-1
@@ -28,6 +28,22 @@ RUN node tools/build/inject-build-commit.mjs
|
||||
RUN pnpm nx build web --configuration=pwa
|
||||
RUN pnpm nx build web-backend
|
||||
|
||||
# better-sqlite3 is a native addon. This stage is NOT pinned to
|
||||
# $BUILDPLATFORM, so multi-arch builds produce it for each target platform.
|
||||
# prebuild-install downloads a musl binary when one exists and otherwise
|
||||
# compiles with the toolchain below.
|
||||
FROM node:24-alpine AS native
|
||||
|
||||
RUN apk add --no-cache python3 make g++
|
||||
|
||||
WORKDIR /opt/native
|
||||
|
||||
COPY package.json ./root-package.json
|
||||
|
||||
RUN BETTER_SQLITE3_VERSION="$(node -p 'require("./root-package.json").dependencies["better-sqlite3"]')" \
|
||||
&& npm init -y >/dev/null \
|
||||
&& npm install --omit=dev --no-audit --no-fund "better-sqlite3@${BETTER_SQLITE3_VERSION}"
|
||||
|
||||
FROM node:24-alpine
|
||||
|
||||
RUN apk add --no-cache gettext nginx
|
||||
@@ -37,13 +53,18 @@ WORKDIR /opt/iptvnator
|
||||
ENV PORT=3000
|
||||
ENV BACKEND_URL=/api
|
||||
ENV CLIENT_URL=http://localhost:4333
|
||||
ENV IPTVNATOR_STORAGE=server
|
||||
ENV IPTVNATOR_DATA_DIR=/data
|
||||
|
||||
COPY --from=build /usr/src/app/dist/apps/web /usr/share/nginx/html
|
||||
COPY --from=build /usr/src/app/dist/apps/web-backend ./web-backend
|
||||
COPY --from=native /opt/native/node_modules ./web-backend/node_modules
|
||||
COPY docker/nginx.conf /etc/nginx/http.d/default.conf.template
|
||||
COPY docker/docker-entrypoint.sh /usr/local/bin/iptvnator-entrypoint
|
||||
|
||||
RUN chmod +x /usr/local/bin/iptvnator-entrypoint
|
||||
RUN chmod +x /usr/local/bin/iptvnator-entrypoint && mkdir -p /data
|
||||
|
||||
VOLUME ["/data"]
|
||||
|
||||
EXPOSE 80
|
||||
|
||||
|
||||
@@ -98,6 +98,11 @@ safe local defaults shown below.
|
||||
| `PORT` | `3000` | Internal Express backend port. nginx proxy config is rendered from the template to match it at startup. |
|
||||
| `IPTVNATOR_PROXY_ALLOW_PRIVATE_NETWORKS` | `0` | Set to `1` or `true` only for trusted local/LAN deployments that intentionally proxy private network IPTV or mock endpoints. |
|
||||
| `NODE_EXTRA_CA_CERTS` | unset | Optional Node.js CA bundle path for providers using private certificate authorities. Mount the CA file into the container and set this path. |
|
||||
| `IPTVNATOR_STORAGE` | `server` in the image | `server` opens the server storage database under `IPTVNATOR_DATA_DIR` and its `/api/v1` routes; `none` keeps the backend stateless. |
|
||||
| `IPTVNATOR_DATA_DIR` | `/data` in the image | Data root; the database is `<root>/databases/iptvnator.db`. Mount a volume here. |
|
||||
| `IPTVNATOR_ACCESS_TOKEN` | unset | When set, every route except `/`, `/health`, `/config.js` and `/v1/instance` requires `Authorization: Bearer <token>`, proxy routes included. The browser app cannot send a token yet: leave it unset until it can. |
|
||||
| `IPTVNATOR_MAX_DOCUMENT_MB` | `64` | Largest playlist document the backend accepts. Keep the nginx `client_max_body_size` in step. |
|
||||
| `IPTVNATOR_BACKEND_START_TIMEOUT` | `300` | Seconds the entrypoint waits for the backend's `/health` before giving up. Database migrations run before the backend listens. |
|
||||
|
||||
The web backend proxy accepts only `http` and `https` provider URLs. The PWA
|
||||
first registers provider URLs through `/provider-targets`, then uses the
|
||||
@@ -127,6 +132,22 @@ define a health check against `/api/health`. If nginx or the backend exits
|
||||
after startup, the entrypoint exits the container so Docker Compose can apply
|
||||
the `restart: unless-stopped` policy.
|
||||
|
||||
## Server Storage
|
||||
|
||||
The image opens a server storage database on the `/data` volume
|
||||
(`IPTVNATOR_STORAGE=server`, `IPTVNATOR_DATA_DIR=/data`) and serves its API
|
||||
under `/api/v1`. It is the backend half of keeping playlists, favorites,
|
||||
history and playback positions on the server for every device; the PWA
|
||||
starts using it in a follow-up change. Until then the volume holds an empty
|
||||
database and each browser keeps its own data.
|
||||
|
||||
Provider credentials will be stored unencrypted in that database, as the
|
||||
desktop app does. Do not expose an instance with server storage to the
|
||||
internet without authentication in your reverse proxy. The backend already
|
||||
supports `IPTVNATOR_ACCESS_TOKEN`, but the browser app cannot send the
|
||||
token yet, so setting it now makes the app unusable. Back up the volume like any database: stop the container, or copy
|
||||
`databases/iptvnator.db` together with its `-wal` file.
|
||||
|
||||
## IPv6, VPNs, And Connection Fallback
|
||||
|
||||
Node races IPv6 and IPv4 addresses when a provider hostname has both ("happy
|
||||
|
||||
@@ -13,9 +13,17 @@ services:
|
||||
CLIENT_URL: http://localhost:4333
|
||||
PORT: "3000"
|
||||
IPTVNATOR_PROXY_ALLOW_PRIVATE_NETWORKS: "0"
|
||||
# Server storage database on the /data volume. Set to "none" to keep
|
||||
# the backend stateless.
|
||||
IPTVNATOR_STORAGE: server
|
||||
volumes:
|
||||
- iptvnator-data:/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1/api/health >/dev/null || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
|
||||
volumes:
|
||||
iptvnator-data: {}
|
||||
@@ -60,8 +60,12 @@ trap handle_signal INT TERM
|
||||
trap cleanup EXIT
|
||||
|
||||
backend_ready=0
|
||||
# With server storage the backend migrates its database before it listens,
|
||||
# which can take a while on a large volume. The loop still exits at once if
|
||||
# the backend process dies.
|
||||
BACKEND_START_TIMEOUT="${IPTVNATOR_BACKEND_START_TIMEOUT:-300}"
|
||||
|
||||
for _ in $(seq 1 30); do
|
||||
for _ in $(seq 1 "$BACKEND_START_TIMEOUT"); do
|
||||
if wget -qO- "http://127.0.0.1:${PORT}/health" >/dev/null 2>&1; then
|
||||
backend_ready=1
|
||||
break
|
||||
@@ -79,7 +83,7 @@ for _ in $(seq 1 30); do
|
||||
done
|
||||
|
||||
if [ "$backend_ready" -ne 1 ]; then
|
||||
echo "IPTVnator web backend did not become healthy on port ${PORT}."
|
||||
echo "IPTVnator web backend did not become healthy on port ${PORT} within ${BACKEND_START_TIMEOUT} seconds (IPTVNATOR_BACKEND_START_TIMEOUT)."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
@@ -13,6 +13,14 @@ server {
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
# Playlist documents carry whole M3U payloads (default backend
|
||||
# limit IPTVNATOR_MAX_DOCUMENT_MB=64).
|
||||
client_max_body_size 64m;
|
||||
proxy_read_timeout 120s;
|
||||
gzip on;
|
||||
gzip_proxied any;
|
||||
gzip_types application/json;
|
||||
gzip_min_length 1024;
|
||||
}
|
||||
|
||||
location /assets/app-config.js {
|
||||
|
||||
Reference in new issue
Block a user