mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-08 17:06:15 -08:00
4grayandClaude Fable 5.1
82295356ed
* perf(database): commit catalog writes in row-budgeted transactions Refreshing or deleting a large Xtream playlist spent most of its time in "Removing cached content": every 100 rows were deleted in their own transaction, so a 300k-row catalog cost ~3,000 commits, each flushing an FTS5 segment, re-appending dirty index pages to the WAL and, about every 4 MB, running an fsync-ing auto-checkpoint. Measured on a 900k-row copy of a real database the delete took 13 s where a single set-based statement takes 3 s, with 2 GB of WAL traffic instead of 140 MB. The re-import wrote its rows the same way. Deletes now read content row counts per category from the covering indexes, pack categories into groups of about 5,000 rows and issue one set-based DELETE per group, then drop the categories (and, for playlist removal, the user-data tables) with one scoped statement each. Inserts keep 100-row statements but commit fifty of them at a time. Cancellation still lands between commits and progress still reports after each one; the worker additionally throttles progress events to one per 100 ms with summed increments, so a large operation no longer floods the renderer. Same subset, same machine: 13.0 s -> 5.6 s for the delete, 16.7 s -> 8.6 s for the insert; the fsync-bound share is larger on Windows and spinning disks. Closes #1292 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(database): pin the row budget and the worker-side progress flush Review follow-up: the default 5,000-row budget and the 50-statement insert commit were only exercised with explicit overrides or sub-budget inputs, and nothing covered the worker controller flushing a coalesced progress report before its terminal event. Both are now pinned, and the docs no longer claim the insert path reports SQLite `changes` or binds 1,600 parameters per statement (it binds the eleven columns a value supplies). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(e2e): keep the stress suites on 100-row commits via a test-only budget knob The Xtream responsiveness and playlist-switcher suites slow the database worker down with IPTVNATOR_DB_WORKER_BATCH_DELAY_MS so they can observe an import mid-flight; the stress catalog is 1,920 rows per type, which at the production budget of 5,000 rows per commit is a single commit per type and too few progress events for their assertions. The new companion knob IPTVNATOR_DB_WORKER_ROWS_PER_TRANSACTION restores 100-row commits for those runs only; unset or invalid it leaves the default untouched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(database): scope refresh and cache-clear deletes to the captured category ids The row-budgeted rewrite deleted a refreshed playlist's categories with a playlist-wide predicate. The worker serves other requests between commits, so a newer import of the same playlist could create categories in that window and lose them to the older refresh, after which its content inserts fail their foreign keys. Count and delete now use the ids the collection step read, as the chunked code did; playlist removal keeps its playlist scope because its final playlist-row delete cascades the same set. deleteCategoriesWhere runs every filter through requireScopedFilter. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>