mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-08 17:06:15 -08:00
fix(downloads): reconnect interrupted transfers and resume without validators (#1446)
* fix(downloads): reconnect interrupted transfers and resume without validators Xtream panels commonly kill each VOD connection after a byte/time burst (~130-260 MB) and send no ETag/Last-Modified. The validator-only resume path then deleted the partial and surfaced a raw "aborted" failure, so every Retry restarted from zero and large files could never finish. - Resume without a validator through overlap verification: the Range request rewinds by 256 KiB and the replayed window must match the partial's tail byte-for-byte before anything is appended; a mismatch truncates the partial and restarts from scratch (download-overlap.ts). - Reconnect automatically on recoverable interruptions and clean short responses (download-reconnect.ts): progress >=64 KiB past the best attempt resets a 3-stall budget; request-phase failures during reconnects are converted into retained interruptions so automation can never delete a partial. - Extract pure response-header helpers into download-resume-validation.ts to keep download-transfer.ts within the file-size guideline. Verified against a real throttling portal: a 1.6 GB and a 3 GB movie completed through 15 and 23 connection resets respectively. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): address review findings on reconnect baseline and small partials - Judge reconnect progress against the previous attempt instead of a high-water mark, so a transfer that legitimately restarted from byte zero mid-loop (overlap mismatch, ignored Range) is measured by its rebuilt file; at most two such regressions are tolerated per transfer to keep the loop structurally bounded (Greptile P1). - Floor reported progress at the partial's retained size while appending, so a response that ends inside the overlap window can never move persisted progress backwards (Greptile P1). - Verify partials smaller than the overlap window in full from byte zero and append, instead of rewriting the .part in place — an early-dying reconnect can now only grow the file (Codex P2). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): keep tolerated regressions off the reconnect stall budget A tolerated restart regression consumed a regression credit AND counted as a stalled attempt (its negative delta is below the progress threshold), so a legitimately rebuilt file that grows in sub-64 KiB steps was failed one reconnect early. Regressions are now charged to their own bounded budget only; the stall budget stays reserved for attempts that genuinely fail to grow the file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): gate success and validator promotion on complete overlap verification Round-3 review findings: - Success is rejected while the overlap verifier has not consumed its entire window: a complete 206 that ends inside the window proves the remote entity shrank, so the transfer truncates and restarts from scratch instead of finalizing the old suffix as a completed file; an early-dying stream stays an ordinary retained interruption (Codex P1). - A verify-append attempt promotes the response's ETag/Last-Modified only after the complete overlap matched — an unverified partial is never blessed with a validator the next resume would If-Range-append onto (Codex P1). - A tolerated restart regression resets stalls accumulated against the discarded representation, so the rebuilt file starts with the full stall budget (Greptile P1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): carry the known total through total-less reconnect responses A resumed response without a usable total (chunked, or an unsatisfiable Content-Range) erased task.totalBytes, so a reset over that response could no longer classify as a retained interruption and generic cleanup deleted the verified partial. The previously learned total is now carried forward for appending attempts; fresh and restarted transfers still drop it, since it described a discarded file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): never fabricate a total for an unverified retained partial A retained failure with an unknown total persisted totalBytes = bytesDownloaded, so after stalled reconnects over a chunked, validator-less response that kept ending inside the overlap window, Retry's completed-partial shortcut saw the .part size equal the fabricated total and finalized the unverified partial without a request. The fallback is now explicit per call site: only a finalization failure after a complete transfer records its byte count as the total; retained interruptions keep an unknown total unknown, forcing Retry to re-verify. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): treat an unsatisfiable resume range as a representation change A range-capable server whose entity shrank below the rewound overlap offset answers 416 before any response body exists, which rejected the request into the generic partial-deleting failure path. The 416 is now recognized as a representation change: the partial is truncated and the transfer restarts against the current entity from byte zero. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): treat an indeterminate Content-Range total as unknown `Content-Range: bytes 200-299/*` fell through to the Content-Length fallback, deriving a "total" equal to the end of the selected range — a resumed response ending there was declared complete and the truncated partial finalized. An indeterminate total now yields null, letting the previously known total carry forward and classify the short response as a retained truncation instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): signal restarts explicitly and keep carried totals informational Round-8 review findings, both fixed at the design level instead of another byte-comparison patch: - Restart epochs (Greptile P1): the transfer layer now reports every rewrite-from-zero via task.transferRestarts (overlap mismatch, shrunk entity, 416, ignored Range), and the reconnect loop opens a fresh progress epoch on that signal — clean stall budget, no baseline. Byte inference could not recognize a rebuild landing near the previous attempt's count; the explicit signal can. Two restarts are tolerated per transfer; an unsignalled regression is an ordinary stall. - Authoritative vs informational totals (Codex P1): completion and truncation decisions now use only the response's own total or its advertised indeterminate range end (`bytes X-Y/*` -> Y+1); a carried total is informational, is dropped once the bytes on disk falsify it, and can flag a short transfer but never authorize finalization. A mid-reset 206 retains the partial even with a falsified total — the response proved range capability — persisting the total as unknown. Also raises the resume spec's jest timeout and tightens its polling: the previous 5 s default flaked on starved CI runners. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): keep unproven state fully uncommitted across the resume model Round-9 review findings, closing the remaining commit-before-proof gaps: - The response's total now stays uncommitted (task and row) until the complete overlap matched, exactly like the validator: a persisted total equal to the unverified partial's size let the completed-partial shortcut finalize unproven bytes after a pause, crash, or retained failure (Codex P1). - Retained-interruption persistence syncs the live task with the row: a stale falsified total left in memory made the next reconnect's resume-offset guard reject the retained partial into generic cleanup (Codex P1). - An observed 206 is remembered as task.serverAcceptsRanges, so a request-phase failure (no response at all) can retain an unknown-length partial on that evidence instead of deleting it (Codex P2). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): treat a reset after the final ranged byte as completion A 206 that delivers every advertised byte but ends in ECONNRESET instead of a clean close was classified as an interruption with a falsified total; the reconnect then resumed at EOF, collected a 416, and truncated the complete file — an endpoint that always resets after its last byte could never finish. A retainable failure whose on-disk bytes exactly equal the response's own completion evidence (and the proven total, with the overlap verified) now completes the transfer. Strict equality keeps oversized partials on the generic-failure path. Also splits the transfer error classes and retention classification into download-transfer-errors.ts to stay under the max-lines rule. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): never let an indeterminate range end authorize completion Reaching Y of `Content-Range: bytes X-Y/*` proves the selected range was delivered, not that the entity ends there — a range-capping server resetting at its cap would have finalized a truncated movie as complete. The reset-after-final-byte completion now requires the response's authoritative total; indeterminate range ends keep flagging short delivery but such resets stay retained interruptions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): retain any nonempty partial and keep indeterminate ranges incomplete Round-12 findings, resolved by removing the last evidence requirements: - Retention no longer needs a total, validator, or range proof: since overlap verification owns resume correctness, the next attempt can safely prove, resume, or restart over ANY retained partial — deleting bytes is the only unrecoverable outcome. This closes the whole family (refused reconnects, chunked responses, falsified totals, and reconstructed retry tasks losing the in-memory range flag) and removes task.serverAcceptsRanges entirely (Codex P2). - A clean EOF at the advertised end of an indeterminate range (`bytes X-Y/*`) stays incomplete, matching the reset path: reaching Y proves the range was delivered, not that the entity ends there, so a range-capping server can no longer finalize a truncated movie; the shrunk-entity restart likewise requires an authoritative total (Codex P1). Responses with no range and no total keep the clean-EOF completion contract of unknown-length HTTP. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): honor 416-confirmed completion and keep proven validators Round-13 findings plus a CI limit: - A 416 whose `Content-Range: bytes */N` equals the partial's size confirms the file IS the complete entity (under If-Range a validator mismatch yields 200, so the 416 also confirms identity): finalize it instead of truncating and redownloading forever (Codex P1). - A validator proven by a complete overlap match is now promoted on the error path too, and retained-failure/pause persistence write resume_validator from the task — later attempts resume via If-Range instead of replaying the 256 KiB window, which stalled out servers whose per-connection cap barely exceeds the window (Codex P1). - download-resume.spec.ts crossed the 1200-line test limit: the shared harness moves to download-resume.test-harness.ts and the overlap-family cases into download-overlap-resume.spec.ts; the runtime spec gets the same CI-load timeout headroom as the resume specs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): settle falsified totals on clean exits and probe EOF after zero-growth replays Round-14 review findings plus the CI build break: - download-resume.test-harness.ts was outside the tsconfig test-helpers exclude glob and broke every app typecheck/build; renamed to download-resume.test-helpers.ts (the excluded pattern). - A clean indeterminate delivery that outgrows a stale carried total now settles that total to unknown on the row AND the live task before persisting or throwing — the reconnect's resume-offset guard would otherwise reject the retained partial into generic cleanup (Codex P1). - A verified overlap replay of an indeterminate range that appends nothing arms a one-shot EOF probe: the next attempt requests the byte after the partial so a compliant 416 (bytes */N) can confirm the file is the complete entity, instead of repeating the rewind until the stall budget fails a finished download; a probe answered with more data is retired unappended and rewound verification resumes (Codex P1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): retain the partial on an inconclusive EOF-probe 416 A probe at the partial's exact end always collects a 416 when the entity ends there, and the confirming Content-Range length is optional — so a length-less 416 is equally consistent with a complete file, and the unconditional restart redownloaded a likely finished movie every cycle. The probe's 416 now restarts only when a stated total BELOW the partial proves the entity shrank; otherwise the partial is retained. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): treat any EOF-offset 416 as inconclusive, not just the probe A validator-backed resume at the partial's exact end IS an EOF request: when its 416 arrived without the optional Content-Range length, the probingEof gate saw false and restartFromScratch truncated the complete partial. The inconclusive-416 retention now keys on the request having started at the partial's end (covering the probe and every If-Range resume alike); a restart still requires a stated total below the partial. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): contradiction-proof carried totals and identity-gated 416 completion Round-17 review findings: - A carried total now falls the moment the response's advertised range end contradicts it, before any byte lands — waiting for the bytes to reach it left a pause/exit window where an N/N row let the completed-partial shortcut finalize a truncated file (Codex P1). - The 416 completion shortcut now requires identity proof: an exact-EOF request backed by If-Range, or the EOF probe that follows a fully verified overlap replay. A bare length match on a rewound request proves nothing about whose bytes are on disk; a contradictory 416 (stated total says the rewound range was satisfiable) retains instead of restarting (Codex P2). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(downloads): extract 416 classification to stay under max-lines The previous commit pushed download-transfer.ts to 404 effective lines; the 416 decision moves into classifyRangeNotSatisfiable() in download-transfer-errors.ts with identical semantics, and the unused re-export-only imports are dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): promote the proven total on the error path and fix the 416 contract doc - The error-path promotion after a complete overlap match now updates task.totalBytes alongside the validator: a pause landing while the partial sits at a stale carried total otherwise persisted an N/N row that Resume's completed-partial shortcut would finalize (Codex P1). - The download-manager contract doc's rewound-416 paragraph now matches classifyRangeNotSatisfiable(): completion requires identity proof at exact EOF, restart requires a proven shrink, everything ambiguous retains (Codex P1 on the doc). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): correct both 416 classification boundaries Round-19 review findings, one in each direction: - A rewound 416 WITHOUT a stated length now retains: unsatisfiability alone never proves the entity shrank relative to the retained bytes, and the canonical contract reserves restart for stated proof (Codex P1). - A stated total EQUAL to a rewound request's first byte now restarts: the entity ending exactly at the rewound offset makes the 416 valid and proves the partial extends past the entity — equality was being misread as a contradiction, stranding the download in retain forever (Codex P1). classifyRangeNotSatisfiable() gains an exhaustive pure table spec covering every branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): restart on reset-ended responses that completed a shorter entity in-window The clean-EOF path already restarted when a response delivered its complete authoritative total inside the verification window, but a reset arriving right after that final byte took the retention path and stranded the oversized partial in a stall loop. The catch path now mirrors the shrink detection: authoritative total reached inside an unproven overlap restarts from scratch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): request identity encoding for byte-exact transfers Axios's Node adapter transparently decodes gzip/brotli responses, which would put decoded bytes on disk while Content-Length, Content-Range, and every Range offset speak the encoded representation — desyncing resume offsets and overlap verification on origins that compress. Downloads now always send Accept-Encoding: identity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): drop carried totals an indeterminate range can exactly reach `bytes 200-249/*` can deliver the partial exactly TO a carried total of 250; the <= guard kept that total, and a pause or exit anywhere in that window persisted a 250/250 row the completed-partial shortcut would finalize without EOF proof. The guard is now strict: a carried total survives only when the advertised indeterminate range cannot reach it, which also closes the mid-stream pause window. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): arm the EOF probe after a reset-ended verified zero-growth replay A verified overlap replay that reset right at the partial's end (zero growth, indeterminate range, no validator) retained without arming the EOF probe, so every retry replayed the same tail until the stall budget expired — the clean-EOF path's probe arming now has its reset-path mirror. Also splits to stay under max-lines: the classifyRangeNotSatisfiable table spec moves to download-transfer-errors.spec.ts and the DB persist helpers to download-transfer-persistence.ts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
0a2fe263db
commit
00aa623b83
20 files changed
+3347
-419
No files matched your search
@@ -1136,10 +1136,26 @@ engine` (restart required) or
|
||||
legacy row whose playlist is already absent receives the same IPTV-player
|
||||
fallback; a known Stalker row remains unchanged. Allowlisted connection
|
||||
resets after bytes reach disk retain the partial and show a credential-safe
|
||||
`DOWNLOAD_NETWORK_INTERRUPTED` code only when the response supplied a strong
|
||||
ETag or Last-Modified validator. Retry then continues with Range/If-Range;
|
||||
without a validator it starts from byte zero and overwrites the unverified
|
||||
partial instead of risking mixed-representation corruption.
|
||||
`DOWNLOAD_NETWORK_INTERRUPTED` code. Retry resumes with Range/If-Range when
|
||||
the response supplied a strong ETag or Last-Modified validator; without one
|
||||
the Range request rewinds by a 256 KiB overlap window whose bytes must match
|
||||
the partial's tail before anything is appended (`download-overlap.ts`); a
|
||||
smaller partial is verified in full from byte zero and appended, never
|
||||
rewritten in place; reported progress is floored at the retained size while
|
||||
appending; and a mismatch truncates the partial and restarts from byte zero
|
||||
instead of risking mixed-representation corruption. The runtime also
|
||||
reconnects interrupted transfers automatically (`download-reconnect.ts`):
|
||||
reconnects continue while attempts end ≥64 KiB past the previous attempt;
|
||||
restarts are an explicit `task.transferRestarts` signal (never byte
|
||||
inference) that opens a fresh progress epoch, at most twice per transfer;
|
||||
three consecutive stalled attempts surface the retained failure; and a
|
||||
reconnect that fails before any response is converted into the same
|
||||
retained interruption so it can never delete the partial. Only the response's own
|
||||
total authorizes completion — an indeterminate `bytes X-Y/*` range stays
|
||||
incomplete even at a clean EOF; carried totals are informational and
|
||||
dropped when falsified; and any retainable network failure retains any
|
||||
nonempty partial (no evidence required), persisting a falsified total as
|
||||
unknown.
|
||||
- The desktop-only manager shares one global download store across the global,
|
||||
Xtream-scoped, and Stalker-scoped routes. Completed movie and grouped-series
|
||||
cards use the global Small/Medium/Large cover-grid tokens; missing completed
|
||||
|
||||
Reference in new issue
Block a user