feat(downloads): download completed Xtream catch-up programmes as TS (#1572)

* feat(epg): copy catch-up programme URLs without changing playback

* feat(downloads): save completed Xtream archive programmes as TS

* fix(epg): let newer archive copy requests supersede pending work

* fix(downloads): protect archive partials and independent submissions

* fix(downloads): verify archive identity through finalization

* fix(downloads): bound archive storage and capture cleanup entries

* fix(downloads): preserve archive ownership across failure paths

* fix(downloads): recover explicitly verified archive completions

* fix(downloads): journal archive promotion before publishing files

* fix(downloads): reset archive proof before an explicit restart

* fix(downloads): preserve archive recovery ownership and interruption

* fix(downloads): verify durable archive identity at resume open

* fix(downloads): fence archive commands during completion commit

* fix(downloads): persist archive ownership throughout its lifecycle

* fix(downloads): protect archive removal and missing-file recovery

* fix(downloads): journal private cleanup captures for recovery

* fix(downloads): journal active archive cleanup before removal

* fix(downloads): clean settled archives before deleting stale rows

* fix(downloads): preserve archive ownership on removal and resubmission

* fix(downloads): recover proven archive completions before retry

* fix(downloads): recover local archives before remote transfer checks

* test(downloads): resolve archive fixture from workspace root

* fix(downloads): distinguish reused archive inodes by creation time

* fix(downloads): bind fresh archive reservations to owned files

* fix(downloads): clean reservations when ownership writes fail

* fix(downloads): commit archive reservation and ownership atomically

* fix(downloads): retain captures until replacement restoration succeeds

* fix(downloads): require durable ownership before cleanup relocation

* fix(downloads): preserve 64-bit archive file identities on Windows

* refactor(release): keep capture fixture constants in their shared module

* fix(downloads): preserve the last link of captured foreign files

* fix(downloads): expose retained archive recovery files

* fix(downloads): keep recovery instructions open while copying
This commit is contained in:
4gray authored and GitHub committed 2026-09-08 20:33:05 +02:00
1 parent a7f3860102
commit bad8a0991e
117 files changed
+7510 -880

No files matched your search

+6
View File
@@ -0,0 +1,6 @@
---
type: feature
area: downloads
---
Download completed Xtream catch-up programmes as TS files from Live TV programme details. Find them in Downloads with the channel and broadcast date and play them offline. Interrupted archive downloads restart from the beginning.
+34
View File
@@ -1045,3 +1045,37 @@ Xtream/M3U archives, including Favorites/Recent. `EpgArchiveCopyService` owns
clipboard feedback; hosts resolve URLs without mutating playback. Stalker and
the currently non-catch-up M3U guide expose no action. See
`docs/architecture/m3u-playlist-module.md` (Copy archive URL).
## Xtream Archive Downloads
Desktop Xtream Live TV programme details can enqueue completed catch-up as
`contentType: catchup`. The queue uses the existing timeshift resolver, original
timestamps and playback headers. `programme_start` plus playlist/channel provides
identity; JSON `catchup` metadata retains channel, broadcast window and known
expiry. `download-schema.ts` owns the transactional CHECK/index migration;
`download-tables.ts` exports the download tables. The cascading
`download_archive_finalizations` table records write-ahead file identity/size
proof before promotion (before writing a fallback copy), allowing startup to
recover completed unknown-length archives and clean only their owned partials.
The same journal stores transfer-phase descriptor identity before truncation;
Resume checks it at open, and rejected replacements are preserved and detached
so Retry can reserve a fresh path. A synchronous completion-commit boundary
rejects late pause/cancel commands before awaited cleanup and persistence.
Archive ownership reads device/inode as BigInt and journals decimal strings
without losing 64-bit Windows file references, alongside positive creation time to reject
reused inodes after unlink; old proofs without creation time remain untrusted.
Fresh reservations atomically commit their row path/name and captured ownership
before the initial HTTP wait;
no preexisting partial is truncated without matching expected ownership.
Captured foreign files retain their recovery copy and journal even after public
restoration, until the user explicitly removes the recovery copy. Remove/Clear
show its full path and recovery instructions in a persistent dialog with Copy
recovery path.
Private cleanup captures are journaled before relocation, keeping failed
Remove/Clear/cancel cleanup retryable across restarts without hardlinks. Active
failures, promotion and startup share that cleanup; Remove waits for active
archive cancellation to settle before deleting its row and journal.
Archive transfers validate TS framing, restart from byte zero after interruption
and check expiry again at transfer start. Completed cards play locally and never
route to VOD details. Contract and EOF/duration limits:
`docs/architecture/download-manager.md` (Xtream archive downloads).
+34
View File
@@ -1856,3 +1856,37 @@ Xtream/M3U archives, including Favorites/Recent. `EpgArchiveCopyService` owns
clipboard feedback; hosts resolve URLs without mutating playback. Stalker and
the currently non-catch-up M3U guide expose no action. See
`docs/architecture/m3u-playlist-module.md` (Copy archive URL).
## Xtream Archive Downloads
Desktop Xtream Live TV programme details can enqueue completed catch-up as
`contentType: catchup`. The queue uses the existing timeshift resolver, original
timestamps and playback headers. `programme_start` plus playlist/channel provides
identity; JSON `catchup` metadata retains channel, broadcast window and known
expiry. `download-schema.ts` owns the transactional CHECK/index migration;
`download-tables.ts` exports the download tables. The cascading
`download_archive_finalizations` table records write-ahead file identity/size
proof before promotion (before writing a fallback copy), allowing startup to
recover completed unknown-length archives and clean only their owned partials.
The same journal stores transfer-phase descriptor identity before truncation;
Resume checks it at open, and rejected replacements are preserved and detached
so Retry can reserve a fresh path. A synchronous completion-commit boundary
rejects late pause/cancel commands before awaited cleanup and persistence.
Archive ownership reads device/inode as BigInt and journals decimal strings
without losing 64-bit Windows file references, alongside positive creation time to reject
reused inodes after unlink; old proofs without creation time remain untrusted.
Fresh reservations atomically commit their row path/name and captured ownership
before the initial HTTP wait;
no preexisting partial is truncated without matching expected ownership.
Captured foreign files retain their recovery copy and journal even after public
restoration, until the user explicitly removes the recovery copy. Remove/Clear
show its full path and recovery instructions in a persistent dialog with Copy
recovery path.
Private cleanup captures are journaled before relocation, keeping failed
Remove/Clear/cancel cleanup retryable across restarts without hardlinks. Active
failures, promotion and startup share that cleanup; Remove waits for active
archive cancellation to settle before deleting its row and journal.
Archive transfers validate TS framing, restart from byte zero after interruption
and check expiry again at transfer start. Completed cards play locally and never
route to VOD details. Contract and EOF/duration limits:
`docs/architecture/download-manager.md` (Xtream archive downloads).
@@ -1,3 +1,17 @@
import {
getDownloadPlayPaths,
installDownloadPlayCapture,
} from './downloads.e2e-support';
import {
mkdirSync,
linkSync,
readFileSync,
readdirSync,
renameSync,
unlinkSync,
writeFileSync,
} from 'node:fs';
import { join } from 'node:path';
import type { Page } from '@playwright/test';
import {
addXtreamPortal,
@@ -15,6 +29,7 @@ import {
switchUnifiedCollectionScope,
test,
waitForXtreamWorkspaceReady,
workspaceRoot,
} from './electron-test-fixtures';
import { fetchXtreamEpgFixture } from './portal-mock-fixtures';
@@ -308,3 +323,462 @@ test('@epg @xtream @electron copies the archive URL without starting archive pla
await closeElectronApp(app);
}
});
test('@downloads @epg @xtream @electron downloads a completed archive into the local library', async ({
dataDir,
request,
}) => {
test.setTimeout(120000);
const credentials = { username: 'epg', password: 'epg' };
await resetMockServers(request, ['xtream']);
const fixture = await fetchXtreamEpgFixture(request, credentials);
let app = await launchElectronApp(dataDir, {
env: { TZ: VIEWER_TIMEZONE },
});
try {
const folder = join(dataDir, 'archive-downloads');
mkdirSync(folder, { recursive: true });
await app.electronApp.evaluate(({ dialog }, target) => {
dialog.showOpenDialog = async () =>
({ canceled: false, filePaths: [target] }) as Awaited<
ReturnType<typeof dialog.showOpenDialog>
>;
}, folder);
await app.mainWindow.evaluate(async () => {
await window.electron.downloadsSelectFolder();
});
await addXtreamPortal(app.mainWindow, {
name: 'Archive downloads',
...credentials,
});
await waitForXtreamWorkspaceReady(app.mainWindow);
await openTimezoneNewsInLiveTv(app.mainWindow, fixture.categoryName);
const captured = captureTimeshiftRequests(app.mainWindow);
const block = app.mainWindow
.locator('app-epg-timeline .epg-timeline__block')
.filter({ hasText: PAST_PROGRAM })
.first();
await expect(block).toBeVisible({ timeout: 20000 });
await block.locator('.epg-timeline__info').click();
await app.mainWindow
.getByRole('dialog')
.getByRole('button', {
name: 'Download programme (TS)',
exact: true,
})
.click();
await expect(
app.mainWindow.getByText('Programme added to Downloads', {
exact: true,
})
).toBeVisible();
await expect
.poll(
async () => {
const rows = await app.mainWindow.evaluate(() =>
window.electron.downloadsGetList()
);
return rows.find((row) => row.contentType === 'catchup')
?.status;
},
{ timeout: 30000 }
)
.toBe('completed');
const rows = await app.mainWindow.evaluate(() =>
window.electron.downloadsGetList()
);
const row = rows.find((entry) => entry.contentType === 'catchup');
expect(row?.catchup?.channelName).toBe(CHANNEL);
expect(row?.title).toBe(PAST_PROGRAM);
expect(row?.filePath).toMatch(/archive-downloads.*\.ts$/);
if (!row?.filePath) throw new Error('No archive file');
expect(readFileSync(row.filePath)).toEqual(
readFileSync(
join(
workspaceRoot,
'apps/xtream-mock-server/src/fixtures/live.mpegts'
)
)
);
expect(captured).toEqual([]);
const ownership = await app.electronApp.evaluate(
(_electron, { dependency, file, id, mediaPath }) => {
const Database = process
.getBuiltinModule('module')
.createRequire(dependency)(dependency);
const db = new Database(file);
try {
const proof = JSON.parse(
db
.prepare(
'SELECT proof FROM download_archive_finalizations WHERE download_id=?'
)
.get(id).proof
);
const stats = process
.getBuiltinModule('fs')
.lstatSync(mediaPath, { bigint: true });
return {
recorded: proof.finalIdentity,
actual: {
dev: String(stats.dev),
ino: String(stats.ino),
},
};
} finally {
db.close();
}
},
{
dependency: join(workspaceRoot, 'node_modules/better-sqlite3'),
file: join(dataDir, 'databases/iptvnator.db'),
id: row.id,
mediaPath: row.filePath,
}
);
expect(ownership.recorded).toMatchObject(ownership.actual);
// A failed completion status write must recover the proven file in place
// on a repeated EPG submission, without a duplicate transfer.
await app.electronApp.evaluate(
(_electron, { dependency, file, id }) => {
const Database = process
.getBuiltinModule('module')
.createRequire(dependency)(dependency);
const db = new Database(file);
try {
db.prepare(
"UPDATE downloads SET status='failed' WHERE id=?"
).run(id);
} finally {
db.close();
}
},
{
dependency: join(workspaceRoot, 'node_modules/better-sqlite3'),
file: join(dataDir, 'databases/iptvnator.db'),
id: row.id,
}
);
await block.locator('.epg-timeline__info').click();
await app.mainWindow
.getByRole('dialog')
.getByRole('button', {
name: 'Download programme (TS)',
exact: true,
})
.click();
await expect(
app.mainWindow.getByText(
'This programme is already in Downloads.',
{ exact: true }
)
).toBeVisible();
expect(
await app.mainWindow.evaluate(
async () => (await window.electron.downloadsGetList()).length
)
).toBe(1);
// Submitting the programme again after its file disappears must use
// journal ownership too, not delete a replacement at the old .part path.
const firstPath = row.filePath;
unlinkSync(firstPath);
writeFileSync(firstPath + '.part', 'unrelated submission partial');
await block.locator('.epg-timeline__info').click();
await app.mainWindow
.getByRole('dialog')
.getByRole('button', {
name: 'Download programme (TS)',
exact: true,
})
.click();
await expect
.poll(async () => {
const current = (
await app.mainWindow.evaluate(() =>
window.electron.downloadsGetList()
)
).find((entry) => entry.id === row.id);
return (
current?.status === 'completed' &&
!!current.filePath &&
current.filePath !== firstPath
);
})
.toBe(true);
const resubmitted = (
await app.mainWindow.evaluate(() =>
window.electron.downloadsGetList()
)
).find((entry) => entry.id === row.id);
if (!resubmitted?.filePath)
throw new Error('Missing resubmitted archive');
expect(resubmitted.filePath).not.toBe(firstPath);
expect(readFileSync(firstPath + '.part', 'utf8')).toBe(
'unrelated submission partial'
);
row.filePath = resubmitted.filePath;
expect(readFileSync(row.filePath)).toEqual(
readFileSync(
join(
workspaceRoot,
'apps/xtream-mock-server/src/fixtures/live.mpegts'
)
)
);
await app.mainWindow
.getByRole('button', { name: 'Open downloads', exact: true })
.click();
const card = app.mainWindow.getByTestId(
`download-library-catchup-${row.id}`
);
await expect(card).toBeVisible();
await expect(card).toContainText('TV programme');
await expect(card).toContainText(CHANNEL);
await app.mainWindow.screenshot({
path: test.info().outputPath('archive-download-library.png'),
});
await installDownloadPlayCapture(app);
await card
.getByRole('button', { name: `Play: ${PAST_PROGRAM}`, exact: true })
.click();
await expect
.poll(() => getDownloadPlayPaths(app))
.toEqual([row.filePath]);
await expect(app.mainWindow).toHaveURL(
/\/workspace\/downloads(?:\?.*)?$/
);
// Model termination after verified promotion but before the completion
// DB write, including a response without Content-Length. Keep only the
// real SQLite journal written by the transfer; the next process has no task.
const durableProof = await app.electronApp.evaluate(
(_electron, { dependency, file, id }) => {
const Database = process
.getBuiltinModule('module')
.createRequire(dependency)(dependency);
const db = new Database(file);
try {
const journal = db
.prepare(
'SELECT proof FROM download_archive_finalizations WHERE download_id=?'
)
.get(id) as { proof: string } | undefined;
if (!journal)
throw new Error(
'Archive promotion journal was not persisted'
);
db.prepare(
"UPDATE downloads SET status='downloading', total_bytes=NULL WHERE id=?"
).run(id);
return JSON.parse(journal.proof) as {
version: number;
filePath: string;
size: number;
};
} finally {
db.close();
}
},
{
dependency: join(workspaceRoot, 'node_modules/better-sqlite3'),
file: join(dataDir, 'databases/iptvnator.db'),
id: row.id,
}
);
expect(durableProof).toEqual(
expect.objectContaining({
version: 1,
filePath: row.filePath,
size: readFileSync(row.filePath).length,
})
);
// Library remains usable after recovery, including archive metadata.
app = await restartElectronApp(app, dataDir, {
env: { TZ: VIEWER_TIMEZONE },
});
await app.mainWindow
.getByRole('button', { name: 'Open downloads', exact: true })
.click();
await expect(
app.mainWindow.getByTestId(`download-library-catchup-${row.id}`)
).toBeVisible();
const recovered = await app.mainWindow.evaluate(async () =>
(await window.electron.downloadsGetList()).find(
(entry) => entry.contentType === 'catchup'
)
);
expect(recovered).toEqual(
expect.objectContaining({
id: row.id,
status: 'completed',
filePath: row.filePath,
totalBytes: durableProof.size,
})
);
expect(
readdirSync(folder).filter((name) => name.endsWith('.ts'))
).toHaveLength(1);
// Missing-file recovery must reserve fresh output and preserve a foreign
// entry at the old partial name, even with a completed journal present.
unlinkSync(row.filePath);
writeFileSync(row.filePath + '.part', 'unrelated retained file');
expect(
await app.mainWindow.evaluate(
(id) => window.electron.downloadsRedownloadMissing(id),
row.id
)
).toEqual({ success: true });
await expect
.poll(
async () =>
(
await app.mainWindow.evaluate(() =>
window.electron.downloadsGetList()
)
).find((entry) => entry.id === row.id)?.status
)
.toBe('completed');
const redownloaded = (
await app.mainWindow.evaluate(() =>
window.electron.downloadsGetList()
)
).find((entry) => entry.id === row.id);
if (!redownloaded?.filePath)
throw new Error('Missing re-downloaded archive');
expect(redownloaded.filePath).not.toBe(row.filePath);
expect(readFileSync(row.filePath + '.part', 'utf8')).toBe(
'unrelated retained file'
);
expect(readFileSync(redownloaded.filePath)).toEqual(
readFileSync(
join(
workspaceRoot,
'apps/xtream-mock-server/src/fixtures/live.mpegts'
)
)
);
// Fail owned cleanup after private capture. Its recovery pointer must
// survive a process restart without relying on hardlink restoration.
linkSync(redownloaded.filePath, redownloaded.filePath + '.part');
await app.electronApp.evaluate(() => {
const fs = process.getBuiltinModule('fs');
const unlink = fs.unlinkSync;
fs.unlinkSync = (path) => {
if (String(path).includes('.iptvnator-cleanup-'))
throw Object.assign(new Error('simulated locked capture'), {
code: 'EACCES',
});
return unlink(path);
};
});
expect(
await app.mainWindow.evaluate(
(id) => window.electron.downloadsRemove(id),
row.id
)
).toEqual({
success: false,
error: 'Could not delete the partial file',
});
const capturedPath = await app.electronApp.evaluate(
(_electron, { dependency, file, id }) => {
const Database = process
.getBuiltinModule('module')
.createRequire(dependency)(dependency);
const db = new Database(file);
try {
const record = db
.prepare(
'SELECT proof FROM download_archive_finalizations WHERE download_id=?'
)
.get(id) as { proof: string };
return (
JSON.parse(record.proof) as {
partialCleanupPath: string;
}
).partialCleanupPath;
} finally {
db.close();
}
},
{
dependency: join(workspaceRoot, 'node_modules/better-sqlite3'),
file: join(dataDir, 'databases/iptvnator.db'),
id: row.id,
}
);
expect(capturedPath).toContain('.iptvnator-cleanup-');
expect(readFileSync(capturedPath)).toEqual(
readFileSync(redownloaded.filePath)
);
// A foreign replacement in the retained capture must survive restart
// and be discoverable without backend logs. Retain the original too.
renameSync(capturedPath, capturedPath + '.owned');
writeFileSync(capturedPath, 'foreign recovery content');
app = await restartElectronApp(app, dataDir, {
env: { TZ: VIEWER_TIMEZONE },
});
await app.mainWindow
.getByRole('button', { name: 'Open downloads', exact: true })
.click();
await app.mainWindow
.getByRole('button', { name: 'Clear finished', exact: true })
.click();
await app.mainWindow
.getByRole('dialog')
.getByRole('button', { name: 'Clear finished', exact: true })
.click();
const recoveryDialog = app.mainWindow.getByRole('dialog');
await expect(recoveryDialog).toContainText('File recovery needed');
await expect(recoveryDialog).toContainText(capturedPath);
await expect(
recoveryDialog.getByRole('button', {
name: 'Copy recovery path',
exact: true,
})
).toBeVisible();
await app.mainWindow.evaluate(() => {
Object.defineProperty(navigator.clipboard, 'writeText', {
configurable: true,
value: () =>
Promise.reject(new Error('simulated clipboard failure')),
});
});
await recoveryDialog
.getByRole('button', { name: 'Copy recovery path', exact: true })
.click();
await expect(recoveryDialog).toBeVisible();
await expect(recoveryDialog).toContainText(capturedPath);
expect(readFileSync(capturedPath, 'utf8')).toBe(
'foreign recovery content'
);
await recoveryDialog
.getByRole('button', { name: 'Close', exact: true })
.click();
// Simulate explicit user recovery; the app never deletes this copy.
unlinkSync(capturedPath);
renameSync(capturedPath + '.owned', capturedPath);
app = await restartElectronApp(app, dataDir, {
env: { TZ: VIEWER_TIMEZONE },
});
writeFileSync(
redownloaded.filePath + '.part',
'unrelated terminal file'
);
expect(
await app.mainWindow.evaluate(
(id) => window.electron.downloadsRemove(id),
row.id
)
).toEqual({ success: true });
expect(readFileSync(redownloaded.filePath + '.part', 'utf8')).toBe(
'unrelated terminal file'
);
expect(
await app.mainWindow.evaluate(() =>
window.electron.downloadsGetList()
)
).toEqual([]);
} finally {
await closeElectronApp(app);
}
});
@@ -16,6 +16,7 @@ export {
epgPrograms,
playbackPositions,
downloads,
downloadArchiveFinalizations,
recordings,
appState,
// Types
@@ -0,0 +1,86 @@
import { readArchiveStatsSync } from './download-catchup-stats';
import { linkSync, rmdirSync, unlinkSync } from 'node:fs';
import { dirname } from 'node:path';
import {
sameArchiveFileIdentity,
type ArchiveFileIdentity,
} from './download-catchup-output';
import type { ArchiveDownloadProof } from './download-catchup-journal';
export class ArchiveRecoveryRequiredError extends Error {
constructor(
readonly recoveryPath: string,
reason?: unknown
) {
super(
`Archive cleanup preserved an unrelated recovery file: ${recoveryPath}${reason instanceof Error ? `. ${reason.message}` : ''}`
);
}
}
/** Retry a journaled private capture without ever deleting a replacement. */
export function cleanupArchiveCapture(
proof: ArchiveDownloadProof | undefined
): void {
if (!proof) return;
cleanupCapture(
proof.partialCleanupPath,
proof.partialIdentity,
proof.filePath + '.part'
);
if (proof.phase !== 'transfer')
cleanupCapture(
proof.finalCleanupPath,
proof.finalIdentity,
proof.filePath
);
}
function cleanupCapture(
path: string | undefined,
identity: ArchiveFileIdentity,
publicPath: string
): void {
if (!path) return;
let file;
try {
file = readArchiveStatsSync(path);
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error;
}
if (file) {
if (!(file.isFile() && sameArchiveFileIdentity(file, identity))) {
restoreArchiveReplacement(path, publicPath);
}
unlinkSync(path);
}
try {
rmdirSync(dirname(path));
} catch {
/* never recursively delete captures */
}
}
/** Never unlink a foreign capture after publishing a raceable public link. */
export function restoreArchiveReplacement(
captured: string,
publicPath: string
): never {
try {
const file = readArchiveStatsSync(captured);
try {
linkSync(captured, publicPath);
} catch (error) {
if (
(error as NodeJS.ErrnoException).code !== 'EEXIST' ||
!sameArchiveFileIdentity(readArchiveStatsSync(publicPath), file)
)
throw error;
}
} catch (error) {
throw new ArchiveRecoveryRequiredError(captured, error);
}
// A second process can remove the public link at any time. Only an explicit
// user cleanup of this recovery copy may release its durable journal entry.
throw new ArchiveRecoveryRequiredError(captured);
}
@@ -0,0 +1,108 @@
import {
link,
lstat,
mkdtemp,
readdir,
readFile,
rename,
rm,
writeFile,
} from 'node:fs/promises';
import { join } from 'node:path';
import { tmpdir } from 'node:os';
import {
cleanupCatchupFile,
cleanupCatchupPartial,
} from './download-catchup-cleanup';
jest.mock('node:fs/promises', () => {
const actual = jest.requireActual('node:fs/promises');
return {
...actual,
rename: jest.fn(actual.rename),
link: jest.fn(actual.link),
};
});
const actual =
jest.requireActual<typeof import('node:fs/promises')>('node:fs/promises');
let directory: string;
beforeEach(async () => {
directory = await mkdtemp(join(tmpdir(), 'archive-cleanup-'));
jest.mocked(rename).mockReset().mockImplementation(actual.rename);
jest.mocked(link).mockReset().mockImplementation(actual.link);
});
afterEach(async () => {
await rm(directory, { recursive: true, force: true });
});
async function prepare() {
const path = join(directory, 'show.ts.part');
await writeFile(path, 'archive');
return { path, identity: await lstat(path) };
}
it('removes the captured owned entry and its empty quarantine', async () => {
const { path, identity } = await prepare();
await cleanupCatchupFile(path, identity);
expect(await readdir(directory)).toEqual([]);
});
it('preserves a replacement at the public pathname after atomic capture', async () => {
const { path, identity } = await prepare();
jest.mocked(rename).mockImplementationOnce(async (from, to) => {
await actual.rename(from, to);
await writeFile(from, 'replacement');
});
await cleanupCatchupFile(path, identity);
expect(await readFile(path, 'utf8')).toBe('replacement');
expect(await readdir(directory)).toEqual(['show.ts.part']);
});
it('restores a replacement that arrives before atomic capture', async () => {
const { path, identity } = await prepare();
jest.mocked(rename).mockImplementationOnce(async (from, to) => {
await actual.rename(from, join(directory, 'original'));
await writeFile(from, 'replacement');
await actual.rename(from, to);
});
await expect(cleanupCatchupFile(path, identity)).rejects.toThrow(
'preserved an unrelated recovery file'
);
expect(await readFile(path, 'utf8')).toBe('replacement');
expect(await readFile(join(directory, 'original'), 'utf8')).toBe('archive');
});
it.each(['EEXIST', 'ENOTSUP'])(
'retains a captured replacement if restoring it fails with %s',
async (code) => {
const { path, identity } = await prepare();
await actual.rename(path, join(directory, 'original'));
await writeFile(path, 'replacement');
jest.mocked(link).mockRejectedValueOnce(
Object.assign(new Error('cannot restore'), { code })
);
const warning = jest
.spyOn(console, 'warn')
.mockImplementation(() => undefined);
try {
await expect(cleanupCatchupFile(path, identity)).rejects.toThrow(
'cannot restore'
);
const quarantine = (await readdir(directory)).find((entry) =>
entry.startsWith('.iptvnator-cleanup-')
);
if (!quarantine)
throw new Error('Expected a retained recovery directory');
expect(
await readFile(join(directory, quarantine, 'entry'), 'utf8')
).toBe('replacement');
expect(warning).toHaveBeenCalled();
} finally {
warning.mockRestore();
}
}
);
it('preserves an unopened partial and removes it only with matching ownership', async () => {
const { path } = await prepare();
const final = path.slice(0, -'.part'.length);
expect(await cleanupCatchupPartial(final, undefined)).toBe(false);
expect(await readFile(path, 'utf8')).toBe('archive');
expect(await cleanupCatchupPartial(final, await lstat(path))).toBe(true);
await expect(lstat(path)).rejects.toMatchObject({ code: 'ENOENT' });
});
@@ -0,0 +1,60 @@
import { readArchiveStats } from './download-catchup-stats';
import { link, mkdtemp, rename, rmdir, unlink } from 'node:fs/promises';
import { dirname, join } from 'node:path';
import {
sameArchiveFileIdentity,
type ArchiveFileIdentity,
} from './download-catchup-output';
/** Capture the directory entry atomically before inspecting or removing it. */
export async function cleanupCatchupFile(
path: string,
identity: ArchiveFileIdentity
): Promise<void> {
// mkdtemp creates an unpredictable, private directory on the same volume.
// Other writers of the public .part/final path cannot race this entry.
const directory = await mkdtemp(join(dirname(path), '.iptvnator-cleanup-'));
const captured = join(directory, 'entry');
try {
await rename(path, captured);
const stats = await readArchiveStats(captured);
if (stats.isFile() && sameArchiveFileIdentity(stats, identity)) {
await unlink(captured);
} else {
// A replacement was captured. Restore without clobbering any newer
// public entry. If restoration is unavailable, retain it privately.
try {
await link(captured, path);
throw new Error(
`Archive cleanup preserved an unrelated recovery file: ${captured}`
);
} catch (error) {
console.warn(
'[Downloads] Replaced file retained for recovery:',
captured
);
throw error;
}
}
} finally {
// Never recursively remove the quarantine: it may hold a replacement or
// a file whose verification/cleanup failed. Empty directories only.
await rmdir(directory).catch(() => undefined);
}
}
/** Failed/canceled transfers may only remove the partial that they opened. */
export async function cleanupCatchupPartial(
filePath: string | null | undefined,
identity: ArchiveFileIdentity | undefined
): Promise<boolean> {
if (!filePath) return true;
const path = filePath + '.part';
try {
if (identity) await cleanupCatchupFile(path, identity);
await readArchiveStats(path);
return false;
} catch (error) {
return (error as NodeJS.ErrnoException).code === 'ENOENT';
}
}
@@ -0,0 +1,51 @@
import { finalizeCatchupPartial } from './download-catchup-finalize';
import { recordArchiveFinalization } from './download-catchup-journal';
import {
cleanupStoredCatchupPartial,
cleanupStoredCatchupFinal,
} from './download-catchup-removal';
import type { ReservedPartialDownloadFile } from './download-file-path';
import type {
DownloadsDatabase,
DownloadTask,
TransferProgress,
} from './download-task';
export async function promoteCatchupDownload(
db: DownloadsDatabase,
task: DownloadTask,
reservation: ReservedPartialDownloadFile,
progress: TransferProgress
): Promise<number> {
const partialIdentity = task.catchupPartialIdentity;
if (!partialIdentity)
throw new Error('Archive transfer identity is unavailable');
const finalized = await finalizeCatchupPartial(
reservation,
partialIdentity,
progress.bytesDownloaded,
(finalIdentity) =>
recordArchiveFinalization(db, task.id, {
version: 1,
filePath: reservation.path,
size: progress.bytesDownloaded,
partialIdentity,
finalIdentity,
}),
() => !!(task.cancelRequested || task.pauseRequested),
() => (task.catchupCommitStarted = true),
{
partial: () =>
cleanupStoredCatchupPartial(db, task.id, reservation.path),
final: (identity) =>
cleanupStoredCatchupFinal(
db,
task.id,
reservation.path,
identity
),
}
);
task.catchupFinalized = { ...finalized, filePath: reservation.path };
return finalized.size;
}
@@ -0,0 +1,220 @@
import { archiveFileStats } from './download-catchup-stats';
import {
mkdtemp,
lstat,
link,
readFile,
rename,
rm,
writeFile,
} from 'node:fs/promises';
import { join } from 'node:path';
import { existsSync, statSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { finalizeCatchupPartial } from './download-catchup-finalize';
jest.mock('node:fs/promises', () => {
const actual = jest.requireActual('node:fs/promises');
return { ...actual, link: jest.fn(actual.link) };
});
const actualLink =
jest.requireActual<typeof import('node:fs/promises')>(
'node:fs/promises'
).link;
describe('archive file promotion', () => {
let directory: string;
beforeEach(async () => {
directory = await mkdtemp(join(tmpdir(), 'archive-promotion-'));
jest.mocked(link).mockImplementation(actualLink);
});
afterEach(async () => {
await rm(directory, { recursive: true, force: true });
});
async function prepare() {
const path = join(directory, 'show.ts');
const reservation = {
path,
partialPath: path + '.part',
filename: 'show.ts',
};
await writeFile(reservation.partialPath, 'validated bytes');
const identity = await lstat(reservation.partialPath);
return { reservation, identity };
}
it('rejects a same-sized replacement between transfer and promotion', async () => {
const { reservation, identity } = await prepare();
await rename(reservation.partialPath, join(directory, 'original'));
await writeFile(reservation.partialPath, 'untrusted bytes');
await expect(
finalizeCatchupPartial(reservation, identity, identity.size)
).rejects.toThrow('changed');
expect(await readFile(reservation.partialPath, 'utf8')).toBe(
'untrusted bytes'
);
await expect(lstat(reservation.path)).rejects.toMatchObject({
code: 'ENOENT',
});
});
it('rejects a replacement during link promotion without deleting the unowned entry', async () => {
const { reservation, identity } = await prepare();
jest.mocked(link).mockImplementationOnce(async (from, to) => {
await rename(from, join(directory, 'original'));
await writeFile(from, 'untrusted bytes');
await actualLink(from, to);
});
await expect(
finalizeCatchupPartial(reservation, identity, identity.size)
).rejects.toThrow('changed');
expect(await readFile(reservation.path, 'utf8')).toBe(
'untrusted bytes'
);
expect(await readFile(reservation.partialPath, 'utf8')).toBe(
'untrusted bytes'
);
});
it('does not claim the identity of a destination replaced after link succeeds', async () => {
const { reservation, identity } = await prepare();
jest.mocked(link).mockImplementationOnce(async (from, to) => {
await actualLink(from, to);
await rename(to, join(directory, 'linked-original'));
await writeFile(to, 'untrusted bytes');
});
await expect(
finalizeCatchupPartial(reservation, identity, identity.size)
).rejects.toThrow('changed');
expect(await readFile(reservation.path, 'utf8')).toBe(
'untrusted bytes'
);
expect(await readFile(reservation.partialPath, 'utf8')).toBe(
'validated bytes'
);
});
it.each(['ENOTSUP', 'EACCES'])(
'copies from the verified descriptor when hardlinks fail with %s',
async (code) => {
const { reservation, identity } = await prepare();
jest.mocked(link).mockImplementationOnce(async (from) => {
await rename(from, join(directory, 'original'));
await writeFile(from, 'untrusted bytes');
throw Object.assign(new Error('unsupported'), { code });
});
await expect(
finalizeCatchupPartial(reservation, identity, identity.size)
).resolves.toEqual({
size: identity.size,
identity: expect.objectContaining({
dev: expect.any(String),
ino: expect.any(String),
}),
});
expect(await readFile(reservation.path, 'utf8')).toBe(
'validated bytes'
);
expect(await readFile(reservation.partialPath, 'utf8')).toBe(
'untrusted bytes'
);
}
);
it('promotes the verified file and removes its partial', async () => {
const { reservation, identity } = await prepare();
await expect(
finalizeCatchupPartial(reservation, identity, identity.size)
).resolves.toEqual({
size: identity.size,
identity: expect.objectContaining({
dev: expect.any(String),
ino: expect.any(String),
}),
});
expect(await readFile(reservation.path, 'utf8')).toBe(
'validated bytes'
);
await expect(lstat(reservation.partialPath)).rejects.toMatchObject({
code: 'ENOENT',
});
});
it.each(['hardlink', 'copy'])(
'records write-ahead identity before %s can publish complete bytes',
async (mode) => {
const { reservation, identity } = await prepare();
if (mode === 'copy')
jest.mocked(link).mockRejectedValueOnce(
Object.assign(new Error('unsupported'), { code: 'ENOTSUP' })
);
let checkpoints = 0;
await finalizeCatchupPartial(
reservation,
identity,
identity.size,
async (expected) => {
checkpoints++;
if (checkpoints === 1) {
expect(expected).toEqual(
expect.objectContaining({
dev: identity.dev,
ino: identity.ino,
})
);
await expect(
lstat(reservation.path)
).rejects.toMatchObject({ code: 'ENOENT' });
} else {
const created = archiveFileStats(
await lstat(reservation.path)
);
expect(created).toEqual(
expect.objectContaining({
dev: expected.dev,
ino: expected.ino,
size: 0,
})
);
}
}
);
expect(checkpoints).toBe(mode === 'copy' ? 2 : 1);
}
);
it('never overwrites an occupied final destination', async () => {
const { reservation, identity } = await prepare();
await writeFile(reservation.path, 'keep me');
await expect(
finalizeCatchupPartial(reservation, identity, identity.size)
).rejects.toMatchObject({ code: 'EEXIST' });
expect(await readFile(reservation.path, 'utf8')).toBe('keep me');
});
it.each(['pause', 'cancel'])(
'stops an in-progress fallback copy on %s',
async () => {
const { reservation } = await prepare();
const bytes = Buffer.alloc(192 * 1024, 0x47);
await writeFile(reservation.partialPath, bytes);
const identity = await lstat(reservation.partialPath);
jest.mocked(link).mockRejectedValueOnce(
Object.assign(new Error('unsupported'), { code: 'ENOTSUP' })
);
let observedBytes = 0;
await expect(
finalizeCatchupPartial(
reservation,
identity,
bytes.length,
undefined,
() => {
observedBytes = existsSync(reservation.path)
? statSync(reservation.path).size
: 0;
return observedBytes > 0;
}
)
).rejects.toThrow('interrupted');
expect(observedBytes).toBeGreaterThan(0);
expect(observedBytes).toBeLessThan(bytes.length);
expect(await readFile(reservation.partialPath)).toEqual(bytes);
await expect(lstat(reservation.path)).rejects.toMatchObject({
code: 'ENOENT',
});
}
);
});
@@ -0,0 +1,178 @@
import {
archiveFileStats,
readArchiveStats,
type ArchiveFileStats,
} from './download-catchup-stats';
import type { DownloadTask, CompletedPartialProgress } from './download-task';
import { cleanupCatchupFile } from './download-catchup-cleanup';
import { constants } from 'node:fs';
import { link, open } from 'node:fs/promises';
import {
archiveFileIdentity,
sameArchiveFileIdentity,
type ArchiveFileIdentity,
} from './download-catchup-output';
import type { ReservedPartialDownloadFile } from './download-file-path';
function verify(
stats: ArchiveFileStats,
identity: ArchiveFileIdentity,
size: number
): void {
if (
!stats.isFile() ||
!sameArchiveFileIdentity(stats, identity) ||
stats.size !== size
) {
throw new Error('Archive partial changed before promotion');
}
}
/** Verify both sides of promotion; fallback copying reads a verified descriptor. */
export async function finalizeCatchupPartial(
reservation: ReservedPartialDownloadFile,
identity: ArchiveFileIdentity | undefined,
size: number,
recordProof?: (identity: ArchiveFileIdentity) => Promise<void>,
shouldInterrupt: () => boolean = () => false,
beginCommit: () => void = () => undefined,
cleanup?: {
partial: () => Promise<unknown>;
final: (identity: ArchiveFileIdentity) => Promise<unknown>;
}
): Promise<{ size: number; identity: ArchiveFileIdentity }> {
const checkInterruption = () => {
if (shouldInterrupt()) throw new Error('Archive promotion interrupted');
};
checkInterruption();
if (!identity) throw new Error('Archive transfer identity is unavailable');
verify(await readArchiveStats(reservation.partialPath), identity, size);
const source = await open(
reservation.partialPath,
constants.O_RDONLY | (constants.O_NOFOLLOW ?? 0)
);
let created: ArchiveFileIdentity | undefined;
let sourceClosed = false;
try {
verify(
archiveFileStats(await source.stat({ bigint: true })),
identity,
size
);
// A hardlink can become complete immediately: persist its expected
// identity before publishing it, while the verified partial still exists.
await recordProof?.(identity);
checkInterruption();
try {
await link(reservation.partialPath, reservation.path);
// The link we created belongs to the verified source, even if
// another writer replaces its public name before lstat completes.
created = identity;
const promoted = await readArchiveStats(reservation.path);
verify(promoted, identity, size);
} catch (error) {
const code = (error as NodeJS.ErrnoException).code;
if (
created ||
![
'EACCES',
'EPERM',
'EXDEV',
'ENOSYS',
'ENOTSUP',
'EOPNOTSUPP',
].includes(code ?? '')
)
throw error;
// FAT/network filesystems may not support links. Never reopen the
// source by pathname for this copy: it could have been replaced.
const target = await open(reservation.path, 'wx', 0o600);
try {
created = archiveFileStats(await target.stat({ bigint: true }));
// For a copy, record the exclusively created target identity
// before the first byte, so a complete file never lacks proof.
await recordProof?.(created);
const buffer = Buffer.alloc(64 * 1024);
let position = 0;
while (position < size) {
checkInterruption();
const { bytesRead } = await source.read(
buffer,
0,
Math.min(buffer.length, size - position),
position
);
if (bytesRead === 0)
throw new Error(
'Archive partial ended during promotion'
);
let written = 0;
while (written < bytesRead) {
checkInterruption();
const { bytesWritten } = await target.write(
buffer,
written,
bytesRead - written,
position + written
);
if (bytesWritten === 0)
throw new Error(
'Archive promotion made no progress'
);
written += bytesWritten;
}
position += bytesRead;
}
} finally {
await target.close();
}
}
await source.close();
sourceClosed = true;
checkInterruption();
verify(await readArchiveStats(reservation.path), created, size);
checkInterruption();
// Publication is verified. Fence new commands before awaited cleanup
// and the completion write; accepted commands were handled above.
beginCommit();
await (
cleanup
? cleanup.partial()
: cleanupCatchupFile(reservation.partialPath, identity)
).catch(() => undefined);
return { size, identity: archiveFileIdentity(created) };
} catch (error) {
if (created) {
await (
cleanup
? cleanup.final(created)
: cleanupCatchupFile(reservation.path, created)
).catch(() => undefined);
}
throw error;
} finally {
if (!sourceClosed) await source.close();
}
}
/** A DB retry can reuse completion only with explicit, still-current proof. */
export async function recoverCatchupCompletion(
task: DownloadTask
): Promise<CompletedPartialProgress | null> {
const proof = task.catchupFinalized;
if (!proof || proof.filePath !== task.filePath) return null;
try {
const file = await readArchiveStats(proof.filePath);
return file.isFile() &&
sameArchiveFileIdentity(file, proof.identity) &&
file.size === proof.size
? {
filePath: proof.filePath,
bytesDownloaded: proof.size,
totalBytes: proof.size,
}
: null;
} catch {
return null;
}
}
@@ -0,0 +1,88 @@
import { execFileSync } from 'node:child_process';
import { createRequire } from 'node:module';
import { resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
function runReservationScenario(scenario: string): {
row: { file_path: string | null; file_name: string };
proofs: { download_id: number; proof: string }[];
error?: string;
} {
const journalUrl = pathToFileURL(
resolve(__dirname, 'download-catchup-journal.ts')
).href;
const script = `
const { default: Database } = await import('better-sqlite3');
const { drizzle } = await import('drizzle-orm/better-sqlite3');
const { recordArchiveReservation } = await import(${JSON.stringify(journalUrl)});
const sqlite = new Database(':memory:');
sqlite.exec(\`
CREATE TABLE downloads (id INTEGER PRIMARY KEY, file_path TEXT, file_name TEXT, updated_at TEXT);
CREATE TABLE download_archive_finalizations (download_id INTEGER PRIMARY KEY REFERENCES downloads(id), proof TEXT NOT NULL);
INSERT INTO downloads (id, file_name) VALUES (1, 'show.ts');
\`);
const db = drizzle(sqlite);
const identity = { dev: 1, ino: 2, birthtimeMs: 1000 };
let error;
${scenario}
console.log(JSON.stringify({ error,
row: sqlite.prepare('SELECT file_path, file_name FROM downloads WHERE id=1').get(),
proofs: sqlite.prepare('SELECT * FROM download_archive_finalizations').all()
}));
sqlite.close();
`;
return JSON.parse(
execFileSync(
createRequire(__filename)('electron'),
['--import', 'tsx', '--eval', script],
{
cwd: process.cwd(),
encoding: 'utf8',
env: {
...process.env,
ELECTRON_RUN_AS_NODE: '1',
TSX_TSCONFIG_PATH: resolve(
process.cwd(),
'tsconfig.base.json'
),
},
}
)
);
}
it('commits the reservation path and ownership together before transfer setup', () => {
const result = runReservationScenario(`
await recordArchiveReservation(db, 1, '/archive/show (1).ts', identity, 'show (1).ts');
`);
expect(result.row).toEqual({
file_path: '/archive/show (1).ts',
file_name: 'show (1).ts',
});
expect(JSON.parse(result.proofs[0].proof)).toMatchObject({
phase: 'transfer',
filePath: result.row.file_path,
partialIdentity: { dev: '1', ino: '2', birthtimeMs: 1000 },
});
});
it('rolls back the row path when the ownership write fails', () => {
const result = runReservationScenario(`
sqlite.exec("CREATE TRIGGER reject_proof BEFORE INSERT ON download_archive_finalizations BEGIN SELECT RAISE(ABORT, 'journal unavailable'); END");
try { await recordArchiveReservation(db, 1, '/archive/new.ts', identity, 'new.ts'); }
catch (failure) { error = failure.message; }
`);
expect(result.error).toContain('journal unavailable');
expect(result.row).toEqual({ file_path: null, file_name: 'show.ts' });
expect(result.proofs).toEqual([]);
});
it('rejects a reservation whose download row disappeared', () => {
const result = runReservationScenario(`
try { await recordArchiveReservation(db, 2, '/archive/new.ts', identity, 'new.ts'); }
catch (failure) { error = failure.message; }
`);
expect(result.error).toContain('row is unavailable');
expect(result.proofs).toEqual([]);
expect(result.row.file_path).toBeNull();
});
@@ -0,0 +1,233 @@
import { readArchiveStatsSync } from './download-catchup-stats';
import { cleanupArchiveCapture } from './download-catchup-capture';
import { eq, inArray, sql } from 'drizzle-orm';
import { isAbsolute } from 'node:path';
import * as schema from '../../database/schema';
import {
archiveFileIdentity,
isArchiveFileId,
sameArchiveFileIdentity,
type ArchiveFileIdentity,
} from './download-catchup-output';
import type { DownloadsDatabase } from './download-task';
export interface ArchiveFinalizationProof {
version: 1;
phase?: 'finalization';
filePath: string;
size: number;
partialIdentity: ArchiveFileIdentity;
partialCleanupPath?: string;
finalCleanupPath?: string;
finalIdentity: ArchiveFileIdentity;
}
export interface ArchivePartialProof {
version: 1;
phase: 'transfer';
filePath: string;
partialIdentity: ArchiveFileIdentity;
partialCleanupPath?: string;
finalCleanupPath?: string;
}
export type ArchiveDownloadProof =
ArchiveFinalizationProof | ArchivePartialProof;
export async function recordArchivePartial(
db: DownloadsDatabase,
downloadId: number,
filePath: string,
partialIdentity: ArchiveFileIdentity
): Promise<void> {
await writeArchiveProof(db, downloadId, {
version: 1,
phase: 'transfer',
filePath,
partialIdentity,
});
}
/** The row path and its ownership proof must become recoverable together. */
export async function recordArchiveReservation(
db: DownloadsDatabase,
downloadId: number,
filePath: string,
partialIdentity: ArchiveFileIdentity,
fileName: string
): Promise<void> {
db.transaction((tx) => {
const result = tx
.update(schema.downloads)
.set({
filePath,
fileName,
updatedAt: sql`CURRENT_TIMESTAMP`,
})
.where(eq(schema.downloads.id, downloadId))
.run();
if (result.changes !== 1)
throw new Error('Archive reservation row is unavailable');
writeArchiveProof(tx, downloadId, {
version: 1,
phase: 'transfer',
filePath,
partialIdentity,
}).run();
});
}
export async function recordArchiveFinalization(
db: DownloadsDatabase,
downloadId: number,
proof: ArchiveFinalizationProof
): Promise<void> {
await writeArchiveProof(db, downloadId, proof);
}
function writeArchiveProof(
db: Pick<DownloadsDatabase, 'insert'>,
downloadId: number,
proof: ArchiveDownloadProof
) {
const serialized = JSON.stringify({
...proof,
partialIdentity: archiveFileIdentity(proof.partialIdentity),
...(proof.phase !== 'transfer'
? {
finalIdentity: archiveFileIdentity(proof.finalIdentity),
}
: {}),
});
return db
.insert(schema.downloadArchiveFinalizations)
.values({ downloadId, proof: serialized })
.onConflictDoUpdate({
target: schema.downloadArchiveFinalizations.downloadId,
set: { proof: serialized },
});
}
/** Commit the recovery pointer synchronously before a public entry is captured. */
export function recordArchiveCleanupPath(
db: DownloadsDatabase,
downloadId: number,
proof: ArchiveDownloadProof,
path: string,
kind: 'partial' | 'final' = 'partial'
): void {
const result = db
.update(schema.downloadArchiveFinalizations)
.set({
proof: JSON.stringify({
...proof,
[kind === 'partial'
? 'partialCleanupPath'
: 'finalCleanupPath']: path,
}),
})
.where(eq(schema.downloadArchiveFinalizations.downloadId, downloadId))
.run();
if (result.changes !== 1)
throw new Error('Archive cleanup ownership is unavailable');
}
/** Fresh reservations must never inherit an earlier attempt's proof. */
export async function clearArchiveFinalization(
db: DownloadsDatabase,
downloadId: number
): Promise<void> {
cleanupArchiveCapture(
(await readArchiveFinalizations(db, [downloadId])).get(downloadId)
);
await db
.delete(schema.downloadArchiveFinalizations)
.where(eq(schema.downloadArchiveFinalizations.downloadId, downloadId));
}
function identity(value: unknown): value is ArchiveFileIdentity {
if (!value || typeof value !== 'object') return false;
const candidate = value as ArchiveFileIdentity;
return (
isArchiveFileId(candidate.dev) &&
isArchiveFileId(candidate.ino) &&
Number.isFinite(candidate.birthtimeMs) &&
candidate.birthtimeMs > 0
);
}
export function parseArchiveFinalization(
value: string
): ArchiveDownloadProof | undefined {
try {
const proof = JSON.parse(value) as ArchiveDownloadProof;
if (
!proof ||
proof.version !== 1 ||
typeof proof.filePath !== 'string' ||
!isAbsolute(proof.filePath) ||
!identity(proof.partialIdentity) ||
(proof.partialCleanupPath !== undefined &&
(typeof proof.partialCleanupPath !== 'string' ||
!isAbsolute(proof.partialCleanupPath)))
)
return undefined;
if (
proof.finalCleanupPath !== undefined &&
(typeof proof.finalCleanupPath !== 'string' ||
!isAbsolute(proof.finalCleanupPath))
)
return undefined;
if (proof.phase === 'transfer')
return proof.finalCleanupPath === undefined ? proof : undefined;
return (proof.phase === undefined || proof.phase === 'finalization') &&
Number.isSafeInteger(proof.size) &&
proof.size > 0 &&
identity(proof.finalIdentity)
? proof
: undefined;
} catch {
return undefined;
}
}
export async function readArchiveFinalizations(
db: DownloadsDatabase,
ids: number[]
): Promise<Map<number, ArchiveDownloadProof>> {
if (ids.length === 0) return new Map();
const result = new Map<number, ArchiveDownloadProof>();
for (let offset = 0; offset < ids.length; offset += 500) {
const rows = await db
.select()
.from(schema.downloadArchiveFinalizations)
.where(
inArray(
schema.downloadArchiveFinalizations.downloadId,
ids.slice(offset, offset + 500)
)
);
for (const row of rows) {
const proof = parseArchiveFinalization(row.proof);
if (proof) result.set(row.downloadId, proof);
}
}
return result;
}
export function verifiedArchiveSize(
filePath: string | null,
proof: ArchiveDownloadProof | undefined
): number | null {
if (!proof || proof.phase === 'transfer' || proof.filePath !== filePath)
return null;
try {
const file = readArchiveStatsSync(proof.filePath);
return file.isFile() &&
sameArchiveFileIdentity(file, proof.finalIdentity) &&
file.size === proof.size
? proof.size
: null;
} catch {
return null;
}
}
@@ -0,0 +1,83 @@
import { statfs } from 'node:fs/promises';
import { Readable, Writable } from 'node:stream';
import { pipeline } from 'node:stream/promises';
import {
ARCHIVE_DISK_RESERVE,
assertArchiveCopyHeadroom,
createArchiveByteGuard,
getArchiveByteLimit,
} from './download-catchup-limits';
jest.mock('node:fs/promises', () => ({ statfs: jest.fn() }));
const disk = (bytes: number) =>
({ bavail: bytes, bsize: 1 }) as Awaited<ReturnType<typeof statfs>>;
beforeEach(() => jest.mocked(statfs).mockReset());
it('bounds even undeclared streams by duration, a hard cap and free space', async () => {
jest.mocked(statfs).mockResolvedValue(disk(1000 * 1024 ** 3));
expect(await getArchiveByteLimit('/downloads', 60, null)).toBe(
120 * 12_500_000
);
expect(await getArchiveByteLimit('/downloads', 86400, null)).toBe(
64 * 1024 ** 3
);
jest.mocked(statfs).mockResolvedValue(disk(ARCHIVE_DISK_RESERVE + 2000));
expect(await getArchiveByteLimit('/downloads', 60, null)).toBe(1000);
});
it('refuses insufficient space and oversized Content-Length before writing', async () => {
jest.mocked(statfs).mockResolvedValue(disk(ARCHIVE_DISK_RESERVE));
await expect(getArchiveByteLimit('/downloads', 60, null)).rejects.toThrow(
'limit'
);
jest.mocked(statfs).mockResolvedValue(disk(ARCHIVE_DISK_RESERVE + 2000));
await expect(getArchiveByteLimit('/downloads', 60, 1001)).rejects.toThrow(
'limit'
);
});
it('never forwards a chunk that exceeds the byte budget', async () => {
let written = 0;
await expect(
pipeline(
Readable.from([Buffer.alloc(600), Buffer.alloc(600)]),
createArchiveByteGuard('/downloads', 1000),
new Writable({
write(chunk, _encoding, callback) {
written += chunk.length;
callback();
},
})
)
).rejects.toThrow('limit');
expect(written).toBe(600);
});
it('stops when other disk activity consumes the reserve during transfer', async () => {
jest.mocked(statfs).mockResolvedValue(disk(ARCHIVE_DISK_RESERVE));
let written = 0;
await expect(
pipeline(
Readable.from([Buffer.alloc(16 * 1024 ** 2)]),
createArchiveByteGuard('/downloads', 64 * 1024 ** 3),
new Writable({
write(chunk, _encoding, callback) {
written += chunk.length;
callback();
},
})
)
).rejects.toThrow('limit');
expect(written).toBe(0);
});
it('rechecks copy headroom at EOF, including tails below the periodic checkpoint', async () => {
jest.mocked(statfs).mockResolvedValue(disk(ARCHIVE_DISK_RESERVE + 1000));
await expect(assertArchiveCopyHeadroom('/downloads', 1001)).rejects.toThrow(
'limit'
);
await expect(
assertArchiveCopyHeadroom('/downloads', 1000)
).resolves.toBeUndefined();
});
@@ -0,0 +1,72 @@
import { statfs } from 'node:fs/promises';
import { Transform } from 'node:stream';
const GiB = 1024 ** 3;
export const ARCHIVE_DISK_RESERVE = GiB;
const DISK_CHECK_INTERVAL = 16 * 1024 ** 2;
const limitError = () =>
new Error('Archive download exceeds the size or free-space limit');
async function availableBytes(directory: string): Promise<number> {
const stats = await statfs(directory);
return Math.max(0, stats.bavail * stats.bsize - ARCHIVE_DISK_RESERVE);
}
/** A generous 100 Mbit/s ceiling, with a minute of padding and a 64 GiB cap. */
export async function getArchiveByteLimit(
directory: string,
durationSeconds: number,
declaredBytes: number | null
): Promise<number> {
const limit = Math.floor(
Math.min(
(durationSeconds + 60) * 12_500_000,
64 * GiB,
(await availableBytes(directory)) / 2
)
);
if (limit < 188 * 3 || (declaredBytes !== null && declaredBytes > limit))
throw limitError();
return limit;
}
/** Count before forwarding any bytes, including unknown-length TS responses. */
export function createArchiveByteGuard(
directory: string,
limit: number
): Transform {
let received = 0;
let sinceDiskCheck = 0;
return new Transform({
transform(chunk: Buffer, _encoding, callback) {
received += chunk.length;
sinceDiskCheck += chunk.length;
if (received > limit) {
callback(limitError());
return;
}
if (sinceDiskCheck >= DISK_CHECK_INTERVAL) {
sinceDiskCheck = 0;
availableBytes(directory).then(
(available) => {
callback(
available < received + chunk.length
? limitError()
: null,
chunk
);
},
(error: Error) => callback(error)
);
} else callback(null, chunk);
},
});
}
/** Recheck after the output stream closes, including a sub-checkpoint tail. */
export async function assertArchiveCopyHeadroom(
directory: string,
finalSize: number
): Promise<void> {
if ((await availableBytes(directory)) < finalSize) throw limitError();
}
@@ -0,0 +1,107 @@
import { archiveFileStats, readArchiveStats } from './download-catchup-stats';
import { constants, type WriteStream } from 'node:fs';
import { open } from 'node:fs/promises';
export interface ArchiveFileIdentity {
readonly dev: number | string;
readonly ino: number | string;
readonly birthtimeMs: number;
}
export function isArchiveFileId(value: unknown): value is number | string {
return typeof value === 'string'
? /^(0|[1-9][0-9]{0,19})$/.test(value) &&
BigInt(value) <= BigInt('18446744073709551615')
: typeof value === 'number' &&
Number.isSafeInteger(value) &&
value >= 0;
}
/** Inodes can be reused after unlink; creation time identifies the generation. */
export function sameArchiveFileIdentity(
a: ArchiveFileIdentity,
b: ArchiveFileIdentity
): boolean {
return (
isArchiveFileId(a.dev) &&
isArchiveFileId(a.ino) &&
isArchiveFileId(b.dev) &&
isArchiveFileId(b.ino) &&
String(a.dev) === String(b.dev) &&
String(a.ino) === String(b.ino) &&
Number.isFinite(a.birthtimeMs) &&
a.birthtimeMs > 0 &&
a.birthtimeMs === b.birthtimeMs
);
}
export function archiveFileIdentity(
file: ArchiveFileIdentity
): ArchiveFileIdentity {
if (!isArchiveFileId(file.dev) || !isArchiveFileId(file.ino))
throw new Error('Archive filesystem identity is unavailable');
if (!Number.isFinite(file.birthtimeMs) || file.birthtimeMs <= 0)
throw new Error('Archive filesystem creation time is unavailable');
return {
dev: String(file.dev),
ino: String(file.ino),
birthtimeMs: file.birthtimeMs,
};
}
export class ArchivePartialReplacedError extends Error {
constructor() {
super('Archive partial changed before opening');
}
}
/** Open without truncation first: a replaced partial must not damage its target. */
export async function openCatchupOutput(
partialPath: string,
expectedIdentity?: ArchiveFileIdentity,
beforeTruncate?: (identity: ArchiveFileIdentity) => Promise<void>
): Promise<{ stream: WriteStream; identity: ArchiveFileIdentity }> {
const before = await readArchiveStats(partialPath).catch(
(error: NodeJS.ErrnoException) => {
if (error.code === 'ENOENT') return undefined;
throw error;
}
);
if (before && (!before.isFile() || before.nlink !== 1)) {
throw new ArchivePartialReplacedError();
}
const handle = await open(
partialPath,
constants.O_WRONLY |
(constants.O_NOFOLLOW ?? 0) |
(before ? 0 : constants.O_CREAT | constants.O_EXCL),
0o600
);
try {
const current = archiveFileStats(await handle.stat({ bigint: true }));
const identity = archiveFileIdentity(current);
// Descriptor identity also protects platforms without O_NOFOLLOW from
// a link/file replacement between lstat and open. Missing files use wx.
if (
!current.isFile() ||
current.nlink !== 1 ||
(before && !sameArchiveFileIdentity(current, before)) ||
(before &&
(!expectedIdentity ||
!sameArchiveFileIdentity(current, expectedIdentity)))
) {
throw new ArchivePartialReplacedError();
}
// Keep durable ownership evidence until the actual descriptor passes.
await beforeTruncate?.(identity);
await handle.truncate(0);
// All writes use this verified descriptor; never reopen by pathname.
return {
stream: handle.createWriteStream({ autoClose: true }),
identity,
};
} catch (error) {
await handle.close();
throw error;
}
}
@@ -0,0 +1,59 @@
import { and, eq, sql } from 'drizzle-orm';
import * as schema from '../../database/schema';
import { broadcastDownloadUpdate } from './download-broadcast';
import {
readArchiveFinalizations,
recordArchiveCleanupPath,
verifiedArchiveSize,
} from './download-catchup-journal';
import { removeJournaledCatchupPartial } from './download-catchup-removal';
import type { DownloadsDatabase } from './download-task';
/** Restore a proven final before a user request can detach it or transfer again. */
export async function recoverStoredCatchupCompletion(
db: DownloadsDatabase,
item: Pick<schema.Download, 'id' | 'contentType' | 'filePath' | 'status'>,
isBusy: () => boolean
): Promise<boolean> {
if (item.contentType !== 'catchup' || !item.filePath) return false;
const proof = (await readArchiveFinalizations(db, [item.id])).get(item.id);
const assertIdle = () => {
if (isBusy()) throw new Error('Download already in progress');
};
assertIdle();
const size = verifiedArchiveSize(item.filePath, proof);
if (size === null) return false;
// A failed source cleanup keeps its journal for later Remove/Clear retry.
try {
removeJournaledCatchupPartial(item.filePath, proof, (path) => {
if (proof) recordArchiveCleanupPath(db, item.id, proof, path);
});
} catch (error) {
console.error(
'[Downloads] Retaining recovered archive cleanup for retry:',
error
);
}
if (verifiedArchiveSize(item.filePath, proof) !== size) return false;
const result = await db
.update(schema.downloads)
.set({
status: 'completed',
bytesDownloaded: size,
totalBytes: size,
errorMessage: null,
resumeValidator: null,
updatedAt: sql`CURRENT_TIMESTAMP`,
})
.where(
and(
eq(schema.downloads.id, item.id),
eq(schema.downloads.status, item.status),
eq(schema.downloads.filePath, item.filePath)
)
);
if (result && 'changes' in result && result.changes === 0)
throw new Error('Download changed during completion recovery');
broadcastDownloadUpdate();
return true;
}
@@ -0,0 +1,338 @@
import {
lstat,
mkdtemp,
readFile,
rename,
rm,
writeFile,
link,
} from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { getDatabase } from '../../database/connection';
import * as schema from '../../database/schema';
import { finalizeCatchupPartial } from './download-catchup-finalize';
import {
recordArchiveFinalization,
recordArchivePartial,
recordArchiveCleanupPath,
parseArchiveFinalization,
} from './download-catchup-journal';
import { resetStaleDownloads } from './download-recovery';
import type { DownloadsDatabase } from './download-task';
jest.mock('../../database/connection', () => ({ getDatabase: jest.fn() }));
jest.mock('node:fs/promises', () => {
const actual = jest.requireActual('node:fs/promises');
return { ...actual, link: jest.fn(actual.link) };
});
let directory: string;
let filePath: string;
let rows: Record<string, unknown>[];
let journals: { downloadId: number; proof: string }[];
let updates: Record<string, unknown>[];
let db: DownloadsDatabase;
beforeEach(async () => {
directory = await mkdtemp(join(tmpdir(), 'archive-recovery-'));
filePath = join(directory, 'show.ts');
rows = [
{
id: 1,
filePath,
contentType: 'catchup',
status: 'downloading',
totalBytes: null,
},
];
journals = [];
updates = [];
db = {
select: () => ({
from: (table: unknown) => ({
where: async () =>
table === schema.downloads ? rows : journals,
}),
}),
insert: () => ({
values: (row: (typeof journals)[number]) => ({
onConflictDoUpdate: async () => {
journals = [JSON.parse(JSON.stringify(row))];
},
}),
}),
update: (table: unknown) => ({
set: (value: Record<string, unknown>) => ({
where: () => ({
then: (resolve: (value: undefined) => unknown) => {
updates.push(value);
return Promise.resolve(resolve(undefined));
},
run: () => {
if (
table === schema.downloadArchiveFinalizations &&
journals.length
) {
journals[0].proof = value.proof as string;
return { changes: 1 };
}
return { changes: 0 };
},
}),
}),
}),
} as unknown as DownloadsDatabase;
jest.mocked(getDatabase).mockResolvedValue(db);
jest.mocked(link)
.mockReset()
.mockImplementation(jest.requireActual('node:fs/promises').link);
});
afterEach(async () => {
await rm(directory, { recursive: true, force: true });
});
async function prepare() {
await writeFile(filePath + '.part', 'verified archive');
const partial = await lstat(filePath + '.part');
return {
partial,
reservation: {
path: filePath,
partialPath: filePath + '.part',
filename: 'show.ts',
},
};
}
it.each(['hardlink', 'copy'])(
'recovers an unknown-length archive from the durable %s journal after restart',
async (mode) => {
const { partial, reservation } = await prepare();
if (mode === 'copy')
jest.mocked(link).mockRejectedValueOnce(
Object.assign(new Error('unsupported'), { code: 'ENOTSUP' })
);
await finalizeCatchupPartial(
reservation,
partial,
partial.size,
(finalIdentity) =>
recordArchiveFinalization(db, 1, {
version: 1,
filePath,
size: partial.size,
partialIdentity: partial,
finalIdentity,
})
);
// No in-memory DownloadTask or completion DB update survives this restart.
await resetStaleDownloads();
expect(updates).toContainEqual(
expect.objectContaining({
status: 'completed',
bytesDownloaded: partial.size,
totalBytes: partial.size,
})
);
expect(await readFile(filePath, 'utf8')).toBe('verified archive');
}
);
it('pauses verified bytes when termination precedes promotion', async () => {
const { partial, reservation } = await prepare();
await expect(
finalizeCatchupPartial(
reservation,
partial,
partial.size,
async (finalIdentity) => {
await recordArchiveFinalization(db, 1, {
version: 1,
filePath,
size: partial.size,
partialIdentity: partial,
finalIdentity,
});
throw new Error('simulated termination');
}
)
).rejects.toThrow('termination');
await resetStaleDownloads();
expect(updates).toContainEqual(
expect.objectContaining({
status: 'paused',
bytesDownloaded: partial.size,
})
);
expect(await readFile(reservation.partialPath, 'utf8')).toBe(
'verified archive'
);
});
it('refuses same-size replacements and preserves unrelated partials during startup cleanup', async () => {
const { partial, reservation } = await prepare();
await finalizeCatchupPartial(
reservation,
partial,
partial.size,
(finalIdentity) =>
recordArchiveFinalization(db, 1, {
version: 1,
filePath,
size: partial.size,
partialIdentity: partial,
finalIdentity,
})
);
await rename(filePath, join(directory, 'original'));
await writeFile(filePath, 'untrusted bytes!');
await writeFile(filePath + '.part', 'leave this alone');
await resetStaleDownloads();
expect(updates.some((value) => value.status === 'completed')).toBe(false);
expect(await readFile(filePath + '.part', 'utf8')).toBe('leave this alone');
expect(await readFile(filePath, 'utf8')).toBe('untrusted bytes!');
});
it('removes an owned incomplete copy before pausing its retained source', async () => {
const { partial, reservation } = await prepare();
await writeFile(filePath, 'half');
await recordArchiveFinalization(db, 1, {
version: 1,
filePath,
size: partial.size,
partialIdentity: partial,
finalIdentity: await lstat(filePath),
});
await resetStaleDownloads();
expect(updates).toContainEqual(
expect.objectContaining({
status: 'paused',
bytesDownloaded: partial.size,
})
);
await expect(lstat(filePath)).rejects.toMatchObject({ code: 'ENOENT' });
expect(await readFile(reservation.partialPath, 'utf8')).toBe(
'verified archive'
);
});
it.each([
'{}',
'null',
'bad json',
JSON.stringify({
version: 1,
filePath: '/tmp/a',
size: -1,
partialIdentity: { dev: 1, ino: 1, birthtimeMs: 1 },
finalIdentity: { dev: 1, ino: 1, birthtimeMs: 1 },
}),
])('ignores malformed journal %s', (value) => {
expect(parseArchiveFinalization(value)).toBeUndefined();
});
it('does not remove a valid journaled final when a retry was queued at shutdown', async () => {
const { partial, reservation } = await prepare();
await finalizeCatchupPartial(
reservation,
partial,
partial.size,
(finalIdentity) =>
recordArchiveFinalization(db, 1, {
version: 1,
filePath,
size: partial.size,
partialIdentity: partial,
finalIdentity,
})
);
rows[0].status = 'queued';
await resetStaleDownloads();
expect(await readFile(filePath, 'utf8')).toBe('verified archive');
expect(updates).toContainEqual(
expect.objectContaining({ status: 'paused' })
);
});
it.each(['downloading', 'queued'])(
'detaches a replaced journaled partial instead of making %s recoverable',
async (status) => {
const { partial } = await prepare();
await recordArchiveFinalization(db, 1, {
version: 1,
filePath,
size: partial.size,
partialIdentity: partial,
finalIdentity: partial,
});
await rename(filePath + '.part', join(directory, 'original'));
await writeFile(filePath + '.part', 'unrelated user file');
rows[0].status = status;
await resetStaleDownloads();
expect(updates).toContainEqual(
expect.objectContaining({
status: 'failed',
filePath: null,
})
);
expect(updates.some((value) => value.status === 'paused')).toBe(false);
expect(await readFile(filePath + '.part', 'utf8')).toBe(
'unrelated user file'
);
}
);
it.each([false, true])(
'recovers transfer-phase identity without treating it as completion (replaced=%s)',
async (replaced) => {
const { partial } = await prepare();
await recordArchivePartial(db, 1, filePath, partial);
// A same-sized final is insufficient without a finalization-phase journal.
await writeFile(filePath, 'verified archive');
if (replaced) {
await rename(filePath + '.part', join(directory, 'original'));
await writeFile(filePath + '.part', 'unrelated bytes!');
}
await resetStaleDownloads();
expect(updates).toContainEqual(
expect.objectContaining(
replaced
? { status: 'failed', filePath: null }
: { status: 'paused' }
)
);
expect(updates.some((value) => value.status === 'completed')).toBe(
false
);
expect(await readFile(filePath, 'utf8')).toBe('verified archive');
expect(await readFile(filePath + '.part', 'utf8')).toBe(
replaced ? 'unrelated bytes!' : 'verified archive'
);
}
);
it.each([0, 1])(
'requires a durable row for the write-ahead capture pointer (changes=%s)',
async (changes) => {
const { partial } = await prepare();
const proof = {
version: 1 as const,
phase: 'transfer' as const,
filePath,
partialIdentity: partial,
};
const run = jest.fn(() => ({ changes }));
const set = jest.fn((_value: { proof: string }) => ({
where: () => ({ run }),
}));
const database = {
update: () => ({ set }),
} as unknown as DownloadsDatabase;
const capturePath = join(directory, '.iptvnator-cleanup-test/entry');
const record = () =>
recordArchiveCleanupPath(database, 1, proof, capturePath);
if (changes === 0) expect(record).toThrow('ownership');
else expect(record).not.toThrow();
const saved = JSON.parse(set.mock.calls[0][0].proof);
expect(saved).toEqual(
expect.objectContaining({
phase: 'transfer',
partialCleanupPath: capturePath,
})
);
expect(run).toHaveBeenCalledTimes(1);
}
);
@@ -0,0 +1,399 @@
import { openCatchupOutput } from './download-catchup-output';
import {
verifiedArchiveSize,
parseArchiveFinalization,
} from './download-catchup-journal';
import {
mkdtempSync,
lstatSync,
readFileSync,
renameSync,
rmSync,
writeFileSync,
unlinkSync,
linkSync,
} from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import {
removeJournaledCatchupPartial,
cleanupStoredCatchupFinal,
cleanupStoredCatchupPartial,
} from './download-catchup-removal';
import type {
ArchivePartialProof,
ArchiveFinalizationProof,
} from './download-catchup-journal';
jest.mock('node:fs', () => {
const actual = jest.requireActual('node:fs');
return {
...actual,
renameSync: jest.fn(actual.renameSync),
unlinkSync: jest.fn(actual.unlinkSync),
linkSync: jest.fn(actual.linkSync),
};
});
const actual = jest.requireActual<typeof import('node:fs')>('node:fs');
let directory: string, filePath: string, proof: ArchivePartialProof;
beforeEach(() => {
directory = mkdtempSync(join(tmpdir(), 'archive-remove-'));
filePath = join(directory, 'show.ts');
writeFileSync(filePath + '.part', 'owned bytes');
proof = {
version: 1,
phase: 'transfer',
filePath,
partialIdentity: lstatSync(filePath + '.part'),
};
jest.mocked(renameSync).mockReset().mockImplementation(actual.renameSync);
jest.mocked(unlinkSync).mockReset().mockImplementation(actual.unlinkSync);
jest.mocked(linkSync).mockReset().mockImplementation(actual.linkSync);
});
afterEach(() => rmSync(directory, { recursive: true, force: true }));
function recordCapture(path: string) {
proof = { ...proof, partialCleanupPath: path };
}
it('removes only the journaled partial', () => {
removeJournaledCatchupPartial(filePath, proof, recordCapture);
expect(() => lstatSync(filePath + '.part')).toThrow();
});
it('preserves entries without proof', () => {
removeJournaledCatchupPartial(filePath, undefined, recordCapture);
expect(readFileSync(filePath + '.part', 'utf8')).toBe('owned bytes');
});
it('preserves a replaced regular partial in place', () => {
renameSync(filePath + '.part', join(directory, 'original'));
writeFileSync(filePath + '.part', 'unrelated bytes');
removeJournaledCatchupPartial(filePath, proof, recordCapture);
expect(readFileSync(filePath + '.part', 'utf8')).toBe('unrelated bytes');
});
it('restores a replacement captured at the cleanup boundary', () => {
jest.mocked(renameSync).mockImplementationOnce((from, to) => {
actual.renameSync(from, join(directory, 'original'));
writeFileSync(from, 'unrelated bytes');
actual.renameSync(from, to);
});
expect(() =>
removeJournaledCatchupPartial(filePath, proof, recordCapture)
).toThrow('preserved an unrelated recovery file');
expect(readFileSync(filePath + '.part', 'utf8')).toBe('unrelated bytes');
expect(readFileSync(proof.partialCleanupPath!, 'utf8')).toBe(
'unrelated bytes'
);
});
it('retains a replacement capture until explicit recovery-copy cleanup', () => {
jest.mocked(renameSync).mockImplementationOnce((from, to) => {
actual.renameSync(from, join(directory, 'original'));
writeFileSync(from, 'foreign bytes');
actual.renameSync(from, to);
});
jest.mocked(linkSync).mockImplementation(() => {
throw Object.assign(new Error('hardlinks unavailable'), {
code: 'ENOTSUP',
});
});
expect(() =>
removeJournaledCatchupPartial(filePath, proof, recordCapture)
).toThrow('hardlinks unavailable');
const capture = proof.partialCleanupPath!;
expect(readFileSync(capture, 'utf8')).toBe('foreign bytes');
expect(() =>
removeJournaledCatchupPartial(filePath, proof, recordCapture)
).toThrow('hardlinks unavailable');
expect(proof.partialCleanupPath).toBe(capture);
jest.mocked(linkSync).mockImplementationOnce(() => {
throw Object.assign(new Error('destination parent missing'), {
code: 'ENOENT',
});
});
expect(() =>
removeJournaledCatchupPartial(filePath, proof, recordCapture)
).toThrow('destination parent missing');
expect(readFileSync(capture, 'utf8')).toBe('foreign bytes');
jest.mocked(linkSync).mockImplementation(actual.linkSync);
writeFileSync(filePath + '.part', 'newer public bytes');
expect(() =>
removeJournaledCatchupPartial(filePath, proof, recordCapture)
).toThrow();
expect(readFileSync(filePath + '.part', 'utf8')).toBe('newer public bytes');
expect(readFileSync(capture, 'utf8')).toBe('foreign bytes');
actual.unlinkSync(filePath + '.part');
expect(() =>
removeJournaledCatchupPartial(filePath, proof, recordCapture)
).toThrow('preserved an unrelated recovery file');
expect(readFileSync(filePath + '.part', 'utf8')).toBe('foreign bytes');
expect(readFileSync(capture, 'utf8')).toBe('foreign bytes');
// The restored public entry may disappear immediately; its recovery copy
// must survive and remain journaled across another cleanup attempt.
actual.unlinkSync(filePath + '.part');
expect(() =>
removeJournaledCatchupPartial(filePath, proof, recordCapture)
).toThrow('preserved an unrelated recovery file');
expect(readFileSync(capture, 'utf8')).toBe('foreign bytes');
actual.unlinkSync(capture); // explicit recovery-copy cleanup by the user
removeJournaledCatchupPartial(filePath, proof, recordCapture);
expect(() => lstatSync(capture)).toThrow();
});
it('retries a durable capture after an I/O error without needing hardlinks', () => {
jest.mocked(unlinkSync).mockImplementationOnce(() => {
throw Object.assign(new Error('locked'), { code: 'EACCES' });
});
jest.mocked(linkSync).mockImplementation(() => {
throw Object.assign(new Error('unsupported'), { code: 'ENOTSUP' });
});
expect(() =>
removeJournaledCatchupPartial(filePath, proof, recordCapture)
).toThrow('locked');
expect(() => lstatSync(filePath + '.part')).toThrow();
expect(proof.partialCleanupPath).toBeDefined();
expect(readFileSync(proof.partialCleanupPath!, 'utf8')).toBe('owned bytes');
removeJournaledCatchupPartial(filePath, proof, recordCapture);
expect(() => lstatSync(proof.partialCleanupPath!)).toThrow();
expect(linkSync).not.toHaveBeenCalled();
});
it('does not capture the entry when write-ahead persistence fails', () => {
expect(() =>
removeJournaledCatchupPartial(filePath, proof, (capture) => {
expect(readFileSync(filePath + '.part', 'utf8')).toBe(
'owned bytes'
);
expect(() => lstatSync(capture)).toThrow();
throw new Error('SQLITE_BUSY');
})
).toThrow('SQLITE_BUSY');
expect(renameSync).not.toHaveBeenCalled();
expect(readFileSync(filePath + '.part', 'utf8')).toBe('owned bytes');
});
it('journals a failed final-file capture without confusing it with the source', async () => {
writeFileSync(filePath, 'partial final copy');
let journal: ArchiveFinalizationProof = {
version: 1,
filePath,
partialIdentity: proof.partialIdentity,
finalIdentity: lstatSync(filePath),
size: 100,
};
const db = {
select: () => ({
from: () => ({
where: async () => [
{ downloadId: 1, proof: JSON.stringify(journal) },
],
}),
}),
update: () => ({
set: (value: { proof: string }) => ({
where: () => ({
run: () => {
journal = JSON.parse(value.proof);
return { changes: 1 };
},
}),
}),
}),
};
jest.mocked(unlinkSync).mockImplementationOnce(() => {
throw new Error('locked copy');
});
await expect(
cleanupStoredCatchupFinal(
db as never,
1,
filePath,
journal.finalIdentity
)
).resolves.toBe(false);
expect(journal.finalCleanupPath).toBeDefined();
expect(readFileSync(journal.finalCleanupPath!, 'utf8')).toBe(
'partial final copy'
);
expect(readFileSync(filePath + '.part', 'utf8')).toBe('owned bytes');
await expect(
cleanupStoredCatchupFinal(db as never, 1, filePath)
).resolves.toBe(true);
expect(() => lstatSync(journal.finalCleanupPath!)).toThrow();
expect(readFileSync(filePath + '.part', 'utf8')).toBe('owned bytes');
});
it.each([false, true])(
'cleans an owned final only for an abandoned attempt (removeFinal=%s)',
(removeFinal) => {
writeFileSync(filePath, 'unfinished copy');
let journal: ArchiveFinalizationProof = {
...proof,
phase: 'finalization',
size: 100,
finalIdentity: lstatSync(filePath),
};
const record = (path: string, kind = 'partial') => {
journal = {
...journal,
[kind === 'final' ? 'finalCleanupPath' : 'partialCleanupPath']:
path,
};
};
if (removeFinal) {
jest.mocked(unlinkSync).mockImplementationOnce(() => {
throw new Error('locked final');
});
expect(() =>
removeJournaledCatchupPartial(filePath, journal, record, true)
).toThrow('locked final');
expect(readFileSync(journal.finalCleanupPath!, 'utf8')).toBe(
'unfinished copy'
);
expect(readFileSync(filePath + '.part', 'utf8')).toBe(
'owned bytes'
);
}
removeJournaledCatchupPartial(filePath, journal, record, removeFinal);
expect(() => lstatSync(filePath + '.part')).toThrow();
if (removeFinal) {
expect(() => lstatSync(filePath)).toThrow();
expect(() => lstatSync(journal.finalCleanupPath!)).toThrow();
} else expect(readFileSync(filePath, 'utf8')).toBe('unfinished copy');
}
);
it.each([false, true])(
'restart cleanup preserves completed promotion and removes unfinished copies (complete=%s)',
async (complete) => {
writeFileSync(filePath, 'archive');
let journal: ArchiveFinalizationProof = {
...proof,
phase: 'finalization',
size: complete ? 7 : 100,
finalIdentity: lstatSync(filePath),
};
const db = {
select: () => ({
from: () => ({
where: async () => [
{ downloadId: 1, proof: JSON.stringify(journal) },
],
}),
}),
update: () => ({
set: (value: { proof: string }) => ({
where: () => ({
run: () => {
journal = JSON.parse(value.proof);
return { changes: 1 };
},
}),
}),
}),
};
await expect(
cleanupStoredCatchupPartial(db as never, 1, filePath, 'incomplete')
).resolves.toBe(true);
expect(() => lstatSync(filePath + '.part')).toThrow();
if (complete) expect(readFileSync(filePath, 'utf8')).toBe('archive');
else expect(() => lstatSync(filePath)).toThrow();
}
);
it('preserves a replacement when the filesystem reuses its predecessor inode', () => {
proof.partialIdentity = {
...proof.partialIdentity,
birthtimeMs: proof.partialIdentity.birthtimeMs - 1,
};
removeJournaledCatchupPartial(filePath, proof, recordCapture);
expect(readFileSync(filePath + '.part', 'utf8')).toBe('owned bytes');
expect(renameSync).not.toHaveBeenCalled();
});
it('rejects a reused inode before truncating or replacing the journal proof', async () => {
const previous = {
...proof.partialIdentity,
birthtimeMs: proof.partialIdentity.birthtimeMs - 1,
};
const record = jest.fn();
await expect(
openCatchupOutput(filePath + '.part', previous, record)
).rejects.toThrow('changed');
expect(record).not.toHaveBeenCalled();
expect(readFileSync(filePath + '.part', 'utf8')).toBe('owned bytes');
});
it('does not recover a same-size final from a reused inode or a legacy proof without creation time', () => {
writeFileSync(filePath, 'foreign archive');
const final = lstatSync(filePath);
const journal: ArchiveFinalizationProof = {
...proof,
phase: 'finalization',
size: final.size,
finalIdentity: { ...final, birthtimeMs: final.birthtimeMs - 1 },
};
expect(verifiedArchiveSize(filePath, journal)).toBeNull();
expect(
parseArchiveFinalization(
JSON.stringify({
...journal,
finalIdentity: { dev: final.dev, ino: final.ino },
})
)
).toBeUndefined();
});
it('never truncates a preexisting file without expected ownership', async () => {
const record = jest.fn();
await expect(
openCatchupOutput(filePath + '.part', undefined, record)
).rejects.toThrow('changed');
expect(record).not.toHaveBeenCalled();
expect(readFileSync(filePath + '.part', 'utf8')).toBe('owned bytes');
});
it.each([false, true])(
'does not relocate an unproved fallback target (replaced=%s)',
async (replaced) => {
writeFileSync(filePath, '');
const createdIdentity = lstatSync(filePath);
if (replaced) {
renameSync(filePath, join(directory, 'empty-original'));
writeFileSync(filePath, 'foreign final');
}
const db = {
select: () => ({ from: () => ({ where: async () => [] }) }),
};
jest.mocked(renameSync).mockClear();
await expect(
cleanupStoredCatchupFinal(db as never, 1, filePath, createdIdentity)
).resolves.toBe(false);
expect(readFileSync(filePath, 'utf8')).toBe(
replaced ? 'foreign final' : ''
);
expect(renameSync).not.toHaveBeenCalled();
}
);
it.each([false, true])(
'preserves the last foreign link if the restored public entry is immediately removed (retry=%s)',
(retry) => {
if (retry) {
const capture = join(directory, 'foreign-capture');
writeFileSync(capture, 'foreign bytes');
proof.partialCleanupPath = capture;
actual.unlinkSync(filePath + '.part');
} else {
jest.mocked(renameSync).mockImplementationOnce((from, to) => {
actual.renameSync(from, join(directory, 'original'));
writeFileSync(from, 'foreign bytes');
actual.renameSync(from, to);
});
}
jest.mocked(linkSync).mockImplementationOnce((from, to) => {
actual.linkSync(from, to);
actual.unlinkSync(to);
});
expect(() =>
removeJournaledCatchupPartial(filePath, proof, recordCapture)
).toThrow('preserved an unrelated recovery file');
expect(readFileSync(proof.partialCleanupPath!, 'utf8')).toBe(
'foreign bytes'
);
}
);
@@ -0,0 +1,138 @@
import {
readArchiveStatsSync,
type ArchiveFileStats,
} from './download-catchup-stats';
import {
cleanupArchiveCapture,
restoreArchiveReplacement,
} from './download-catchup-capture';
import { mkdtempSync, renameSync, unlinkSync, rmdirSync } from 'node:fs';
import { dirname, join } from 'node:path';
import {
readArchiveFinalizations,
verifiedArchiveSize,
recordArchiveCleanupPath,
type ArchiveDownloadProof,
} from './download-catchup-journal';
import type { DownloadsDatabase } from './download-task';
import {
sameArchiveFileIdentity,
type ArchiveFileIdentity,
} from './download-catchup-output';
/** IPC removal stays synchronous after its runtime guard, like VOD cleanup. */
export function removeJournaledCatchupPartial(
filePath: string | null,
proof: ArchiveDownloadProof | undefined,
recordCapture: (path: string, kind?: 'partial' | 'final') => void,
removeFinal = false
): void {
// No proof means no authority to remove the retained entry.
if (!filePath || !proof || proof.filePath !== filePath) return;
cleanupArchiveCapture(proof);
if (removeFinal && proof.phase !== 'transfer')
removeOwnedEntry(filePath, proof.finalIdentity, (path) =>
recordCapture(path, 'final')
);
removeOwnedEntry(`${filePath}.part`, proof.partialIdentity, (path) =>
recordCapture(path, 'partial')
);
}
function removeOwnedEntry(
path: string,
identity: ArchiveFileIdentity,
recordCapture: (path: string) => void
): void {
const matches = (file: ArchiveFileStats, identity: ArchiveFileIdentity) =>
file.isFile() && sameArchiveFileIdentity(file, identity);
try {
if (!matches(readArchiveStatsSync(path), identity)) return;
} catch (error) {
if ((error as NodeJS.ErrnoException).code === 'ENOENT') return;
throw error;
}
const directory = mkdtempSync(join(dirname(path), '.iptvnator-cleanup-'));
const captured = join(directory, 'entry');
try {
recordCapture(captured);
renameSync(path, captured);
if (matches(readArchiveStatsSync(captured), identity)) {
unlinkSync(captured);
} else {
restoreArchiveReplacement(captured, path);
}
} finally {
try {
rmdirSync(directory);
} catch {
/* never recursively remove a capture */
}
}
}
/** Queue/paused cancellation uses the same durable ownership as explicit Remove. */
export async function cleanupStoredCatchupPartial(
db: DownloadsDatabase,
downloadId: number,
filePath: string | null | undefined,
removeFinal: boolean | 'incomplete' = false
): Promise<boolean> {
if (!filePath) return true;
try {
const proof = (await readArchiveFinalizations(db, [downloadId])).get(
downloadId
);
removeJournaledCatchupPartial(
filePath,
proof,
(path, kind) => {
if (proof)
recordArchiveCleanupPath(db, downloadId, proof, path, kind);
},
removeFinal === true ||
(removeFinal === 'incomplete' &&
verifiedArchiveSize(filePath, proof) === null)
);
return true;
} catch (error) {
console.error(
'[Downloads] Failed to clean canceled archive partial:',
error
);
return false;
}
}
/** Failed promotion/startup cleanup retains every owned nonempty final capture. */
export async function cleanupStoredCatchupFinal(
db: DownloadsDatabase,
downloadId: number,
filePath: string,
createdIdentity?: ArchiveFileIdentity
): Promise<boolean> {
try {
const proof = (await readArchiveFinalizations(db, [downloadId])).get(
downloadId
);
if (
!proof ||
proof.phase === 'transfer' ||
proof.filePath !== filePath ||
(createdIdentity &&
!sameArchiveFileIdentity(createdIdentity, proof.finalIdentity))
) {
// No copy bytes precede final proof. Preserve an unproved empty
// destination in place: relocation needs a durable capture pointer.
return !createdIdentity;
}
cleanupArchiveCapture(proof);
removeOwnedEntry(filePath, proof.finalIdentity, (path) =>
recordArchiveCleanupPath(db, downloadId, proof, path, 'final')
);
return true;
} catch (error) {
console.error('[Downloads] Failed to clean archive promotion:', error);
return false;
}
}
@@ -0,0 +1,292 @@
import { archiveFileStats } from './download-catchup-stats';
import { openCatchupOutput } from './download-catchup-output';
import {
mkdtemp,
lstat,
readFile,
rename,
rm,
writeFile,
} from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { reserveTarget } from './download-runtime-reservation';
import { reserveFreshCatchupTarget } from './download-catchup-reservation';
import {
clearArchiveFinalization,
recordArchiveReservation,
readArchiveFinalizations,
} from './download-catchup-journal';
import type { DownloadsDatabase, DownloadTask } from './download-task';
jest.mock('./download-catchup-journal', () => ({
...jest.requireActual('./download-catchup-journal'),
clearArchiveFinalization: jest.fn().mockResolvedValue(undefined),
readArchiveFinalizations: jest.fn(),
recordArchiveCleanupPath: jest.fn(),
recordArchiveReservation: jest.fn().mockResolvedValue(undefined),
}));
it.each([false, true])(
'handles an occupied final without relocating a partial (replaced=%s)',
async (replaced) => {
const directory = await mkdtemp(join(tmpdir(), 'archive-collision-'));
const filePath = join(directory, 'show.ts');
try {
await writeFile(filePath, 'unrelated final');
await writeFile(filePath + '.part', 'owned bytes');
const task: DownloadTask = {
id: 1,
filePath,
directory,
fileName: 'show.ts',
url: 'https://provider.test/archive.ts',
catchupExpectedPartialIdentity: await lstat(filePath + '.part'),
};
jest.mocked(readArchiveFinalizations).mockResolvedValue(
new Map([
[
1,
{
version: 1,
phase: 'transfer',
filePath,
partialIdentity:
task.catchupExpectedPartialIdentity!,
},
],
])
);
if (replaced) {
await rename(
filePath + '.part',
join(directory, 'owned-original')
);
await writeFile(filePath + '.part', 'unrelated partial');
}
jest.mocked(clearArchiveFinalization).mockClear();
const reserve = reserveFreshCatchupTarget(
{} as DownloadsDatabase,
task
);
if (replaced) {
await expect(reserve).rejects.toThrow('changed');
expect(await readFile(filePath + '.part', 'utf8')).toBe(
'unrelated partial'
);
await expect(
lstat(join(directory, 'show (1).ts.part'))
).rejects.toMatchObject({ code: 'ENOENT' });
expect(clearArchiveFinalization).not.toHaveBeenCalled();
} else {
await expect(reserve).resolves.toEqual({
path: join(directory, 'show (1).ts'),
partialPath: join(directory, 'show (1).ts.part'),
filename: 'show (1).ts',
});
await expect(lstat(filePath + '.part')).rejects.toMatchObject({
code: 'ENOENT',
});
expect(
archiveFileStats(
await lstat(join(directory, 'show (1).ts.part'))
)
).toEqual(
expect.objectContaining(
task.catchupExpectedPartialIdentity!
)
);
}
expect(await readFile(filePath, 'utf8')).toBe('unrelated final');
} finally {
await rm(directory, { recursive: true, force: true });
}
}
);
it('removes a journaled incomplete final before retrying the retained archive', async () => {
const directory = await mkdtemp(join(tmpdir(), 'archive-retry-copy-'));
const filePath = join(directory, 'show.ts');
try {
await writeFile(filePath, 'incomplete');
await writeFile(filePath + '.part', 'complete source');
const proof = {
version: 1 as const,
filePath,
size: 100,
partialIdentity: await lstat(filePath + '.part'),
finalIdentity: await lstat(filePath),
};
jest.mocked(readArchiveFinalizations).mockResolvedValue(
new Map([[1, proof]])
);
const task: DownloadTask = {
id: 1,
filePath,
fileName: 'show.ts',
directory,
url: 'https://provider.test/archive.ts',
catchup: {
channelName: 'News',
startTimestamp: 100,
stopTimestamp: 200,
},
};
await expect(
reserveTarget({} as DownloadsDatabase, task)
).resolves.toMatchObject({ path: filePath });
await expect(lstat(filePath)).rejects.toMatchObject({ code: 'ENOENT' });
expect(await readFile(filePath + '.part', 'utf8')).toBe(
'complete source'
);
} finally {
await rm(directory, { recursive: true, force: true });
}
});
it('binds a fresh reservation before a replacement can arrive during the HTTP wait', async () => {
const directory = await mkdtemp(join(tmpdir(), 'archive-fresh-owner-'));
const task: DownloadTask = {
id: 19,
directory,
fileName: 'show.ts',
url: 'https://provider.test/archive.ts',
catchup: {
channelName: 'News',
startTimestamp: 100,
stopTimestamp: 200,
},
};
try {
const reservation = await reserveTarget({} as DownloadsDatabase, task);
expect(recordArchiveReservation).toHaveBeenCalledWith(
expect.anything(),
task.id,
reservation.path,
task.catchupExpectedPartialIdentity,
reservation.filename
);
await rename(reservation.partialPath, join(directory, 'original'));
await writeFile(
reservation.partialPath,
'foreign file created before the response'
);
await expect(
openCatchupOutput(
reservation.partialPath,
task.catchupExpectedPartialIdentity
)
).rejects.toThrow('changed');
expect(await readFile(reservation.partialPath, 'utf8')).toBe(
'foreign file created before the response'
);
} finally {
await rm(directory, { recursive: true, force: true });
}
});
it.each([false, true])(
'cleans a failed reservation journal without deleting a replacement (replaced=%s)',
async (replaced) => {
const directory = await mkdtemp(
join(tmpdir(), 'archive-reservation-failure-')
);
const filePath = join(directory, 'show.ts');
const task: DownloadTask = {
id: 20,
directory,
fileName: 'show.ts',
url: 'https://provider.test/archive.ts',
catchup: {
channelName: 'News',
startTimestamp: 100,
stopTimestamp: 200,
},
};
const failure = new Error('SQLite write failed');
jest.mocked(recordArchiveReservation)
.mockImplementationOnce(async () => {
if (replaced) {
await rename(
filePath + '.part',
join(directory, 'original')
);
await writeFile(filePath + '.part', 'foreign bytes');
}
throw failure;
})
.mockImplementationOnce(async (_db, id, path, identity) => {
jest.mocked(readArchiveFinalizations).mockResolvedValue(
new Map([
[
id,
{
version: 1,
phase: 'transfer',
filePath: path,
partialIdentity: identity,
},
],
])
);
});
try {
await expect(
reserveTarget({} as DownloadsDatabase, task)
).rejects.toBe(failure);
if (replaced) {
expect(await readFile(filePath + '.part', 'utf8')).toBe(
'foreign bytes'
);
} else {
await expect(lstat(filePath + '.part')).rejects.toMatchObject({
code: 'ENOENT',
});
}
} finally {
await rm(directory, { recursive: true, force: true });
}
}
);
it.each([false, true])(
'never relocates an unjournaled reservation when SQLite remains unavailable (replaced=%s)',
async (replaced) => {
const directory = await mkdtemp(join(tmpdir(), 'archive-no-journal-'));
const filePath = join(directory, 'show.ts');
const task: DownloadTask = {
id: 21,
directory,
fileName: 'show.ts',
url: 'https://provider.test/archive.ts',
catchup: {
channelName: 'News',
startTimestamp: 100,
stopTimestamp: 200,
},
};
const failure = new Error('SQLite unavailable');
jest.mocked(recordArchiveReservation)
.mockImplementationOnce(async () => {
if (replaced) {
await rename(
filePath + '.part',
join(directory, 'original')
);
await writeFile(filePath + '.part', 'foreign bytes');
}
throw failure;
})
.mockRejectedValueOnce(failure);
try {
await expect(
reserveTarget({} as DownloadsDatabase, task)
).rejects.toBe(failure);
expect(await readFile(filePath + '.part', 'utf8')).toBe(
replaced ? 'foreign bytes' : ''
);
} finally {
await rm(directory, { recursive: true, force: true });
}
}
);
@@ -0,0 +1,97 @@
import { archiveFileStats, readArchiveStats } from './download-catchup-stats';
import { closeSync, fstatSync, openSync } from 'node:fs';
import { cleanupStoredCatchupPartial } from './download-catchup-removal';
import {
clearArchiveFinalization,
recordArchiveReservation,
} from './download-catchup-journal';
import {
ArchivePartialReplacedError,
archiveFileIdentity,
sameArchiveFileIdentity,
} from './download-catchup-output';
import { reserveAvailablePartialDownloadFile } from './download-file-path';
import type { DownloadsDatabase, DownloadTask } from './download-task';
/** Archives restart at zero, so never relocate a retained entry on collision. */
export async function reserveFreshCatchupTarget(
db: DownloadsDatabase,
task: DownloadTask
) {
if (task.filePath) {
const partial = await readArchiveStats(`${task.filePath}.part`).catch(
(error: NodeJS.ErrnoException) => {
if (error.code === 'ENOENT') return undefined;
throw error;
}
);
const expected = task.catchupExpectedPartialIdentity;
if (
partial &&
(!expected ||
!partial.isFile() ||
!sameArchiveFileIdentity(partial, expected))
) {
throw new ArchivePartialReplacedError();
}
if (!(await cleanupStoredCatchupPartial(db, task.id, task.filePath))) {
throw new Error('Could not remove the owned archive partial');
}
}
await clearArchiveFinalization(db, task.id);
return reserveOwnedCatchupTarget(db, task);
}
/** Capture ownership from the exclusive creation descriptor, before any network wait. */
export async function reserveOwnedCatchupTarget(
db: DownloadsDatabase,
task: DownloadTask
) {
task.catchupExpectedPartialIdentity = undefined;
const reservation = reserveAvailablePartialDownloadFile(
task.directory,
task.fileName,
(partialPath) => {
const descriptor = openSync(partialPath, 'wx', 0o600);
try {
task.catchupExpectedPartialIdentity = archiveFileIdentity(
archiveFileStats(fstatSync(descriptor, { bigint: true }))
);
} finally {
closeSync(descriptor);
}
}
);
task.filePath = reservation.path;
task.fileName = reservation.filename;
const identity = task.catchupExpectedPartialIdentity;
if (!identity)
throw new Error('Archive reservation identity is unavailable');
try {
await recordArchiveReservation(
db,
task.id,
reservation.path,
identity,
reservation.filename
);
} catch (error) {
// Retry persistence once before cleanup. Never relocate an unjournaled
// entry: if SQLite remains unavailable, leave the empty placeholder in
// place rather than risk losing the recovery path of a replacement.
try {
await recordArchiveReservation(
db,
task.id,
reservation.path,
identity,
reservation.filename
);
await cleanupStoredCatchupPartial(db, task.id, reservation.path);
} catch {
/* no media bytes exist before the first ownership commit */
}
throw error;
}
return reservation;
}
@@ -0,0 +1,500 @@
import { unlinkSync } from 'node:fs';
import { cleanupStoredCatchupPartial } from './download-catchup-removal';
import { ArchivePartialReplacedError } from './download-catchup-output';
import { handleDownloadFailure } from './download-finalize';
import {
lstat,
mkdtemp,
readFile,
rename,
rm,
writeFile,
} from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { getDatabase } from '../../database/connection';
import { transferCatchupToPartialFile } from './download-catchup-transfer';
import {
enqueueDownload,
pauseDownload,
cancelDownload,
isDownloadCommitting,
removeDownloadFromRuntime,
prepareArchiveRemoval,
hasRuntimeDownload,
} from './download-runtime';
import {
readArchiveFinalizations,
recordArchiveCleanupPath,
type ArchiveDownloadProof,
} from './download-catchup-journal';
import type { DownloadTask } from './download-task';
jest.mock('node:fs', () => {
const actual = jest.requireActual('node:fs');
return { ...actual, unlinkSync: jest.fn(actual.unlinkSync) };
});
const mockProofs = new Map<number, ArchiveDownloadProof>();
function mockRememberCapture(
_db: unknown,
id: number,
proof: ArchiveDownloadProof,
path: string,
kind = 'partial'
) {
mockProofs.set(id, {
...proof,
[kind === 'partial' ? 'partialCleanupPath' : 'finalCleanupPath']: path,
});
}
jest.mock('./download-catchup-journal', () => ({
...jest.requireActual('./download-catchup-journal'),
recordArchiveReservation: jest.fn(
async (_db, id, filePath, partialIdentity) => {
mockProofs.set(id, {
version: 1,
phase: 'transfer',
filePath,
partialIdentity,
});
}
),
recordArchiveFinalization: jest.fn(async (_db, id, proof) => {
mockProofs.set(id, proof);
}),
recordArchiveCleanupPath: jest.fn(mockRememberCapture),
clearArchiveFinalization: jest.fn(async (_db, id) => {
mockProofs.delete(id);
}),
readArchiveFinalizations: jest.fn(async () => new Map(mockProofs)),
}));
jest.mock('../../database/connection', () => ({ getDatabase: jest.fn() }));
jest.mock('./download-catchup-transfer', () => ({
transferCatchupToPartialFile: jest.fn(),
}));
jest.mock('./download-broadcast', () => ({
broadcastDownloadUpdate: jest.fn(),
}));
beforeEach(() => {
mockProofs.clear();
jest.mocked(unlinkSync)
.mockReset()
.mockImplementation(jest.requireActual('node:fs').unlinkSync);
jest.mocked(transferCatchupToPartialFile).mockReset();
jest.mocked(readArchiveFinalizations)
.mockReset()
.mockImplementation(async () => new Map(mockProofs));
jest.mocked(recordArchiveCleanupPath)
.mockReset()
.mockImplementation(mockRememberCapture);
});
it.each(['failed', 'canceled'])(
'preserves a replaced partial when the active archive becomes %s',
async (status) => {
const directory = await mkdtemp(join(tmpdir(), 'archive-runtime-'));
const task: DownloadTask = {
id: 991,
directory,
fileName: 'show.ts',
url: 'https://provider.test/show.ts',
catchup: {
channelName: 'News',
startTimestamp: 100,
stopTimestamp: 200,
},
};
let done!: () => void;
const completed = new Promise<void>((resolve) => {
done = resolve;
});
const updates: Record<string, unknown>[] = [];
const db = {
update: () => ({
set: (value: Record<string, unknown>) => ({
where: async () => {
updates.push(value);
if (value.status === status) done();
},
}),
}),
};
jest.mocked(getDatabase).mockResolvedValue(db as never);
jest.mocked(transferCatchupToPartialFile).mockImplementationOnce(
async (_db, active, reservation) => {
active.catchupPartialIdentity = await lstat(
reservation.partialPath
);
expect(active.catchupExpectedPartialIdentity).toEqual(
expect.objectContaining({
dev: active.catchupPartialIdentity.dev,
ino: active.catchupPartialIdentity.ino,
birthtimeMs: active.catchupPartialIdentity.birthtimeMs,
})
);
await rename(
reservation.partialPath,
join(directory, 'original')
);
await writeFile(reservation.partialPath, 'keep replacement');
if (status === 'canceled') active.cancelRequested = true;
throw new Error('transfer interrupted');
}
);
const errorLog = jest
.spyOn(console, 'error')
.mockImplementation(() => undefined);
try {
task.filePath = join(directory, 'show.ts');
await writeFile(task.filePath + '.part', 'retained archive');
const partial = await lstat(task.filePath + '.part');
jest.mocked(readArchiveFinalizations).mockResolvedValueOnce(
new Map([
[
task.id,
{
version: 1,
filePath: task.filePath,
size: partial.size,
partialIdentity: partial,
finalIdentity: partial,
},
],
])
);
enqueueDownload(task);
await completed;
// Let the queue's finally block retire this task before the next case.
await new Promise((resolve) => setImmediate(resolve));
expect(
await readFile(join(directory, 'show.ts.part'), 'utf8')
).toBe('keep replacement');
expect(
updates.some((update) => update.status === 'completed')
).toBe(false);
expect(updates.at(-1)).toEqual(
expect.objectContaining({
status,
filePath: null,
})
);
} finally {
errorLog.mockRestore();
await rm(directory, { recursive: true, force: true });
}
}
);
it.each([1880, null])(
'recovers a verified archive after one completion-write failure (response length %s)',
async (totalBytes) => {
const directory = await mkdtemp(join(tmpdir(), 'archive-persistence-'));
const body = Buffer.alloc(1880, 0x47);
const task: DownloadTask = {
id: 992,
directory,
fileName: 'show.ts',
url: 'https://provider.test/show.ts',
catchup: {
channelName: 'News',
startTimestamp: 100,
stopTimestamp: 200,
},
};
let done!: () => void;
const settled = new Promise<void>((resolve) => {
done = resolve;
});
let completionAttempts = 0;
let terminalStatus: unknown;
const db = {
update: () => ({
set: (value: Record<string, unknown>) => ({
where: async () => {
if (
value.status === 'completed' &&
++completionAttempts === 1
)
throw new Error('SQLITE_BUSY');
if (
value.status === 'completed' ||
value.status === 'failed'
) {
terminalStatus = value.status;
done();
}
},
}),
}),
};
jest.mocked(getDatabase).mockResolvedValue(db as never);
jest.mocked(transferCatchupToPartialFile).mockImplementationOnce(
async (_db, active, reservation) => {
await writeFile(reservation.partialPath, body);
active.catchupPartialIdentity = await lstat(
reservation.partialPath
);
active.totalBytes = totalBytes;
return {
bytesDownloaded: body.length,
totalBytes: body.length,
};
}
);
const lateCommands: Array<boolean | Promise<boolean>> = [];
jest.mocked(recordArchiveCleanupPath).mockImplementationOnce(
(database, id, proof, path, kind) => {
mockRememberCapture(database, id, proof, path, kind);
lateCommands.push(pauseDownload(task.id));
lateCommands.push(cancelDownload(task.id));
lateCommands.push(removeDownloadFromRuntime(task.id));
}
);
try {
if (totalBytes === null) {
task.filePath = join(directory, 'show.ts');
await writeFile(
task.filePath + '.part',
'unproven retained file'
);
}
enqueueDownload(task);
await settled;
await new Promise((resolve) => setImmediate(resolve));
expect(await Promise.all(lateCommands)).toEqual([
false,
false,
false,
]);
expect(terminalStatus).toBe('completed');
expect(completionAttempts).toBe(2);
expect(await readFile(task.filePath!)).toEqual(body);
if (totalBytes === null) {
expect(task.filePath).toBe(join(directory, 'show (1).ts'));
expect(
await readFile(join(directory, 'show.ts.part'), 'utf8')
).toBe('unproven retained file');
}
await expect(lstat(task.filePath + '.part')).rejects.toMatchObject({
code: 'ENOENT',
});
expect(transferCatchupToPartialFile).toHaveBeenCalledTimes(1);
} finally {
await rm(directory, { recursive: true, force: true });
}
}
);
it('detaches a rejected replacement so Retry can reserve a fresh path', async () => {
const directory = await mkdtemp(join(tmpdir(), 'archive-detached-'));
const filePath = join(directory, 'show.ts');
const task: DownloadTask = {
id: 994,
directory,
filePath,
fileName: 'show.ts',
url: 'https://provider.test/show.ts',
catchup: {
channelName: 'News',
startTimestamp: 100,
stopTimestamp: 200,
},
};
const updates: Record<string, unknown>[] = [];
const db = {
update: () => ({
set: (value: Record<string, unknown>) => ({
where: async () => {
updates.push(value);
},
}),
}),
};
try {
await writeFile(filePath + '.part', 'unrelated user file');
await handleDownloadFailure(
db as never,
task,
undefined,
new ArchivePartialReplacedError()
);
expect(updates.at(-1)).toEqual(
expect.objectContaining({ status: 'failed', filePath: null })
);
expect(await readFile(filePath + '.part', 'utf8')).toBe(
'unrelated user file'
);
} finally {
await rm(directory, { recursive: true, force: true });
}
});
it.each([false, true])(
'paused cancellation honors durable ownership (replaced=%s)',
async (replaced) => {
const directory = await mkdtemp(
join(tmpdir(), 'archive-cancel-owned-')
);
const filePath = join(directory, 'show.ts');
try {
await writeFile(filePath + '.part', 'owned archive');
const original = await lstat(filePath + '.part');
jest.mocked(readArchiveFinalizations).mockResolvedValueOnce(
new Map([
[
995,
{
version: 1,
phase: 'transfer',
filePath,
partialIdentity: original,
},
],
])
);
if (replaced) {
await rename(filePath + '.part', join(directory, 'original'));
await writeFile(filePath + '.part', 'unrelated file');
}
const updates: Record<string, unknown>[] = [];
const db = {
select: () => ({
from: () => ({
where: () => ({
limit: async () => [
{
filePath,
status: 'paused',
contentType: 'catchup',
},
],
}),
}),
}),
update: () => ({
set: (value: Record<string, unknown>) => ({
where: async () => {
updates.push(value);
},
}),
}),
};
jest.mocked(getDatabase).mockResolvedValue(db as never);
await expect(cancelDownload(995)).resolves.toBe(true);
expect(updates).toContainEqual(
expect.objectContaining({ status: 'canceled', filePath: null })
);
if (replaced)
expect(await readFile(filePath + '.part', 'utf8')).toBe(
'unrelated file'
);
else
await expect(lstat(filePath + '.part')).rejects.toMatchObject({
code: 'ENOENT',
});
} finally {
await rm(directory, { recursive: true, force: true });
}
}
);
it.each(['failed', 'canceled'])(
'keeps active %s cleanup captures durable until retry',
async (status) => {
const directory = await mkdtemp(
join(tmpdir(), 'active-archive-cleanup-')
);
const task: DownloadTask = {
id: 996,
directory,
fileName: 'show.ts',
url: 'https://provider.test/archive.ts',
catchup: {
channelName: 'News',
startTimestamp: 100,
stopTimestamp: 200,
},
};
let ready!: () => void, release!: () => void, finish!: () => void;
const started = new Promise<void>((resolve) => {
ready = resolve;
});
const gate = new Promise<void>((resolve) => {
release = resolve;
});
const settled = new Promise<void>((resolve) => {
finish = resolve;
});
const updates: Record<string, unknown>[] = [];
const db = {
update: () => ({
set: (value: Record<string, unknown>) => ({
where: async () => {
updates.push(value);
if (value.status === status) finish();
},
}),
}),
};
jest.mocked(getDatabase).mockResolvedValue(db as never);
jest.mocked(transferCatchupToPartialFile).mockImplementationOnce(
async (_db, active, reservation) => {
await writeFile(
reservation.partialPath,
'verified archive bytes'
);
active.catchupPartialIdentity = await lstat(
reservation.partialPath
);
mockProofs.set(active.id, {
version: 1,
phase: 'transfer',
filePath: reservation.path,
partialIdentity: active.catchupPartialIdentity,
});
ready();
await gate;
throw new Error('interrupted transfer');
}
);
jest.mocked(unlinkSync).mockImplementationOnce(() => {
throw Object.assign(new Error('locked'), { code: 'EACCES' });
});
try {
enqueueDownload(task);
await started;
let removal: Promise<boolean> | undefined;
if (status === 'canceled') {
removal = prepareArchiveRemoval(task.id);
let returned = false;
void removal.then(() => {
returned = true;
});
await new Promise((resolve) => setImmediate(resolve));
expect(returned).toBe(false);
expect(hasRuntimeDownload(task.id)).toBe(true);
}
release();
await settled;
if (removal) await expect(removal).resolves.toBe(true);
await new Promise((resolve) => setImmediate(resolve));
const pointer = mockProofs.get(task.id)?.partialCleanupPath;
expect(pointer).toBeDefined();
expect(await readFile(pointer!, 'utf8')).toBe(
'verified archive bytes'
);
expect(updates.at(-1)).toEqual(
expect.objectContaining({ status, filePath: task.filePath })
);
await expect(
cleanupStoredCatchupPartial(db as never, task.id, task.filePath)
).resolves.toBe(true);
await expect(lstat(pointer!)).rejects.toMatchObject({
code: 'ENOENT',
});
} finally {
release();
await rm(directory, { recursive: true, force: true });
}
}
);
@@ -0,0 +1,52 @@
import { type BigIntStats } from 'node:fs';
import { archiveFileStats } from './download-catchup-stats';
import {
archiveFileIdentity,
sameArchiveFileIdentity,
} from './download-catchup-output';
import { parseArchiveFinalization } from './download-catchup-journal';
function stats(ino: string) {
return archiveFileStats({
dev: BigInt(1),
ino: BigInt(ino),
birthtimeMs: BigInt(1000),
birthtimeNs: BigInt('1000123456'),
size: BigInt(188),
nlink: BigInt(1),
isFile: () => true,
} as BigIntStats);
}
it('round-trips 64-bit Windows file references without losing ownership proof', () => {
const file = stats('18446744073709551000');
const identity = archiveFileIdentity(file);
const proof = parseArchiveFinalization(
JSON.stringify({
version: 1,
phase: 'transfer',
filePath: '/archive.ts',
partialIdentity: identity,
})
);
expect(proof?.partialIdentity.ino).toBe('18446744073709551000');
expect(sameArchiveFileIdentity(file, proof!.partialIdentity)).toBe(true);
expect(identity.birthtimeMs).toBe(1000.123456);
});
it('distinguishes adjacent file references that collide when converted to Number', () => {
const a = stats('18446744073709551000');
const b = stats('18446744073709551001');
expect(Number(a.ino)).toBe(Number(b.ino));
expect(sameArchiveFileIdentity(a, b)).toBe(false);
});
it('rejects already-rounded numeric ownership evidence', () => {
const rounded = {
dev: 1,
ino: Number('18446744073709551000'),
birthtimeMs: 1000,
};
expect(sameArchiveFileIdentity(rounded, rounded)).toBe(false);
expect(() => archiveFileIdentity(rounded)).toThrow(
'identity is unavailable'
);
});
@@ -0,0 +1,31 @@
import { lstatSync, type BigIntStats, type Stats } from 'node:fs';
import { lstat } from 'node:fs/promises';
/** Preserve 64-bit file IDs; Number stats can round Windows file references. */
export function archiveFileStats(file: BigIntStats | Stats) {
if (
(typeof file.dev === 'number' && !Number.isSafeInteger(file.dev)) ||
(typeof file.ino === 'number' && !Number.isSafeInteger(file.ino))
)
throw new Error('Archive filesystem identity was rounded');
const birthtimeMs =
'birthtimeNs' in file
? Number(file.birthtimeNs / BigInt(1000000000)) * 1000 +
Number(file.birthtimeNs % BigInt(1000000000)) / 1000000
: file.birthtimeMs;
return {
dev: String(file.dev),
ino: String(file.ino),
birthtimeMs,
size: Number(file.size),
nlink: Number(file.nlink),
isFile: () => file.isFile(),
};
}
export type ArchiveFileStats = ReturnType<typeof archiveFileStats>;
export async function readArchiveStats(path: string) {
return archiveFileStats(await lstat(path, { bigint: true }));
}
export function readArchiveStatsSync(path: string) {
return archiveFileStats(lstatSync(path, { bigint: true }));
}
@@ -0,0 +1,322 @@
import { recordArchivePartial } from './download-catchup-journal';
import {
mkdtemp,
readFile,
rm,
writeFile,
symlink,
link,
} from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { PassThrough, Readable, Writable } from 'node:stream';
import { pipeline } from 'node:stream/promises';
import {
createTsValidator,
transferCatchupToPartialFile,
} from './download-catchup-transfer';
import { requestWithValidatedRedirects } from '../../util/validated-axios';
import type { DownloadsDatabase, DownloadTask } from './download-task';
import {
getCompletedPartialProgress,
getExistingCompletedFileProgress,
} from './download-finalize';
import {
assertArchiveCopyHeadroom,
getArchiveByteLimit,
} from './download-catchup-limits';
import { stat } from 'node:fs/promises';
jest.mock('./download-catchup-journal', () => ({
...jest.requireActual('./download-catchup-journal'),
recordArchivePartial: jest.fn().mockResolvedValue(undefined),
}));
jest.mock('./download-catchup-limits', () => {
const actual = jest.requireActual('./download-catchup-limits');
return {
...actual,
getArchiveByteLimit: jest.fn(actual.getArchiveByteLimit),
assertArchiveCopyHeadroom: jest.fn(actual.assertArchiveCopyHeadroom),
};
});
jest.mock('../../util/validated-axios', () => ({
requestWithValidatedRedirects: jest.fn(),
}));
jest.mock('./download-transfer-persistence', () => ({
persistProgress: jest.fn().mockResolvedValue(undefined),
persistTransferStart: jest.fn().mockResolvedValue(undefined),
}));
jest.mock('./download-broadcast', () => ({
broadcastDownloadUpdate: jest.fn(),
}));
const packets = Buffer.alloc(188 * 10, 0xff);
for (let i = 0; i < packets.length; i += 188) packets[i] = 0x47;
const metadata = {
channelName: 'News',
startTimestamp: Math.floor(Date.now() / 1000) - 7200,
stopTimestamp: Math.floor(Date.now() / 1000) - 3600,
};
const sink = () =>
new Writable({
write(_chunk, _encoding, callback) {
callback();
},
});
describe('TS archive transfer', () => {
it('accepts TS packet boundaries split across arbitrary chunks', async () => {
await pipeline(
Readable.from([packets.subarray(0, 199), packets.subarray(199)]),
createTsValidator(),
sink()
);
});
it.each([
Buffer.from('#EXTM3U\n#EXTINF:10,\na.ts'),
Buffer.from('<html>expired</html>'),
packets.subarray(0, packets.length - 1),
Buffer.alloc(0),
])(
'rejects playlists, error bodies and truncated packets',
async (body) => {
await expect(
pipeline(Readable.from([body]), createTsValidator(), sink())
).rejects.toThrow();
}
);
it('does not consider a byte-complete retained archive safe to finalize', async () => {
expect(
await getExistingCompletedFileProgress({
id: 1,
url: 'https://host/1.ts',
fileName: 'a.ts',
directory: '/tmp',
filePath: '/tmp/a.ts',
totalBytes: 1880,
catchup: metadata,
})
).toBeNull();
expect(
getCompletedPartialProgress({
id: 1,
url: 'https://host/1.ts',
fileName: 'a.ts',
directory: '/tmp',
filePath: '/tmp/a.ts',
totalBytes: 1880,
catchup: metadata,
})
).toBeNull();
});
it('overwrites retained bytes and requests a fresh response without Range', async () => {
const dir = await mkdtemp(join(tmpdir(), 'archive-transfer-'));
const path = join(dir, 'show.ts');
const task: DownloadTask = {
id: 1,
url: 'https://host/archive.ts',
fileName: 'show.ts',
directory: dir,
catchup: metadata,
totalBytes: 999,
resumeValidator: 'old',
headers: { 'User-Agent': 'custom' },
};
try {
await writeFile(path + '.part', 'old data');
task.catchupExpectedPartialIdentity = await stat(path + '.part');
jest.mocked(recordArchivePartial).mockImplementationOnce(
async (_db, id, filePath, identity) => {
expect(filePath).toBe(path);
expect(identity).toEqual(
expect.objectContaining({
ino: String((await stat(path + '.part')).ino),
})
);
expect(id).toBe(task.id);
expect(await readFile(path + '.part', 'utf8')).toBe(
'old data'
);
}
);
const actualLimit = jest.requireActual<
typeof import('./download-catchup-limits')
>('./download-catchup-limits').getArchiveByteLimit;
jest.mocked(getArchiveByteLimit).mockImplementationOnce(
async (...args) => {
expect((await stat(path + '.part')).size).toBe(0);
return actualLimit(...args);
}
);
jest.mocked(requestWithValidatedRedirects).mockResolvedValue({
status: 200,
headers: {
'content-length': String(packets.length),
'content-type': 'video/mp2t',
},
data: Readable.from([packets]),
} as never);
jest.mocked(assertArchiveCopyHeadroom).mockImplementationOnce(
async (directory, size) => {
expect(await readFile(path + '.part')).toEqual(packets);
expect(size).toBe(packets.length);
return jest
.requireActual<
typeof import('./download-catchup-limits')
>('./download-catchup-limits')
.assertArchiveCopyHeadroom(directory, size);
}
);
const progress = await transferCatchupToPartialFile(
{} as DownloadsDatabase,
task,
{ path, partialPath: path + '.part', filename: 'show.ts' }
);
expect(progress).toEqual({
bytesDownloaded: packets.length,
totalBytes: packets.length,
});
expect(await readFile(path + '.part')).toEqual(packets);
expect(requestWithValidatedRedirects).toHaveBeenLastCalledWith(
task.url,
expect.objectContaining({
headers: {
'User-Agent': 'custom',
'Accept-Encoding': 'identity',
},
}),
expect.anything()
);
expect(task.resumeValidator).toBeNull();
expect(recordArchivePartial).toHaveBeenCalledWith(
expect.anything(),
task.id,
path,
expect.objectContaining({
dev: expect.any(String),
ino: expect.any(String),
})
);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
it('aborts a stalled response and never reports completion', async () => {
const dir = await mkdtemp(join(tmpdir(), 'archive-abort-'));
const task: DownloadTask = {
id: 2,
url: 'https://host/archive.ts',
fileName: 'show.ts',
directory: dir,
catchup: metadata,
};
const body = new PassThrough();
jest.mocked(requestWithValidatedRedirects).mockResolvedValue({
status: 200,
headers: {},
data: body,
} as never);
try {
const transfer = transferCatchupToPartialFile(
{} as DownloadsDatabase,
task,
{
path: join(dir, 'show.ts'),
partialPath: join(dir, 'show.ts.part'),
filename: 'show.ts',
}
);
const rejected = expect(transfer).rejects.toThrow('interrupted');
await new Promise((resolve) => setImmediate(resolve));
task.abortController?.abort();
await rejected;
expect(body.destroyed).toBe(true);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
});
it.each(['symlink', 'hardlink', 'regular file'])(
'refuses a retained archive partial replaced by a %s',
async (kind) => {
const directory = await mkdtemp(join(tmpdir(), 'archive-replaced-'));
const path = join(directory, 'archive.ts'),
target = join(directory, 'unrelated.txt');
try {
await writeFile(target, 'keep this file intact');
await (kind === 'symlink'
? symlink(target, path + '.part')
: kind === 'hardlink'
? link(target, path + '.part')
: writeFile(path + '.part', 'keep this file intact'));
jest.mocked(requestWithValidatedRedirects).mockResolvedValue({
status: 200,
headers: { 'content-type': 'video/mp2t' },
data: Readable.from([packets]),
} as never);
const task: DownloadTask = {
id: 3,
catchupExpectedPartialIdentity: await stat(target),
url: 'https://host/archive.ts',
fileName: 'archive.ts',
directory,
catchup: metadata,
};
const journalClears =
jest.mocked(recordArchivePartial).mock.calls.length;
await expect(
transferCatchupToPartialFile({} as DownloadsDatabase, task, {
path,
partialPath: path + '.part',
filename: 'archive.ts',
})
).rejects.toThrow();
expect(jest.mocked(recordArchivePartial).mock.calls.length).toBe(
journalClears
);
expect(await readFile(path + '.part', 'utf8')).toBe(
'keep this file intact'
);
expect(await readFile(target, 'utf8')).toBe(
'keep this file intact'
);
} finally {
await rm(directory, { recursive: true, force: true });
}
}
);
it('requires both finalization proof and unchanged identity to recover completion', async () => {
const directory = await mkdtemp(join(tmpdir(), 'archive-proof-'));
const filePath = join(directory, 'show.ts');
try {
await writeFile(filePath, packets);
const task: DownloadTask = {
id: 9,
directory,
fileName: 'show.ts',
filePath,
url: 'https://host/show.ts',
catchup: metadata,
totalBytes: packets.length,
};
expect(await getExistingCompletedFileProgress(task)).toBeNull();
task.catchupFinalized = {
filePath,
identity: await stat(filePath),
size: packets.length,
};
expect(await getExistingCompletedFileProgress(task)).toEqual({
filePath,
bytesDownloaded: packets.length,
totalBytes: packets.length,
});
await (
await import('node:fs/promises')
).rename(filePath, join(directory, 'original'));
await writeFile(filePath, packets);
expect(await getExistingCompletedFileProgress(task)).toBeNull();
} finally {
await rm(directory, { recursive: true, force: true });
}
});
@@ -0,0 +1,166 @@
import { recordArchivePartial } from './download-catchup-journal';
import {
assertArchiveCopyHeadroom,
createArchiveByteGuard,
getArchiveByteLimit,
} from './download-catchup-limits';
import { openCatchupOutput } from './download-catchup-output';
import { Readable, Transform } from 'node:stream';
import { finished, pipeline } from 'node:stream/promises';
import { requestWithValidatedRedirects } from '../../util/validated-axios';
import { validateCatchupDownload } from './download-catchup';
import type { ReservedPartialDownloadFile } from './download-file-path';
import type {
DownloadsDatabase,
DownloadTask,
TransferProgress,
} from './download-task';
import {
persistProgress,
persistTransferStart,
} from './download-transfer-persistence';
/** Check every 188-byte MPEG-TS packet without retaining the media in memory. */
export function createTsValidator(): Transform {
let bytes = 0;
return new Transform({
transform(chunk: Buffer, _encoding, callback) {
for (
let offset = (188 - (bytes % 188)) % 188;
offset < chunk.length;
offset += 188
) {
if (chunk[offset] !== 0x47) {
callback(
new Error(
'The provider did not return a valid TS archive'
)
);
return;
}
}
bytes += chunk.length;
callback(null, chunk);
},
flush(callback) {
callback(
bytes < 188 * 3 || bytes % 188 !== 0
? new Error('The archive stream is empty or incomplete')
: undefined
);
},
});
}
export async function transferCatchupToPartialFile(
db: DownloadsDatabase,
task: DownloadTask,
reservation: ReservedPartialDownloadFile
): Promise<TransferProgress> {
const metadata = validateCatchupDownload(task.catchup, task.url);
const controller = new AbortController();
task.abortController = controller;
if (task.cancelRequested || task.pauseRequested) controller.abort();
// An archive endpoint must terminate. Bound both stalled and endless streams.
const timeoutMs = Math.min(
24 * 3600_000,
(metadata.stopTimestamp - metadata.startTimestamp) * 2000 + 600_000
);
const timer = setTimeout(() => controller.abort(), timeoutMs);
let readable: Readable | undefined;
let output: Awaited<ReturnType<typeof openCatchupOutput>> | undefined;
let pendingProgress = Promise.resolve();
try {
const response = await requestWithValidatedRedirects<Readable>(
task.url,
{
headers: { ...task.headers, 'Accept-Encoding': 'identity' },
method: 'GET',
responseType: 'stream',
decompress: false,
signal: controller.signal,
timeout: 30_000,
validateStatus: (status) => status === 200,
},
{ allowPrivateNetworks: true }
);
readable = response.data;
readable.on('error', () => undefined);
const contentType = String(response.headers['content-type'] ?? '');
if (
/mpegurl|html|json|xml|text\/plain/i.test(contentType) ||
(response.headers['content-encoding'] &&
response.headers['content-encoding'] !== 'identity')
) {
throw new Error('The provider did not return a TS archive');
}
const length = Number(response.headers['content-length']);
const totalBytes =
Number.isSafeInteger(length) && length > 0 ? length : null;
// Restart safely before measuring space: the retained file is
// reclaimable only after its verified descriptor has been truncated.
output = await openCatchupOutput(
reservation.partialPath,
task.catchupExpectedPartialIdentity,
(identity) =>
recordArchivePartial(db, task.id, reservation.path, identity)
);
output.stream.on('error', () => undefined);
task.catchupPartialIdentity = output.identity;
const byteLimit = await getArchiveByteLimit(
task.directory,
metadata.stopTimestamp - metadata.startTimestamp,
totalBytes
);
task.totalBytes = totalBytes;
task.resumeValidator = null;
await persistTransferStart(db, task, 0, totalBytes);
let bytesDownloaded = 0;
let lastProgress = Date.now();
const progress = new Transform({
transform(chunk: Buffer, _encoding, callback) {
bytesDownloaded += chunk.length;
if (Date.now() - lastProgress >= 500) {
lastProgress = Date.now();
const snapshot = { bytesDownloaded, totalBytes };
pendingProgress = pendingProgress.then(() =>
persistProgress(db, task, snapshot)
);
// Apply backpressure while persisting, so no updates outlive finalization.
pendingProgress.then(
() => callback(null, chunk),
(error: Error) => callback(error)
);
} else callback(null, chunk);
},
});
await pipeline(
readable,
createArchiveByteGuard(task.directory, byteLimit),
createTsValidator(),
progress,
output.stream,
{ signal: controller.signal }
);
if (totalBytes !== null && bytesDownloaded !== totalBytes) {
throw new Error('The archive stream ended before it was complete');
}
await assertArchiveCopyHeadroom(task.directory, bytesDownloaded);
return { bytesDownloaded, totalBytes: totalBytes ?? bytesDownloaded };
} catch {
// Network errors can embed credential-bearing request URLs.
throw new Error(
'Archive download failed: the stream was interrupted, expired, exceeded the size or free-space limit, or is not a complete TS response. Retry starts from the beginning.'
);
} finally {
clearTimeout(timer);
readable?.on('error', () => undefined);
readable?.destroy();
if (output) {
const closed = finished(output.stream).catch(() => undefined);
output.stream.destroy();
await closed;
}
await pendingProgress.catch(() => undefined);
}
}
@@ -0,0 +1,37 @@
import { validateCatchupDownload } from './download-catchup';
describe('catch-up download validation', () => {
const now = 1788800000;
const metadata = {
channelName: 'News',
startTimestamp: now - 7200,
stopTimestamp: now - 3600,
expiresAt: now + 86400,
};
it('accepts a completed programme inside its archive window', () => {
expect(
validateCatchupDownload(
metadata,
'https://host/timeshift/u/p/60/date/1.ts',
now
)
).toEqual(metadata);
});
it.each([
{ ...metadata, stopTimestamp: now + 1 },
{ ...metadata, stopTimestamp: metadata.startTimestamp },
{ ...metadata, startTimestamp: NaN },
{ ...metadata, expiresAt: now },
null,
])('refuses incomplete, invalid or expired programmes', (data) => {
expect(() =>
validateCatchupDownload(data, 'https://host/1.ts', now)
).toThrow();
});
it.each([
'https://host/archive.m3u8',
'https://host/timeshift.php?extension=m3u8',
])('refuses HLS URLs', (url) => {
expect(() => validateCatchupDownload(metadata, url, now)).toThrow(/TS/);
});
});
@@ -0,0 +1,49 @@
import type { CatchupDownloadMetadata } from '@iptvnator/shared/interfaces';
export function validateCatchupDownload(
value: CatchupDownloadMetadata | null | undefined,
url: string,
now = Math.floor(Date.now() / 1000)
): CatchupDownloadMetadata {
if (
!value ||
typeof value.channelName !== 'string' ||
!Number.isSafeInteger(value.startTimestamp) ||
value.startTimestamp <= 0 ||
!Number.isSafeInteger(value.stopTimestamp) ||
value.stopTimestamp <= value.startTimestamp ||
value.stopTimestamp > now
) {
throw new Error('Only completed archive programmes can be downloaded');
}
if (
value.expiresAt !== undefined &&
(!Number.isSafeInteger(value.expiresAt) || value.expiresAt <= now)
) {
throw new Error('This programme is no longer available in the archive');
}
let parsed: URL;
try {
parsed = new URL(url);
} catch {
throw new Error('Invalid archive URL');
}
if (
!['http:', 'https:'].includes(parsed.protocol) ||
/\.m3u8$/i.test(parsed.pathname) ||
[...parsed.searchParams.values()].some((v) => /^m3u8$/i.test(v))
) {
throw new Error('Archive downloads currently require a TS stream');
}
return value;
}
export function catchupForDownload(item: {
contentType: string;
catchup?: CatchupDownloadMetadata | null;
url: string;
}): CatchupDownloadMetadata | undefined {
return item.contentType === 'catchup'
? validateCatchupDownload(item.catchup, item.url)
: undefined;
}
@@ -0,0 +1,90 @@
import type { DownloadTask } from './download-task';
import {
getPartialDownloadSize,
removePartialDownloadFile,
} from './download-file-path';
import { constants } from 'node:fs';
import { copyFile, link, stat, unlink } from 'node:fs/promises';
import type { ReservedPartialDownloadFile } from './download-file-path';
export async function finalizePartialDownload(
reservation: ReservedPartialDownloadFile,
expectedFileSize: number
): Promise<number> {
try {
await link(reservation.partialPath, reservation.path);
} catch (error) {
if (!canCopyCompletedPartialAfterLinkFailure(error)) {
throw error;
}
await copyFile(
reservation.partialPath,
reservation.path,
constants.COPYFILE_EXCL
);
}
try {
await unlink(reservation.partialPath);
} catch (error) {
const fileSize = await getExpectedFinalFileSize(
reservation.path,
expectedFileSize
);
if (fileSize !== null) {
console.error(
'[Downloads] Failed to delete completed partial file:',
error
);
return fileSize;
}
throw error;
}
const fileStats = await stat(reservation.path);
return fileStats.size;
}
async function getExpectedFinalFileSize(
filePath: string,
expectedFileSize: number
): Promise<number | null> {
try {
const fileStats = await stat(filePath);
return fileStats.size === expectedFileSize ? fileStats.size : null;
} catch {
return null;
}
}
function canCopyCompletedPartialAfterLinkFailure(error: unknown): boolean {
const errorCode = (error as NodeJS.ErrnoException).code;
return (
errorCode === 'EACCES' ||
errorCode === 'ENOSYS' ||
errorCode === 'ENOTSUP' ||
errorCode === 'EOPNOTSUPP' ||
errorCode === 'EPERM' ||
errorCode === 'EXDEV'
);
}
/** @returns false when a .part exists but could not be deleted. */
export function removePartialFile(
filePath: string | null | undefined
): boolean {
try {
removePartialDownloadFile(filePath);
return true;
} catch (error) {
console.error('[Downloads] Failed to delete partial file:', error);
return false;
}
}
export function getPausedByteCount(task: DownloadTask): number {
try {
return getPartialDownloadSize(task.filePath);
} catch (error) {
console.error('[Downloads] Failed to inspect partial file:', error);
return 0;
}
}
@@ -1,11 +1,18 @@
import { ArchivePartialReplacedError } from './download-catchup-output';
import {
finalizePartialDownload,
removePartialFile,
} from './download-file-finalize';
import { cleanupStoredCatchupPartial } from './download-catchup-removal';
import { promoteCatchupDownload } from './download-catchup-completion';
import { recoverCatchupCompletion } from './download-catchup-finalize';
import { eq, sql } from 'drizzle-orm';
import { constants, existsSync } from 'node:fs';
import { copyFile, link, stat, unlink } from 'node:fs/promises';
import { existsSync } from 'node:fs';
import { stat } from 'node:fs/promises';
import * as schema from '../../database/schema';
import {
getPartialDownloadPath,
getPartialDownloadSize,
removePartialDownloadFile,
type ReservedPartialDownloadFile,
} from './download-file-path';
import type {
@@ -55,7 +62,9 @@ export async function handleDownloadFailure(
const existingCompletedFileProgress =
await getExistingCompletedFileProgress(task);
if (existingCompletedFileProgress) {
removePartialFile(existingCompletedFileProgress.filePath);
if (task.catchup) {
await cleanupStoredCatchupPartial(db, task.id, task.filePath);
} else removePartialFile(existingCompletedFileProgress.filePath);
await persistCompletion(
db,
task,
@@ -82,12 +91,19 @@ export async function handleDownloadFailure(
`[Downloads] Error downloading ${task.fileName}:`,
describeError(error)
);
removePartialFile(task.filePath);
const removed = task.catchup
? await cleanupStoredCatchupPartial(db, task.id, task.filePath, true)
: removePartialFile(task.filePath);
await db
.update(schema.downloads)
.set({
errorMessage: describeError(error),
filePath: null,
filePath:
task.catchup &&
!removed &&
!(error instanceof ArchivePartialReplacedError)
? (task.filePath ?? null)
: null,
resumeValidator: null,
status: 'failed',
updatedAt: sql`CURRENT_TIMESTAMP`,
@@ -103,10 +119,19 @@ export async function completeDownloadFromPartial(
): Promise<void> {
let fileSize: number;
try {
fileSize = await finalizePartialDownload(
reservation,
progress.bytesDownloaded
);
if (task.catchup) {
fileSize = await promoteCatchupDownload(
db,
task,
reservation,
progress
);
} else {
fileSize = await finalizePartialDownload(
reservation,
progress.bytesDownloaded
);
}
} catch (error) {
if (task.cancelRequested || task.pauseRequested) {
throw error;
@@ -234,6 +259,7 @@ async function persistRetainedPartialFailure(
export async function getExistingCompletedFileProgress(
task: DownloadTask
): Promise<CompletedPartialProgress | null> {
if (task.catchup) return recoverCatchupCompletion(task);
if (
!task.filePath ||
task.totalBytes === null ||
@@ -263,6 +289,7 @@ export async function getExistingCompletedFileProgress(
export function getCompletedPartialProgress(
task: DownloadTask
): CompletedPartialProgress | null {
if (task.catchup) return null;
if (
!task.filePath ||
task.totalBytes === null ||
@@ -291,84 +318,7 @@ export function getCompletedPartialProgress(
}
}
export function getPausedByteCount(task: DownloadTask): number {
try {
return getPartialDownloadSize(task.filePath);
} catch (error) {
console.error('[Downloads] Failed to inspect partial file:', error);
return 0;
}
}
async function finalizePartialDownload(
reservation: ReservedPartialDownloadFile,
expectedFileSize: number
): Promise<number> {
try {
await link(reservation.partialPath, reservation.path);
} catch (error) {
if (!canCopyCompletedPartialAfterLinkFailure(error)) {
throw error;
}
await copyFile(
reservation.partialPath,
reservation.path,
constants.COPYFILE_EXCL
);
}
try {
await unlink(reservation.partialPath);
} catch (error) {
const fileSize = await getExpectedFinalFileSize(
reservation.path,
expectedFileSize
);
if (fileSize !== null) {
console.error(
'[Downloads] Failed to delete completed partial file:',
error
);
return fileSize;
}
throw error;
}
const fileStats = await stat(reservation.path);
return fileStats.size;
}
async function getExpectedFinalFileSize(
filePath: string,
expectedFileSize: number
): Promise<number | null> {
try {
const fileStats = await stat(filePath);
return fileStats.size === expectedFileSize ? fileStats.size : null;
} catch {
return null;
}
}
function canCopyCompletedPartialAfterLinkFailure(error: unknown): boolean {
const errorCode = (error as NodeJS.ErrnoException).code;
return (
errorCode === 'EACCES' ||
errorCode === 'ENOSYS' ||
errorCode === 'ENOTSUP' ||
errorCode === 'EOPNOTSUPP' ||
errorCode === 'EPERM' ||
errorCode === 'EXDEV'
);
}
/** @returns false when a .part exists but could not be deleted. */
export function removePartialFile(
filePath: string | null | undefined
): boolean {
try {
removePartialDownloadFile(filePath);
return true;
} catch (error) {
console.error('[Downloads] Failed to delete partial file:', error);
return false;
}
}
export {
getPausedByteCount,
removePartialFile,
} from './download-file-finalize';
@@ -1,3 +1,15 @@
import { readArchiveStatsSync } from './download-catchup-stats';
import { sameArchiveFileIdentity } from './download-catchup-output';
import {
cleanupStoredCatchupPartial,
cleanupStoredCatchupFinal,
} from './download-catchup-removal';
import type { DownloadsDatabase } from './download-task';
import {
readArchiveFinalizations,
verifiedArchiveSize,
type ArchiveDownloadProof,
} from './download-catchup-journal';
import { inArray, sql } from 'drizzle-orm';
import { statSync } from 'node:fs';
import { getDatabase } from '../../database/connection';
@@ -8,12 +20,31 @@ import {
} from './download-file-path';
interface StaleDownload {
contentType?: string;
proof?: ArchiveDownloadProof;
filePath: string | null;
id: number;
status: string;
totalBytes: number | null;
}
/** A journaled source cannot be adopted again after an unrelated replacement. */
function hasReplacedArchivePartial(download: StaleDownload): boolean {
if (
download.contentType !== 'catchup' ||
!download.proof ||
!download.filePath
)
return false;
try {
const file = readArchiveStatsSync(`${download.filePath}.part`);
const expected = download.proof.partialIdentity;
return !file.isFile() || !sameArchiveFileIdentity(file, expected);
} catch (error) {
return (error as NodeJS.ErrnoException).code !== 'ENOENT';
}
}
function getRecoverablePartialSize(download: StaleDownload): number {
if (
(download.status !== 'downloading' && download.status !== 'queued') ||
@@ -40,6 +71,10 @@ function getRecoverablePartialSize(download: StaleDownload): number {
* commits the completion instead of orphaning the file and re-downloading.
*/
function getFinalizedFileSize(download: StaleDownload): number | null {
if (download.contentType === 'catchup')
return download.status === 'downloading'
? verifiedArchiveSize(download.filePath, download.proof)
: null;
if (
download.status !== 'downloading' ||
!download.filePath ||
@@ -58,7 +93,17 @@ function getFinalizedFileSize(download: StaleDownload): number | null {
}
}
function removeFailedPartial(download: StaleDownload): boolean {
async function removeFailedPartial(
db: DownloadsDatabase,
download: StaleDownload
): Promise<boolean> {
if (download.contentType === 'catchup')
return cleanupStoredCatchupPartial(
db,
download.id,
download.filePath,
true
);
if (!download.filePath) {
return true;
}
@@ -76,7 +121,14 @@ function removeFailedPartial(download: StaleDownload): boolean {
}
}
function removeCompletedPartial(download: StaleDownload): void {
async function removeCompletedPartial(
db: DownloadsDatabase,
download: StaleDownload
): Promise<void> {
if (download.contentType === 'catchup') {
await cleanupStoredCatchupPartial(db, download.id, download.filePath);
return;
}
if (!download.filePath) {
return;
}
@@ -95,9 +147,10 @@ function removeCompletedPartial(download: StaleDownload): void {
export async function resetStaleDownloads(): Promise<void> {
try {
const db = await getDatabase();
const downloads = await db
const rows = await db
.select({
filePath: schema.downloads.filePath,
contentType: schema.downloads.contentType,
id: schema.downloads.id,
status: schema.downloads.status,
totalBytes: schema.downloads.totalBytes,
@@ -110,6 +163,19 @@ export async function resetStaleDownloads(): Promise<void> {
'completed',
])
);
const proofs = await readArchiveFinalizations(
db,
rows
.filter((row) => row.contentType === 'catchup')
.map((row) => row.id)
);
const downloads = rows.map((row) => {
const proof = proofs.get(row.id);
return {
...row,
proof: proof?.filePath === row.filePath ? proof : undefined,
};
});
const completedDownloads = downloads.filter(
(download) => download.status === 'completed'
);
@@ -125,6 +191,23 @@ export async function resetStaleDownloads(): Promise<void> {
const finalizedIds = new Set(
finalizedDownloads.map((download) => download.id)
);
// A killed copy may have left an incomplete owned destination. Remove
// only that journal-bound entry before resuming the retained source.
for (const download of staleDownloads) {
if (
download.contentType === 'catchup' &&
download.proof &&
download.proof.phase !== 'transfer' &&
!finalizedIds.has(download.id) &&
verifiedArchiveSize(download.filePath, download.proof) === null
) {
await cleanupStoredCatchupFinal(
db,
download.id,
download.proof.filePath
);
}
}
// Queued rows are recoverable even without partial bytes: a resumed
// download waiting behind an active one is persisted as 'queued' with
// its retained .part, and a never-started queued row loses nothing by
@@ -137,7 +220,9 @@ export async function resetStaleDownloads(): Promise<void> {
}))
.filter(
(download) =>
download.status === 'queued' || download.bytesDownloaded > 0
!hasReplacedArchivePartial(download) &&
(download.status === 'queued' ||
download.bytesDownloaded > 0)
);
const recoverableIds = new Set(
recoverableDownloads.map((download) => download.id)
@@ -147,10 +232,16 @@ export async function resetStaleDownloads(): Promise<void> {
!recoverableIds.has(download.id) &&
!finalizedIds.has(download.id)
);
const cleanupResult = failedDownloads.map((download) => ({
...download,
partialRemoved: removeFailedPartial(download),
}));
const cleanupResult = await Promise.all(
failedDownloads.map(async (download) => ({
...download,
// Detach a known replacement without touching it. A later retry
// reserves another path instead of truncating the unrelated file.
partialRemoved:
hasReplacedArchivePartial(download) ||
(await removeFailedPartial(db, download)),
}))
);
const failedIdsWithRemovedPartials = cleanupResult
.filter((download) => download.partialRemoved)
.map((download) => download.id);
@@ -158,15 +249,20 @@ export async function resetStaleDownloads(): Promise<void> {
.filter((download) => !download.partialRemoved)
.map((download) => download.id);
completedDownloads.forEach(removeCompletedPartial);
await Promise.all(
completedDownloads.map((download) =>
removeCompletedPartial(db, download)
)
);
for (const download of finalizedDownloads) {
// The interrupted commit may also have left the .part behind.
removeCompletedPartial(download);
await removeCompletedPartial(db, download);
await db
.update(schema.downloads)
.set({
bytesDownloaded: download.finalizedSize,
totalBytes: download.finalizedSize,
errorMessage: null,
status: 'completed',
updatedAt: sql`CURRENT_TIMESTAMP`,
@@ -196,7 +292,9 @@ export async function resetStaleDownloads(): Promise<void> {
status: 'failed',
updatedAt: sql`CURRENT_TIMESTAMP`,
})
.where(inArray(schema.downloads.id, failedIdsWithRemovedPartials));
.where(
inArray(schema.downloads.id, failedIdsWithRemovedPartials)
);
}
if (failedIdsWithRetainedPartials.length > 0) {
@@ -207,7 +305,9 @@ export async function resetStaleDownloads(): Promise<void> {
status: 'failed',
updatedAt: sql`CURRENT_TIMESTAMP`,
})
.where(inArray(schema.downloads.id, failedIdsWithRetainedPartials));
.where(
inArray(schema.downloads.id, failedIdsWithRetainedPartials)
);
}
console.log('[Downloads] Reset stale downloads');
@@ -67,6 +67,11 @@ async function setup(options: SetupOptions = {}) {
}
});
const enqueueDownload = jest.fn();
const proof = { version: 1, phase: 'transfer' };
const removeJournaledCatchupPartial = jest.fn((_path, _proof, record) =>
record('/downloads/.iptvnator-cleanup-test/entry')
);
const recordArchiveCleanupPath = jest.fn();
jest.doMock('node:fs', () => ({
...jest.requireActual<typeof import('node:fs')>('node:fs'),
@@ -78,6 +83,15 @@ async function setup(options: SetupOptions = {}) {
}));
jest.doMock('../url-safety', () => ({ assertRemoteUrlAllowed }));
jest.doMock('./download-file-path', () => ({ removePartialDownloadFile }));
jest.doMock('./download-catchup-journal', () => ({
recordArchiveCleanupPath,
readArchiveFinalizations: jest
.fn()
.mockResolvedValue(new Map([[42, proof]])),
}));
jest.doMock('./download-catchup-removal', () => ({
removeJournaledCatchupPartial,
}));
jest.doMock('./download-runtime', () => ({ enqueueDownload }));
const { redownloadMissingRequest } = await import('./download-redownload');
@@ -85,6 +99,8 @@ async function setup(options: SetupOptions = {}) {
accessSync,
assertRemoteUrlAllowed,
enqueueDownload,
removeJournaledCatchupPartial,
proof,
lstatSync,
redownloadMissingRequest,
removePartialDownloadFile,
@@ -124,6 +140,30 @@ describe('redownload missing completed file', () => {
});
});
it('reserves a fresh archive path and uses journal-backed previous-partial cleanup', async () => {
const catchup = {
channelName: 'News',
startTimestamp: 100,
stopTimestamp: 200,
};
const h = await setup({ row: { contentType: 'catchup', catchup } });
await expect(h.redownloadMissingRequest(42)).resolves.toEqual({
success: true,
});
expect(h.removeJournaledCatchupPartial).toHaveBeenCalledWith(
'/downloads/movie.mp4',
h.proof,
expect.any(Function)
);
expect(h.removePartialDownloadFile).not.toHaveBeenCalled();
expect(h.set).toHaveBeenCalledWith(
expect.objectContaining({ status: 'queued', filePath: null })
);
expect(h.enqueueDownload).toHaveBeenCalledWith(
expect.objectContaining({ catchup, filePath: null })
);
});
it('adds the Xtream fallback User-Agent to a legacy row', async () => {
const harness = await setup({
playlistType: 'xtream',
@@ -1,3 +1,9 @@
import {
readArchiveFinalizations,
recordArchiveCleanupPath,
} from './download-catchup-journal';
import { removeJournaledCatchupPartial } from './download-catchup-removal';
import { catchupForDownload } from './download-catchup';
import { and, eq, sql } from 'drizzle-orm';
import { accessSync, constants } from 'node:fs';
import { basename, dirname } from 'node:path';
@@ -60,11 +66,19 @@ export async function redownloadMissingRequest(
return { error: 'Download folder is unavailable', success: false };
}
const catchup = catchupForDownload(item);
await assertRemoteUrlAllowed(item.url, { allowPrivateNetworks: true });
const headers = await resolveStoredDownloadHeaders(db, item);
try {
removePartialDownloadFile(item.filePath);
if (catchup) {
const proof = (await readArchiveFinalizations(db, [item.id])).get(
item.id
);
removeJournaledCatchupPartial(item.filePath, proof, (path) => {
if (proof) recordArchiveCleanupPath(db, item.id, proof, path);
});
} else removePartialDownloadFile(item.filePath);
} catch (error) {
console.error(
'[Downloads] Failed to delete partial before missing-file recovery:',
@@ -80,6 +94,7 @@ export async function redownloadMissingRequest(
.update(schema.downloads)
.set({
bytesDownloaded: 0,
...(catchup ? { filePath: null } : {}),
errorMessage: null,
resumeValidator: null,
status: 'queued',
@@ -100,9 +115,10 @@ export async function redownloadMissingRequest(
}
enqueueDownload({
catchup,
directory: dirname(item.filePath),
fileName: basename(item.filePath),
filePath: item.filePath,
filePath: catchup ? null : item.filePath,
headers,
id: item.id,
resumeValidator: null,
@@ -0,0 +1,206 @@
import { ArchiveRecoveryRequiredError } from './download-catchup-capture';
import { and, eq, inArray } from 'drizzle-orm';
import { getDatabase } from '../../database/connection';
import * as schema from '../../database/schema';
import {
readArchiveFinalizations,
verifiedArchiveSize,
recordArchiveCleanupPath,
} from './download-catchup-journal';
import { removeJournaledCatchupPartial } from './download-catchup-removal';
import { removePartialDownloadFile } from './download-file-path';
import {
broadcastDownloadUpdate,
isDownloadCommitting,
hasRuntimeDownload,
prepareArchiveRemoval,
removeDownloadFromRuntime,
} from './download-runtime';
const removablePartialStatuses = new Set([
'queued',
'paused',
'completed',
'failed',
'canceled',
]);
export async function removeDownloadRequest(downloadId: number) {
try {
if (!(await prepareArchiveRemoval(downloadId)))
return {
success: false,
error: 'Download is completing; try again shortly',
};
console.log('[Downloads] Remove download:', downloadId);
const db = await getDatabase();
const rows = await db
.select({
filePath: schema.downloads.filePath,
contentType: schema.downloads.contentType,
status: schema.downloads.status,
})
.from(schema.downloads)
.where(eq(schema.downloads.id, downloadId))
.limit(1);
const row = rows[0];
const proof =
row?.contentType === 'catchup'
? (await readArchiveFinalizations(db, [downloadId])).get(
downloadId
)
: undefined;
if (
isDownloadCommitting(downloadId) ||
(row?.contentType === 'catchup' && hasRuntimeDownload(downloadId))
)
return {
success: false,
error: 'Download is completing; try again shortly',
};
// A settled archive can still say downloading when its cancellation
// status write failed. Its journal must be drained before row deletion.
if (
row?.filePath &&
(row.contentType === 'catchup' ||
removablePartialStatuses.has(row.status))
) {
try {
if (row.contentType === 'catchup')
removeJournaledCatchupPartial(
row.filePath,
proof,
(path, kind) => {
if (proof)
recordArchiveCleanupPath(
db,
downloadId,
proof,
path,
kind
);
},
row.status !== 'completed' &&
verifiedArchiveSize(row.filePath, proof) === null
);
else removePartialDownloadFile(row.filePath);
} catch (cleanupError) {
// Keep the row (and its runtime entry) so the .part is never
// orphaned, but answer with a structured failure the UI can
// surface instead of an opaque IPC rejection. Retrying the
// remove re-attempts the deletion.
console.error(
'[Downloads] Failed to delete partial file on remove:',
row.filePath,
cleanupError
);
return {
error: 'Could not delete the partial file',
...(cleanupError instanceof ArchiveRecoveryRequiredError
? { recoveryPath: cleanupError.recoveryPath }
: {}),
success: false,
};
}
}
if (removeDownloadFromRuntime(downloadId) === false)
return {
success: false,
error: 'Download is completing; try again shortly',
};
await db
.delete(schema.downloads)
.where(eq(schema.downloads.id, downloadId));
broadcastDownloadUpdate();
return { success: true };
} catch (error) {
console.error('[Downloads] Error removing download:', error);
throw error;
}
}
export async function clearCompletedDownloadsRequest(playlistId?: string) {
try {
const db = await getDatabase();
const terminalStatus = inArray(schema.downloads.status, [
'completed',
'failed',
'canceled',
]);
const terminalFilter = playlistId
? and(eq(schema.downloads.playlistId, playlistId), terminalStatus)
: terminalStatus;
const rows = await db
.select({
id: schema.downloads.id,
filePath: schema.downloads.filePath,
contentType: schema.downloads.contentType,
status: schema.downloads.status,
})
.from(schema.downloads)
.where(terminalFilter);
const proofs = await readArchiveFinalizations(
db,
rows
.filter((row) => row.contentType === 'catchup')
.map((row) => row.id)
);
const downloadIdsToDelete: number[] = [];
let recoveryPath: string | undefined;
for (const row of rows) {
if (hasRuntimeDownload(row.id)) continue;
if (row.filePath && removablePartialStatuses.has(row.status)) {
try {
if (row.contentType === 'catchup')
removeJournaledCatchupPartial(
row.filePath,
proofs.get(row.id),
(path, kind) => {
const proof = proofs.get(row.id);
if (proof)
recordArchiveCleanupPath(
db,
row.id,
proof,
path,
kind
);
},
row.status !== 'completed' &&
verifiedArchiveSize(
row.filePath,
proofs.get(row.id)
) === null
);
else removePartialDownloadFile(row.filePath);
} catch (error) {
if (error instanceof ArchiveRecoveryRequiredError)
recoveryPath ??= error.recoveryPath;
console.error(
'[Downloads] Retaining download after partial cleanup failed:',
error
);
continue;
}
}
downloadIdsToDelete.push(row.id);
}
if (downloadIdsToDelete.length > 0) {
await db
.delete(schema.downloads)
.where(
and(
terminalFilter,
inArray(schema.downloads.id, downloadIdsToDelete)
)
);
broadcastDownloadUpdate();
}
return recoveryPath
? { success: false, recoveryPath }
: { success: true };
} catch (error) {
console.error('[Downloads] Error clearing completed:', error);
throw error;
}
}
@@ -18,6 +18,8 @@ const episodeRequest = {
function createDownloadRow(overrides: Partial<DownloadRow> = {}): DownloadRow {
return {
catchup: null,
programmeStart: 0,
bytesDownloaded: 0,
contentType: 'episode',
createdAt: '2026-08-02 10:00:00',
@@ -1,5 +1,8 @@
import { and, eq, isNull, or } from 'drizzle-orm';
import type { ElectronBridgeEpisodeIdentityScope } from '@iptvnator/shared/interfaces';
import type {
CatchupDownloadMetadata,
ElectronBridgeEpisodeIdentityScope,
} from '@iptvnator/shared/interfaces';
import * as schema from '../../database/schema';
import type { DownloadsDatabase } from './download-task';
@@ -29,6 +32,7 @@ export type ExistingDownloadIdentityResolution =
DownloadIdentityNone | DownloadIdentityConflict | DownloadIdentityMatch;
export interface DownloadIdentityRequest {
catchup?: CatchupDownloadMetadata | null;
contentType: DownloadRow['contentType'];
episodeNumber?: number;
episodeIdentityScope?: ElectronBridgeEpisodeIdentityScope;
@@ -92,7 +96,13 @@ export async function resolveExistingDownloadIdentity(
and(
eq(schema.downloads.playlistId, request.playlistId),
eq(schema.downloads.contentType, request.contentType),
eq(schema.downloads.xtreamId, request.xtreamId)
eq(schema.downloads.xtreamId, request.xtreamId),
request.contentType === 'catchup'
? eq(
schema.downloads.programmeStart,
request.catchup?.startTimestamp ?? 0
)
: undefined
)
)
.limit(1);
@@ -0,0 +1,75 @@
import type {
CatchupDownloadMetadata,
DownloadContentType,
DownloadMetadataSnapshot,
ElectronBridgeEpisodeIdentityScope,
} from '@iptvnator/shared/interfaces';
import { extname } from 'node:path';
export interface StartDownloadRequest {
playlistId: string;
xtreamId: number;
contentType: DownloadContentType;
catchup?: CatchupDownloadMetadata | null;
title: string;
url: string;
posterUrl?: string;
metadataSnapshot?: DownloadMetadataSnapshot;
downloadFolder: string;
headers?: { userAgent?: string; referer?: string; origin?: string };
seriesXtreamId?: number;
seasonNumber?: number;
episodeNumber?: number;
episodeIdentityScope?: ElectronBridgeEpisodeIdentityScope;
playlistName?: string;
playlistType?: 'xtream' | 'stalker' | 'm3u-file' | 'm3u-text' | 'm3u-url';
serverUrl?: string;
portalUrl?: string;
macAddress?: string;
}
export function sanitizeFilename(name: string): string {
return name.replace(/[<>:"/\\|?*]/g, '_').trim();
}
function getExtensionFromUrl(url: string): string {
try {
// Sanitize too: URL pathnames may legally contain characters like ':'
// that would create NTFS alternate data streams on Windows.
const extension = sanitizeFilename(extname(new URL(url).pathname));
return extension.startsWith('.') ? extension : '.mp4';
} catch {
return '.mp4';
}
}
export function createFileName(title: string, url: string): string {
return sanitizeFilename(title) + getExtensionFromUrl(url);
}
export function createHeaders(
headers: StartDownloadRequest['headers']
): Record<string, string> | undefined {
if (!headers) {
return undefined;
}
const result: Record<string, string> = {};
if (headers.userAgent) {
result['User-Agent'] = headers.userAgent;
}
if (headers.origin) {
result.Origin = headers.origin;
}
if (headers.referer) {
result.Referer = headers.referer;
}
return Object.keys(result).length > 0 ? result : undefined;
}
export function serializeHeaders(
headers: Record<string, string> | undefined
): string | null {
return headers ? JSON.stringify(headers) : null;
}
@@ -1,3 +1,13 @@
import {
mkdtemp,
writeFile,
readFile,
lstat,
rename,
rm,
} from 'node:fs/promises';
import { join } from 'node:path';
import { tmpdir } from 'node:os';
import type { DownloadMetadataSnapshot } from '@iptvnator/shared/interfaces';
import type { Download } from '../../database/schema';
import type { DownloadDirectoryAuthorizer } from './download-directory-authorization';
@@ -62,6 +72,7 @@ async function setupStartMetadataRequest(
}));
jest.doMock('./download-runtime', () => ({
enqueueDownload,
hasRuntimeDownload: jest.fn().mockReturnValue(false),
}));
jest.doMock('./download-file-availability', () => ({
getDownloadFileAvailabilityAsync,
@@ -86,6 +97,8 @@ async function setupStartMetadataRequest(
function createStartDownloadRow(overrides: Partial<Download> = {}): Download {
return {
catchup: null,
programmeStart: 0,
bytesDownloaded: 0,
contentType: 'episode',
createdAt: '2026-08-02 10:00:00',
@@ -1055,3 +1068,192 @@ describe('download requests resume', () => {
expect(enqueueDownload).not.toHaveBeenCalled();
});
});
describe('catch-up submissions restarting terminal rows', () => {
it.each([false, true])(
'preserves replaced partials and retains ownership on cleanup errors (locked=%s)',
async (locked) => {
const directory = await mkdtemp(
join(tmpdir(), 'archive-resubmit-')
);
const filePath = join(directory, 'show.ts');
try {
await writeFile(filePath + '.part', 'owned bytes');
const partialIdentity = await lstat(filePath + '.part');
await rename(filePath + '.part', join(directory, 'original'));
await writeFile(filePath + '.part', 'unrelated replacement');
const catchup = {
channelName: 'News',
startTimestamp: 100,
stopTimestamp: 200,
};
const proof = {
version: 1,
phase: 'transfer',
filePath,
partialIdentity,
};
jest.doMock('./download-catchup-journal', () => ({
...jest.requireActual('./download-catchup-journal'),
readArchiveFinalizations: jest.fn(async () => {
if (locked) throw new Error('SQLITE_BUSY');
return new Map([[42, proof]]);
}),
recordArchiveCleanupPath: jest.fn(),
}));
jest.dontMock('./download-catchup-removal');
const h = await setupStartMetadataRequest(
createStartDownloadRow({
id: 42,
contentType: 'catchup',
status: 'completed',
filePath,
catchup,
programmeStart: 100,
})
);
const result = h.startDownloadRequest(
{
contentType: 'catchup',
catchup,
playlistId: 'playlist-1',
xtreamId: 77,
title: 'Show',
url: 'https://provider.test/archive.ts',
downloadFolder: directory,
},
h.authorizer
);
if (locked) await expect(result).rejects.toThrow('SQLITE_BUSY');
else
await expect(result).resolves.toMatchObject({
success: true,
});
expect(await readFile(filePath + '.part', 'utf8')).toBe(
'unrelated replacement'
);
if (locked) {
expect(h.set).not.toHaveBeenCalled();
expect(h.enqueueDownload).not.toHaveBeenCalled();
} else {
expect(h.set).toHaveBeenCalledWith(
expect.objectContaining({
filePath: null,
status: 'queued',
})
);
expect(h.enqueueDownload).toHaveBeenCalledWith(
expect.not.objectContaining({ filePath })
);
}
} finally {
jest.dontMock('./download-catchup-journal');
await rm(directory, { recursive: true, force: true });
}
}
);
});
it.each([
['failed', 'start'],
['canceled', 'start'],
['failed', 'retry'],
['canceled', 'retry'],
['paused', 'resume'],
] as const)(
'restores a journal-proven %s archive on %s instead of downloading again',
async (status, action) => {
const directory = await mkdtemp(
join(tmpdir(), 'archive-completed-retry-')
);
const filePath = join(directory, 'show.ts');
try {
await writeFile(filePath, 'complete archive');
const identity = await lstat(filePath);
const catchup = {
channelName: 'News',
startTimestamp: 100,
stopTimestamp: 200,
expiresAt: 201,
};
jest.doMock('./download-catchup-journal', () => ({
...jest.requireActual('./download-catchup-journal'),
readArchiveFinalizations: jest.fn(
async () =>
new Map([
[
42,
{
version: 1,
filePath,
size: identity.size,
partialIdentity: identity,
finalIdentity: identity,
},
],
])
),
}));
const h = await setupStartMetadataRequest(
createStartDownloadRow({
id: 42,
contentType: 'catchup',
status,
filePath,
catchup,
programmeStart: 100,
})
);
const { assertRemoteUrlAllowed } = await import('../url-safety');
jest.mocked(assertRemoteUrlAllowed).mockRejectedValue(
new Error('ENOTFOUND')
);
jest.mocked(h.authorizer.requireAuthorized).mockRejectedValue(
new Error('Folder unavailable')
);
const result =
action === 'start'
? h.startDownloadRequest(
{
contentType: 'catchup',
catchup,
playlistId: 'playlist-1',
xtreamId: 77,
title: 'Show',
url: 'https://provider.test/archive.ts',
downloadFolder: directory,
},
h.authorizer
)
: action === 'retry'
? (
await import('./download-resume-requests')
).retryDownloadRequest(42, directory, h.authorizer)
: (
await import('./download-resume-requests')
).resumeDownloadRequest(42, directory, h.authorizer);
await expect(result).resolves.toMatchObject(
action === 'start'
? { success: false, reason: 'already-downloaded' }
: { success: true }
);
expect(h.set).toHaveBeenCalledWith(
expect.objectContaining({
status: 'completed',
bytesDownloaded: identity.size,
totalBytes: identity.size,
})
);
expect(h.set).not.toHaveBeenCalledWith(
expect.objectContaining({ filePath: null })
);
expect(h.enqueueDownload).not.toHaveBeenCalled();
expect(assertRemoteUrlAllowed).not.toHaveBeenCalled();
expect(h.authorizer.requireAuthorized).not.toHaveBeenCalled();
expect(await readFile(filePath, 'utf8')).toBe('complete archive');
} finally {
jest.dontMock('./download-catchup-journal');
await rm(directory, { recursive: true, force: true });
}
}
);
@@ -1,11 +1,17 @@
import type {
DownloadMetadataSnapshot,
ElectronBridgeEpisodeIdentityScope,
ElectronBridgeDownloadStartResult,
} from '@iptvnator/shared/interfaces';
import {
sanitizeFilename,
createFileName,
createHeaders,
serializeHeaders,
type StartDownloadRequest,
} from './download-request-options';
export type { StartDownloadRequest } from './download-request-options';
import { recoverStoredCatchupCompletion } from './download-catchup-recover-completion';
import { cleanupStoredCatchupPartial } from './download-catchup-removal';
import { catchupForDownload } from './download-catchup';
import type { ElectronBridgeDownloadStartResult } from '@iptvnator/shared/interfaces';
import { ELECTRON_BRIDGE_DOWNLOAD_START_REASONS } from '@iptvnator/shared/interfaces';
import { and, eq, sql } from 'drizzle-orm';
import { basename, dirname, extname } from 'node:path';
import { eq, sql } from 'drizzle-orm';
import { getDatabase } from '../../database/connection';
import * as schema from '../../database/schema';
import { assertRemoteUrlAllowed } from '../url-safety';
@@ -13,81 +19,13 @@ import { DownloadDirectoryAuthorizer } from './download-directory-authorization'
import { getDownloadFileAvailabilityWithTimeoutAsync } from './download-file-availability';
import { removePartialDownloadFileAsync } from './download-partial-cleanup';
import { resolveExistingDownloadIdentity } from './download-request-identity';
import { resolveStoredDownloadHeaders } from './download-request-headers';
import {
assertDownloadMetadataArtworkDiffersFromStream,
assertDownloadMetadataMatchesContentType,
decodeDownloadMetadataSnapshot,
encodeDownloadMetadataSnapshot,
} from './download-metadata-snapshot';
import { enqueueDownload } from './download-runtime';
export interface StartDownloadRequest {
playlistId: string;
xtreamId: number;
contentType: 'vod' | 'episode';
title: string;
url: string;
posterUrl?: string;
metadataSnapshot?: DownloadMetadataSnapshot;
downloadFolder: string;
headers?: { userAgent?: string; referer?: string; origin?: string };
seriesXtreamId?: number;
seasonNumber?: number;
episodeNumber?: number;
episodeIdentityScope?: ElectronBridgeEpisodeIdentityScope;
playlistName?: string;
playlistType?: 'xtream' | 'stalker' | 'm3u-file' | 'm3u-text' | 'm3u-url';
serverUrl?: string;
portalUrl?: string;
macAddress?: string;
}
function sanitizeFilename(name: string): string {
return name.replace(/[<>:"/\\|?*]/g, '_').trim();
}
function getExtensionFromUrl(url: string): string {
try {
// Sanitize too: URL pathnames may legally contain characters like ':'
// that would create NTFS alternate data streams on Windows.
const extension = sanitizeFilename(extname(new URL(url).pathname));
return extension.startsWith('.') ? extension : '.mp4';
} catch {
return '.mp4';
}
}
function createFileName(title: string, url: string): string {
return sanitizeFilename(title) + getExtensionFromUrl(url);
}
function createHeaders(
headers: StartDownloadRequest['headers']
): Record<string, string> | undefined {
if (!headers) {
return undefined;
}
const result: Record<string, string> = {};
if (headers.userAgent) {
result['User-Agent'] = headers.userAgent;
}
if (headers.origin) {
result.Origin = headers.origin;
}
if (headers.referer) {
result.Referer = headers.referer;
}
return Object.keys(result).length > 0 ? result : undefined;
}
function serializeHeaders(
headers: Record<string, string> | undefined
): string | null {
return headers ? JSON.stringify(headers) : null;
}
import { enqueueDownload, hasRuntimeDownload } from './download-runtime';
export async function startDownloadRequest(
data: StartDownloadRequest,
@@ -108,8 +46,6 @@ export async function startDownloadRequest(
throw new Error('Invalid download metadata snapshot');
}
console.log('[Downloads] Enqueue download:', data.title);
const directory = await authorizer.requireAuthorized(data.downloadFolder);
await assertRemoteUrlAllowed(data.url, { allowPrivateNetworks: true });
const db = await getDatabase();
if (!data.playlistId) {
@@ -128,8 +64,6 @@ export async function startDownloadRequest(
success: false,
};
}
const fileName = createFileName(data.title, data.url);
const headers = createHeaders(data.headers);
if (identity.kind === 'match') {
const item = identity.item;
@@ -147,7 +81,11 @@ export async function startDownloadRequest(
data.url
);
}
if (item.contentType === 'episode' && item.status === 'completed') {
if (
(item.contentType === 'episode' ||
item.contentType === 'catchup') &&
item.status === 'completed'
) {
const completedFileAvailability =
await getDownloadFileAvailabilityWithTimeoutAsync(item);
if (completedFileAvailability === 'unknown') {
@@ -175,6 +113,31 @@ export async function startDownloadRequest(
};
}
if (
await recoverStoredCatchupCompletion(db, item, () =>
hasRuntimeDownload(item.id)
)
) {
return {
id: item.id,
success: false,
error: 'Download already completed',
reason: ELECTRON_BRIDGE_DOWNLOAD_START_REASONS.AlreadyDownloaded,
};
}
}
// These checks authorize another remote transfer, not reuse of a proven file.
const catchup = catchupForDownload(data);
const directory = await authorizer.requireAuthorized(data.downloadFolder);
await assertRemoteUrlAllowed(data.url, { allowPrivateNetworks: true });
const fileName = catchup
? sanitizeFilename(data.title) + '.ts'
: createFileName(data.title, data.url);
const headers = createHeaders(data.headers);
if (identity.kind === 'match') {
const item = identity.item;
if (
['completed', 'failed', 'canceled'].includes(item.status) &&
item.filePath
@@ -182,8 +145,17 @@ export async function startDownloadRequest(
// A terminal row can still reference a retained .part; delete it
// before the restart clears filePath, or the file is orphaned.
// An unavailable or slow .part must keep its database owner.
const cleanup = await removePartialDownloadFileAsync(item.filePath);
if (cleanup === 'unknown') {
const cleaned =
item.contentType === 'catchup'
? await cleanupStoredCatchupPartial(
db,
item.id,
item.filePath,
'incomplete'
)
: (await removePartialDownloadFileAsync(item.filePath)) !==
'unknown';
if (!cleaned) {
console.error(
'[Downloads] Could not verify retained partial cleanup'
);
@@ -198,6 +170,8 @@ export async function startDownloadRequest(
await db
.update(schema.downloads)
.set({
catchup,
programmeStart: catchup?.startTimestamp ?? 0,
bytesDownloaded: 0,
errorMessage: null,
fileName,
@@ -220,6 +194,7 @@ export async function startDownloadRequest(
})
.where(eq(schema.downloads.id, item.id));
enqueueDownload({
catchup,
directory,
fileName,
headers,
@@ -256,6 +231,8 @@ export async function startDownloadRequest(
const result = await db.insert(schema.downloads).values({
contentType: data.contentType,
catchup,
programmeStart: catchup?.startTimestamp ?? 0,
episodeNumber: data.episodeNumber,
episodeIdentityScope: data.episodeIdentityScope,
fileName,
@@ -272,6 +249,7 @@ export async function startDownloadRequest(
});
const insertedId = Number(result.lastInsertRowid);
enqueueDownload({
catchup,
directory,
fileName,
headers,
@@ -281,155 +259,7 @@ export async function startDownloadRequest(
return { id: insertedId, success: true };
}
export async function retryDownloadRequest(
downloadId: number,
downloadFolder: string,
authorizer: DownloadDirectoryAuthorizer
): Promise<{ success: boolean; error?: string }> {
console.log('[Downloads] Retry download:', downloadId);
const db = await getDatabase();
const existing = await db
.select()
.from(schema.downloads)
.where(eq(schema.downloads.id, downloadId))
.limit(1);
if (existing.length === 0) {
return { error: 'Download not found', success: false };
}
const item = existing[0];
await assertRemoteUrlAllowed(item.url, { allowPrivateNetworks: true });
if (!['failed', 'canceled'].includes(item.status)) {
return {
error: 'Can only retry failed or canceled downloads',
success: false,
};
}
const retainedFilePath =
item.status === 'failed' && item.filePath ? item.filePath : null;
// A retained filePath was written by the main process after its folder
// was authorized; requiring the folder to still be the CURRENT selection
// would strand the retry after the user switches download folders.
const directory = retainedFilePath
? dirname(retainedFilePath)
: await authorizer.requireAuthorized(downloadFolder);
const fileName = retainedFilePath
? basename(retainedFilePath)
: createFileName(item.title, item.url);
const headers = await resolveStoredDownloadHeaders(db, item);
const queuedUpdate = retainedFilePath
? {
errorMessage: null,
fileName,
status: 'queued' as const,
updatedAt: sql`CURRENT_TIMESTAMP`,
}
: {
bytesDownloaded: 0,
errorMessage: null,
fileName,
filePath: null,
resumeValidator: null,
status: 'queued' as const,
totalBytes: null,
updatedAt: sql`CURRENT_TIMESTAMP`,
};
await db
.update(schema.downloads)
.set(queuedUpdate)
.where(eq(schema.downloads.id, downloadId));
enqueueDownload({
directory,
fileName,
filePath: retainedFilePath,
headers,
id: item.id,
resumeValidator: retainedFilePath ? item.resumeValidator : null,
totalBytes: retainedFilePath ? item.totalBytes : null,
url: item.url,
});
return { success: true };
}
export async function resumeDownloadRequest(
downloadId: number,
downloadFolder: string,
authorizer: DownloadDirectoryAuthorizer
): Promise<{ success: boolean; error?: string }> {
console.log('[Downloads] Resume download:', downloadId);
const db = await getDatabase();
const existing = await db
.select()
.from(schema.downloads)
.where(eq(schema.downloads.id, downloadId))
.limit(1);
if (existing.length === 0) {
return { error: 'Download not found', success: false };
}
const item = existing[0];
await assertRemoteUrlAllowed(item.url, { allowPrivateNetworks: true });
if (item.status !== 'paused') {
return {
error: 'Can only resume paused downloads',
success: false,
};
}
// See retryDownloadRequest: DB-recorded retained paths stay usable after
// the user switches download folders.
const directory = item.filePath
? dirname(item.filePath)
: await authorizer.requireAuthorized(downloadFolder);
const fileName = item.filePath
? basename(item.filePath)
: createFileName(item.title, item.url);
const headers = await resolveStoredDownloadHeaders(db, item);
// Claim the row atomically: a concurrent resume for the same id loses
// this conditional update and must not enqueue a second task.
const claim = await db
.update(schema.downloads)
.set({
errorMessage: null,
fileName,
status: 'queued',
updatedAt: sql`CURRENT_TIMESTAMP`,
})
.where(
and(
eq(schema.downloads.id, downloadId),
eq(schema.downloads.status, 'paused')
)
);
if (hasNoChanges(claim)) {
return {
error: 'Can only resume paused downloads',
success: false,
};
}
enqueueDownload({
directory,
fileName,
filePath: item.filePath,
headers,
id: item.id,
resumeValidator: item.resumeValidator,
totalBytes: item.totalBytes,
url: item.url,
});
return { success: true };
}
function hasNoChanges(result: unknown): boolean {
return (
typeof result === 'object' &&
result !== null &&
'changes' in result &&
(result as { changes: number }).changes === 0
);
}
export {
retryDownloadRequest,
resumeDownloadRequest,
} from './download-resume-requests';
@@ -0,0 +1,188 @@
import { recoverStoredCatchupCompletion } from './download-catchup-recover-completion';
import { and, eq, sql } from 'drizzle-orm';
import { basename, dirname } from 'node:path';
import { getDatabase } from '../../database/connection';
import * as schema from '../../database/schema';
import { assertRemoteUrlAllowed } from '../url-safety';
import { DownloadDirectoryAuthorizer } from './download-directory-authorization';
import { catchupForDownload } from './download-catchup';
import { sanitizeFilename, createFileName } from './download-request-options';
import { resolveStoredDownloadHeaders } from './download-request-headers';
import { enqueueDownload, hasRuntimeDownload } from './download-runtime';
export async function retryDownloadRequest(
downloadId: number,
downloadFolder: string,
authorizer: DownloadDirectoryAuthorizer
): Promise<{ success: boolean; error?: string }> {
console.log('[Downloads] Retry download:', downloadId);
const db = await getDatabase();
const existing = await db
.select()
.from(schema.downloads)
.where(eq(schema.downloads.id, downloadId))
.limit(1);
if (existing.length === 0) {
return { error: 'Download not found', success: false };
}
const item = existing[0];
if (!['failed', 'canceled'].includes(item.status)) {
return {
error: 'Can only retry failed or canceled downloads',
success: false,
};
}
if (
await recoverStoredCatchupCompletion(db, item, () =>
hasRuntimeDownload(item.id)
)
)
return { success: true };
const catchup = catchupForDownload(item);
await assertRemoteUrlAllowed(item.url, { allowPrivateNetworks: true });
const retainedFilePath =
(item.status === 'failed' || catchup) && item.filePath
? item.filePath
: null;
// A retained filePath was written by the main process after its folder
// was authorized; requiring the folder to still be the CURRENT selection
// would strand the retry after the user switches download folders.
const directory = retainedFilePath
? dirname(retainedFilePath)
: await authorizer.requireAuthorized(downloadFolder);
const fileName = retainedFilePath
? basename(retainedFilePath)
: catchup
? sanitizeFilename(item.title) + '.ts'
: createFileName(item.title, item.url);
const headers = await resolveStoredDownloadHeaders(db, item);
const queuedUpdate = retainedFilePath
? {
errorMessage: null,
fileName,
status: 'queued' as const,
updatedAt: sql`CURRENT_TIMESTAMP`,
}
: {
bytesDownloaded: 0,
errorMessage: null,
fileName,
filePath: null,
resumeValidator: null,
status: 'queued' as const,
totalBytes: null,
updatedAt: sql`CURRENT_TIMESTAMP`,
};
await db
.update(schema.downloads)
.set(queuedUpdate)
.where(eq(schema.downloads.id, downloadId));
enqueueDownload({
catchup,
directory,
fileName,
filePath: retainedFilePath,
headers,
id: item.id,
resumeValidator: retainedFilePath ? item.resumeValidator : null,
totalBytes: retainedFilePath ? item.totalBytes : null,
url: item.url,
});
return { success: true };
}
export async function resumeDownloadRequest(
downloadId: number,
downloadFolder: string,
authorizer: DownloadDirectoryAuthorizer
): Promise<{ success: boolean; error?: string }> {
console.log('[Downloads] Resume download:', downloadId);
const db = await getDatabase();
const existing = await db
.select()
.from(schema.downloads)
.where(eq(schema.downloads.id, downloadId))
.limit(1);
if (existing.length === 0) {
return { error: 'Download not found', success: false };
}
const item = existing[0];
if (item.status !== 'paused') {
return {
error: 'Can only resume paused downloads',
success: false,
};
}
if (
await recoverStoredCatchupCompletion(db, item, () =>
hasRuntimeDownload(item.id)
)
)
return { success: true };
const catchup = catchupForDownload(item);
await assertRemoteUrlAllowed(item.url, { allowPrivateNetworks: true });
// See retryDownloadRequest: DB-recorded retained paths stay usable after
// the user switches download folders.
const directory = item.filePath
? dirname(item.filePath)
: await authorizer.requireAuthorized(downloadFolder);
const fileName = item.filePath
? basename(item.filePath)
: catchup
? sanitizeFilename(item.title) + '.ts'
: createFileName(item.title, item.url);
const headers = await resolveStoredDownloadHeaders(db, item);
// Claim the row atomically: a concurrent resume for the same id loses
// this conditional update and must not enqueue a second task.
const claim = await db
.update(schema.downloads)
.set({
errorMessage: null,
fileName,
status: 'queued',
updatedAt: sql`CURRENT_TIMESTAMP`,
})
.where(
and(
eq(schema.downloads.id, downloadId),
eq(schema.downloads.status, 'paused')
)
);
if (hasNoChanges(claim)) {
return {
error: 'Can only resume paused downloads',
success: false,
};
}
enqueueDownload({
catchup,
directory,
fileName,
filePath: item.filePath,
headers,
id: item.id,
resumeValidator: item.resumeValidator,
totalBytes: item.totalBytes,
url: item.url,
});
return { success: true };
}
function hasNoChanges(result: unknown): boolean {
return (
typeof result === 'object' &&
result !== null &&
'changes' in result &&
(result as { changes: number }).changes === 0
);
}
@@ -0,0 +1,78 @@
import { eq, sql } from 'drizzle-orm';
import * as schema from '../../database/schema';
import { cleanupStoredCatchupPartial } from './download-catchup-removal';
import { getPausedByteCount, removePartialFile } from './download-finalize';
import type { DownloadsDatabase, DownloadTask } from './download-task';
export async function persistCancellation(
db: DownloadsDatabase,
task: DownloadTask
): Promise<void> {
console.log(`[Downloads] Canceled: ${task.fileName}`);
const removed = task.catchup
? await cleanupStoredCatchupPartial(db, task.id, task.filePath, true)
: removePartialFile(task.filePath);
try {
await db
.update(schema.downloads)
.set({
bytesDownloaded: 0,
errorMessage: null,
filePath: removed ? null : (task.filePath ?? null),
resumeValidator: null,
status: 'canceled',
totalBytes: null,
updatedAt: sql`CURRENT_TIMESTAMP`,
})
.where(eq(schema.downloads.id, task.id));
} catch (error) {
console.error('[Downloads] Failed to persist cancellation:', error);
}
}
export async function persistPause(
db: DownloadsDatabase,
task: DownloadTask
): Promise<void> {
console.log(`[Downloads] Paused: ${task.fileName}`);
const bytesDownloaded = getPausedByteCount(task);
try {
await db
.update(schema.downloads)
.set({
bytesDownloaded,
errorMessage: null,
fileName: task.fileName,
filePath: task.filePath ?? null,
// Keep a mid-attempt validator promotion (complete overlap
// match) across pause/resume.
resumeValidator: task.resumeValidator ?? null,
status: 'paused',
totalBytes: task.totalBytes ?? null,
updatedAt: sql`CURRENT_TIMESTAMP`,
})
.where(eq(schema.downloads.id, task.id));
} catch (error) {
console.error('[Downloads] Failed to persist pause:', error);
}
}
export async function persistQueuedCancellation(
db: DownloadsDatabase,
downloadId: number,
// Keep the path when the retained .part could not be deleted, so a later
// remove/clear can retry the cleanup instead of orphaning the file.
retainedFilePath: string | null = null
): Promise<void> {
await db
.update(schema.downloads)
.set({
bytesDownloaded: 0,
errorMessage: null,
filePath: retainedFilePath,
resumeValidator: null,
status: 'canceled',
totalBytes: null,
updatedAt: sql`CURRENT_TIMESTAMP`,
})
.where(eq(schema.downloads.id, downloadId));
}
@@ -0,0 +1,71 @@
import { existsSync } from 'node:fs';
import { rename } from 'node:fs/promises';
import {
readArchiveFinalizations,
verifiedArchiveSize,
} from './download-catchup-journal';
import { cleanupStoredCatchupFinal } from './download-catchup-removal';
import {
reserveFreshCatchupTarget,
reserveOwnedCatchupTarget,
} from './download-catchup-reservation';
import {
findAvailableFinalPath,
getPartialDownloadPath,
reserveAvailablePartialDownloadFile,
type ReservedPartialDownloadFile,
} from './download-file-path';
import type { DownloadsDatabase, DownloadTask } from './download-task';
export async function reserveTarget(
db: DownloadsDatabase,
task: DownloadTask
): Promise<ReservedPartialDownloadFile> {
if (task.filePath) {
if (task.catchup) {
const proof = (await readArchiveFinalizations(db, [task.id])).get(
task.id
);
if (
proof &&
proof.phase !== 'transfer' &&
verifiedArchiveSize(task.filePath, proof) === null &&
!(await cleanupStoredCatchupFinal(db, task.id, task.filePath))
) {
throw new Error(
'Could not clean the interrupted archive promotion'
);
}
}
if (!existsSync(task.filePath)) {
return {
filename: task.fileName,
partialPath: getPartialDownloadPath(task.filePath),
path: task.filePath,
};
}
if (task.catchup) return reserveFreshCatchupTarget(db, task);
// Something now occupies the recorded destination — possibly a file
// the user created while this download was paused or failed. Never
// inspect or delete it: move the retained .part to the next free
// numbered destination and finalize there instead.
const redirected = findAvailableFinalPath(task.filePath);
const currentPartial = getPartialDownloadPath(task.filePath);
const redirectedPartial = getPartialDownloadPath(redirected.path);
if (existsSync(currentPartial)) {
await rename(currentPartial, redirectedPartial);
}
return {
filename: redirected.filename,
partialPath: redirectedPartial,
path: redirected.path,
};
}
return task.catchup
? reserveOwnedCatchupTarget(db, task)
: reserveAvailablePartialDownloadFile(task.directory, task.fileName);
}
@@ -1,19 +1,24 @@
import { cleanupArchiveCapture } from './download-catchup-capture';
import { reserveTarget } from './download-runtime-reservation';
import {
clearArchiveFinalization,
readArchiveFinalizations,
} from './download-catchup-journal';
import { cleanupStoredCatchupPartial } from './download-catchup-removal';
import {
persistCancellation,
persistPause,
persistQueuedCancellation,
} from './download-runtime-persistence';
import { transferCatchupToPartialFile } from './download-catchup-transfer';
import { eq, sql } from 'drizzle-orm';
import { existsSync } from 'node:fs';
import { rename } from 'node:fs/promises';
import { getDatabase } from '../../database/connection';
import * as schema from '../../database/schema';
import { broadcastDownloadUpdate } from './download-broadcast';
import {
findAvailableFinalPath,
getPartialDownloadPath,
reserveAvailablePartialDownloadFile,
type ReservedPartialDownloadFile,
} from './download-file-path';
import type { ReservedPartialDownloadFile } from './download-file-path';
import {
completeDownloadFromPartial,
getCompletedPartialProgress,
getPausedByteCount,
handleDownloadFailure,
removePartialFile,
} from './download-finalize';
@@ -21,7 +26,6 @@ import { transferWithReconnects } from './download-reconnect';
import {
requestDownloadCancellation,
requestDownloadPause,
type DownloadsDatabase,
type DownloadTask,
} from './download-task';
import { describeError } from './download-transfer';
@@ -30,6 +34,7 @@ export { broadcastDownloadUpdate, setMainWindow } from './download-broadcast';
const downloadQueue: DownloadTask[] = [];
let activeDownload: DownloadTask | null = null;
const settlementWaiters = new Map<number, Array<() => void>>();
export function enqueueDownload(task: DownloadTask): void {
// A duplicate id (e.g. two rapid Resume clicks racing the status
@@ -48,8 +53,7 @@ export function enqueueDownload(task: DownloadTask): void {
export async function pauseDownload(downloadId: number): Promise<boolean> {
if (activeDownload?.id === downloadId) {
requestDownloadPause(activeDownload);
return true;
return requestDownloadPause(activeDownload);
}
const queueIndex = downloadQueue.findIndex(
@@ -75,8 +79,7 @@ export async function pauseDownload(downloadId: number): Promise<boolean> {
export async function cancelDownload(downloadId: number): Promise<boolean> {
if (activeDownload?.id === downloadId) {
requestDownloadCancellation(activeDownload);
return true;
return requestDownloadCancellation(activeDownload);
}
const queueIndex = downloadQueue.findIndex(
@@ -84,8 +87,15 @@ export async function cancelDownload(downloadId: number): Promise<boolean> {
);
if (queueIndex !== -1) {
const [queuedTask] = downloadQueue.splice(queueIndex, 1);
const removed = removePartialFile(queuedTask?.filePath);
const db = await getDatabase();
const removed = queuedTask?.catchup
? await cleanupStoredCatchupPartial(
db,
downloadId,
queuedTask.filePath,
true
)
: removePartialFile(queuedTask?.filePath);
await persistQueuedCancellation(
db,
downloadId,
@@ -99,6 +109,7 @@ export async function cancelDownload(downloadId: number): Promise<boolean> {
const rows = await db
.select({
filePath: schema.downloads.filePath,
contentType: schema.downloads.contentType,
status: schema.downloads.status,
})
.from(schema.downloads)
@@ -109,7 +120,15 @@ export async function cancelDownload(downloadId: number): Promise<boolean> {
return false;
}
const removed = removePartialFile(item.filePath);
const removed =
item.contentType === 'catchup'
? await cleanupStoredCatchupPartial(
db,
downloadId,
item.filePath,
true
)
: removePartialFile(item.filePath);
await persistQueuedCancellation(
db,
downloadId,
@@ -119,9 +138,42 @@ export async function cancelDownload(downloadId: number): Promise<boolean> {
return true;
}
export function removeDownloadFromRuntime(downloadId: number): void {
/** Keep the journal row until an accepted active cancellation has settled. */
export async function prepareArchiveRemoval(
downloadId: number
): Promise<boolean> {
if (activeDownload?.id === downloadId && activeDownload.catchup) {
if (!requestDownloadCancellation(activeDownload)) return false;
await new Promise<void>((resolve) => {
const waiters = settlementWaiters.get(downloadId) ?? [];
waiters.push(resolve);
settlementWaiters.set(downloadId, waiters);
});
} else if (
downloadQueue.some((task) => task.id === downloadId && task.catchup)
) {
await cancelDownload(downloadId);
}
return true;
}
export function hasRuntimeDownload(downloadId: number): boolean {
return (
activeDownload?.id === downloadId ||
downloadQueue.some((task) => task.id === downloadId)
);
}
export function isDownloadCommitting(downloadId: number): boolean {
return (
activeDownload?.id === downloadId &&
!!activeDownload.catchupCommitStarted
);
}
export function removeDownloadFromRuntime(downloadId: number): boolean {
if (activeDownload?.id === downloadId) {
requestDownloadCancellation(activeDownload);
if (!requestDownloadCancellation(activeDownload)) return false;
}
const queueIndex = downloadQueue.findIndex(
@@ -130,6 +182,7 @@ export function removeDownloadFromRuntime(downloadId: number): void {
if (queueIndex !== -1) {
downloadQueue.splice(queueIndex, 1);
}
return true;
}
async function processQueue(): Promise<void> {
@@ -158,31 +211,12 @@ function finishTask(task: DownloadTask): void {
if (activeDownload === task) {
activeDownload = null;
}
for (const resolve of settlementWaiters.get(task.id) ?? []) resolve();
settlementWaiters.delete(task.id);
broadcastDownloadUpdate();
void processQueue();
}
async function persistQueuedCancellation(
db: DownloadsDatabase,
downloadId: number,
// Keep the path when the retained .part could not be deleted, so a later
// remove/clear can retry the cleanup instead of orphaning the file.
retainedFilePath: string | null = null
): Promise<void> {
await db
.update(schema.downloads)
.set({
bytesDownloaded: 0,
errorMessage: null,
filePath: retainedFilePath,
resumeValidator: null,
status: 'canceled',
totalBytes: null,
updatedAt: sql`CURRENT_TIMESTAMP`,
})
.where(eq(schema.downloads.id, downloadId));
}
async function startDownload(task: DownloadTask): Promise<void> {
const db = await getDatabase();
await db
@@ -206,7 +240,23 @@ async function startDownload(task: DownloadTask): Promise<void> {
return;
}
reservation = await reserveTarget(task);
if (task.catchup) {
if (task.filePath) {
const proof = (
await readArchiveFinalizations(db, [task.id])
).get(task.id);
cleanupArchiveCapture(proof);
task.catchupExpectedPartialIdentity = proof?.partialIdentity;
// No durable ownership evidence: preserve the old entry and
// reserve a fresh destination instead of adopting it.
if (!proof) task.filePath = null;
}
if (!task.filePath) {
// A fresh reservation must not inherit a previous attempt's proof.
await clearArchiveFinalization(db, task.id);
}
}
reservation = await reserveTarget(db, task);
task.fileName = reservation.filename;
task.filePath = reservation.path;
await db
@@ -239,7 +289,9 @@ async function startDownload(task: DownloadTask): Promise<void> {
return;
}
const progress = await transferWithReconnects(db, task, reservation);
const progress = await (task.catchup
? transferCatchupToPartialFile(db, task, reservation)
: transferWithReconnects(db, task, reservation));
if (task.cancelRequested) {
await persistCancellation(db, task);
return;
@@ -266,87 +318,3 @@ async function startDownload(task: DownloadTask): Promise<void> {
finishTask(task);
}
}
async function reserveTarget(
task: DownloadTask
): Promise<ReservedPartialDownloadFile> {
if (task.filePath) {
if (!existsSync(task.filePath)) {
return {
filename: task.fileName,
partialPath: getPartialDownloadPath(task.filePath),
path: task.filePath,
};
}
// Something now occupies the recorded destination — possibly a file
// the user created while this download was paused or failed. Never
// inspect or delete it: move the retained .part to the next free
// numbered destination and finalize there instead.
const redirected = findAvailableFinalPath(task.filePath);
const currentPartial = getPartialDownloadPath(task.filePath);
const redirectedPartial = getPartialDownloadPath(redirected.path);
if (existsSync(currentPartial)) {
await rename(currentPartial, redirectedPartial);
}
return {
filename: redirected.filename,
partialPath: redirectedPartial,
path: redirected.path,
};
}
return reserveAvailablePartialDownloadFile(task.directory, task.fileName);
}
async function persistCancellation(
db: DownloadsDatabase,
task: DownloadTask
): Promise<void> {
console.log(`[Downloads] Canceled: ${task.fileName}`);
const removed = removePartialFile(task.filePath);
try {
await db
.update(schema.downloads)
.set({
bytesDownloaded: 0,
errorMessage: null,
filePath: removed ? null : (task.filePath ?? null),
resumeValidator: null,
status: 'canceled',
totalBytes: null,
updatedAt: sql`CURRENT_TIMESTAMP`,
})
.where(eq(schema.downloads.id, task.id));
} catch (error) {
console.error('[Downloads] Failed to persist cancellation:', error);
}
}
async function persistPause(
db: DownloadsDatabase,
task: DownloadTask
): Promise<void> {
console.log(`[Downloads] Paused: ${task.fileName}`);
const bytesDownloaded = getPausedByteCount(task);
try {
await db
.update(schema.downloads)
.set({
bytesDownloaded,
errorMessage: null,
fileName: task.fileName,
filePath: task.filePath ?? null,
// Keep a mid-attempt validator promotion (complete overlap
// match) across pause/resume.
resumeValidator: task.resumeValidator ?? null,
status: 'paused',
totalBytes: task.totalBytes ?? null,
updatedAt: sql`CURRENT_TIMESTAMP`,
})
.where(eq(schema.downloads.id, task.id));
} catch (error) {
console.error('[Downloads] Failed to persist pause:', error);
}
}
@@ -1,3 +1,5 @@
import type { ArchiveFileIdentity } from './download-catchup-output';
import type { CatchupDownloadMetadata } from '@iptvnator/shared/interfaces';
import type { getDatabase } from '../../database/connection';
export type DownloadsDatabase = Awaited<ReturnType<typeof getDatabase>>;
@@ -12,6 +14,15 @@ export interface CompletedPartialProgress extends TransferProgress {
}
export interface DownloadTask {
catchup?: CatchupDownloadMetadata;
catchupPartialIdentity?: ArchiveFileIdentity;
catchupExpectedPartialIdentity?: ArchiveFileIdentity;
catchupCommitStarted?: boolean;
catchupFinalized?: {
filePath: string;
identity: ArchiveFileIdentity;
size: number;
};
id: number;
url: string;
fileName: string;
@@ -42,12 +53,16 @@ export interface DownloadTask {
probeEof?: boolean;
}
export function requestDownloadCancellation(task: DownloadTask): void {
export function requestDownloadCancellation(task: DownloadTask): boolean {
if (task.catchupCommitStarted) return false;
task.cancelRequested = true;
task.abortController?.abort();
return true;
}
export function requestDownloadPause(task: DownloadTask): void {
export function requestDownloadPause(task: DownloadTask): boolean {
if (task.catchupCommitStarted) return false;
task.pauseRequested = true;
task.abortController?.abort();
return true;
}
@@ -4,6 +4,13 @@ import {
mockBroadcastDownloadUpdate,
mockDownloadRow,
mockRemoveDownloadFromRuntime,
mockIsDownloadCommitting,
mockHasRuntimeDownload,
mockPrepareArchiveRemoval,
mockRemoveJournaledPartial,
mockArchiveProofs,
mockVerifiedArchiveSize,
mockRecordArchiveCleanupPath,
mockRemovePartialDownloadFile,
mockTerminalRows,
setupDownloadsEventsHarness,
@@ -14,6 +21,248 @@ describe('downloads events: partial-file cleanup', () => {
await setupDownloadsEventsHarness();
});
it.each(['remove', 'clear'])(
'returns a recoverable path and retains the row on %s',
async (action) => {
const { ArchiveRecoveryRequiredError } =
await import('./download-catchup-capture');
const row = {
...createDownloadRow('failed'),
contentType: 'catchup',
};
const { deleteWhere } =
action === 'remove'
? mockDownloadRow(row)
: mockTerminalRows([row]);
const recoveryPath = '/downloads/.iptvnator-cleanup-test/entry';
mockRemoveJournaledPartial.mockImplementation(() => {
throw new ArchiveRecoveryRequiredError(recoveryPath);
});
const consoleError = jest
.spyOn(console, 'error')
.mockImplementation(() => undefined);
try {
const result =
action === 'remove'
? await getHandler('DOWNLOADS_REMOVE')(null, 42)
: await getHandler('DOWNLOADS_CLEAR_COMPLETED')(null);
expect(result).toMatchObject({ success: false, recoveryPath });
expect(deleteWhere).not.toHaveBeenCalled();
expect(mockRemoveDownloadFromRuntime).not.toHaveBeenCalled();
} finally {
consoleError.mockRestore();
}
}
);
it('waits for active archive cancellation before reading or deleting the row', async () => {
const { db, deleteWhere } = mockDownloadRow(
createDownloadRow('canceled')
);
let release!: (ready: boolean) => void;
mockPrepareArchiveRemoval.mockReturnValue(
new Promise<boolean>((resolve) => {
release = resolve;
})
);
const remove = getHandler('DOWNLOADS_REMOVE')(null, 42);
expect(db.select).not.toHaveBeenCalled();
expect(deleteWhere).not.toHaveBeenCalled();
release(true);
await expect(remove).resolves.toEqual({ success: true });
});
it('rejects Remove before reading storage when completion already owns the task', async () => {
const { db, deleteWhere } = mockDownloadRow(
createDownloadRow('downloading')
);
mockPrepareArchiveRemoval.mockResolvedValue(false);
await expect(
getHandler('DOWNLOADS_REMOVE')(null, 42)
).resolves.toMatchObject({ success: false });
expect(db.select).not.toHaveBeenCalled();
expect(deleteWhere).not.toHaveBeenCalled();
});
it('keeps a new archive attempt that starts while Remove reads storage', async () => {
const { deleteWhere } = mockDownloadRow({
...createDownloadRow('failed'),
contentType: 'catchup',
});
mockHasRuntimeDownload.mockReturnValue(true);
await expect(
getHandler('DOWNLOADS_REMOVE')(null, 42)
).resolves.toMatchObject({ success: false });
expect(mockRemoveJournaledPartial).not.toHaveBeenCalled();
expect(deleteWhere).not.toHaveBeenCalled();
});
it('preserves completed archive media when removing its library row', async () => {
mockDownloadRow({
...createDownloadRow('completed'),
contentType: 'catchup',
});
await expect(getHandler('DOWNLOADS_REMOVE')(null, 42)).resolves.toEqual(
{ success: true }
);
expect(mockRemoveJournaledPartial).toHaveBeenCalledWith(
'/downloads/resume.mp4',
undefined,
expect.any(Function),
false
);
});
it.each([false, true])(
'retries settled archive cleanup despite a failed cancellation status write (locked=%s)',
async (locked) => {
const { deleteWhere } = mockDownloadRow({
...createDownloadRow('downloading'),
contentType: 'catchup',
});
const proof = {
version: 1,
phase: 'transfer',
partialCleanupPath: '/downloads/.iptvnator-cleanup-test/entry',
};
mockArchiveProofs.mockResolvedValue(new Map([[42, proof]]));
if (locked)
mockRemoveJournaledPartial.mockImplementation(() => {
throw new Error('EACCES');
});
await expect(
getHandler('DOWNLOADS_REMOVE')(null, 42)
).resolves.toMatchObject({ success: !locked });
expect(mockRemoveJournaledPartial).toHaveBeenCalledWith(
'/downloads/resume.mp4',
proof,
expect.any(Function),
true
);
if (locked) expect(deleteWhere).not.toHaveBeenCalled();
else expect(deleteWhere).toHaveBeenCalled();
}
);
it.each(['remove', 'clear'])(
'preserves a proven completed archive after failed status writes on %s',
async (action) => {
const proof = { version: 1, phase: 'finalization', size: 188 };
const id = action === 'remove' ? 42 : 1;
const row = {
...createDownloadRow(
action === 'remove' ? 'downloading' : 'failed'
),
contentType: 'catchup',
};
if (action === 'remove') mockDownloadRow(row);
else mockTerminalRows([row]);
mockArchiveProofs.mockResolvedValue(new Map([[id, proof]]));
mockVerifiedArchiveSize.mockReturnValue(188);
const result =
action === 'remove'
? getHandler('DOWNLOADS_REMOVE')(null, id)
: getHandler('DOWNLOADS_CLEAR_COMPLETED')(null);
await expect(result).resolves.toEqual({ success: true });
expect(mockVerifiedArchiveSize).toHaveBeenCalledWith(
row.filePath,
proof
);
expect(mockRemoveJournaledPartial).toHaveBeenCalledWith(
row.filePath,
proof,
expect.any(Function),
false
);
}
);
it('keeps a committing archive row and journal intact on Remove', async () => {
const { deleteWhere } = mockDownloadRow(
createDownloadRow('downloading')
);
mockIsDownloadCommitting.mockReturnValue(true);
await expect(getHandler('DOWNLOADS_REMOVE')(null, 42)).resolves.toEqual(
{
success: false,
error: 'Download is completing; try again shortly',
}
);
expect(mockRemovePartialDownloadFile).not.toHaveBeenCalled();
expect(mockRemoveDownloadFromRuntime).not.toHaveBeenCalled();
expect(deleteWhere).not.toHaveBeenCalled();
});
it('skips completion commits when clearing terminal rows', async () => {
const { deleteWhere } = mockTerminalRows([
createDownloadRow('completed'),
]);
mockHasRuntimeDownload.mockReturnValue(true);
await expect(
getHandler('DOWNLOADS_CLEAR_COMPLETED')(null)
).resolves.toEqual({ success: true });
expect(mockRemovePartialDownloadFile).not.toHaveBeenCalled();
expect(deleteWhere).not.toHaveBeenCalled();
});
it('routes archive Remove through durable cleanup instead of pathname unlink', async () => {
const { deleteWhere } = mockDownloadRow({
...createDownloadRow('failed'),
contentType: 'catchup',
});
const proof = { version: 1, phase: 'transfer' };
mockRemoveJournaledPartial.mockImplementation((_path, _proof, record) =>
record('/downloads/.iptvnator-cleanup-test/entry', 'partial')
);
mockArchiveProofs.mockResolvedValue(new Map([[42, proof]]));
await expect(getHandler('DOWNLOADS_REMOVE')(null, 42)).resolves.toEqual(
{ success: true }
);
expect(mockRemoveJournaledPartial).toHaveBeenCalledWith(
'/downloads/resume.mp4',
proof,
expect.any(Function),
true
);
expect(mockRemovePartialDownloadFile).not.toHaveBeenCalled();
expect(mockRecordArchiveCleanupPath).toHaveBeenCalledWith(
expect.anything(),
42,
proof,
'/downloads/.iptvnator-cleanup-test/entry',
'partial'
);
expect(deleteWhere).toHaveBeenCalled();
});
it('routes archive Clear completed through durable cleanup', async () => {
mockTerminalRows([
{ ...createDownloadRow('canceled'), contentType: 'catchup' },
]);
const proof = { version: 1, phase: 'transfer' };
mockRemoveJournaledPartial.mockImplementation((_path, _proof, record) =>
record('/downloads/.iptvnator-cleanup-test/entry', 'partial')
);
mockArchiveProofs.mockResolvedValue(new Map([[1, proof]]));
await expect(
getHandler('DOWNLOADS_CLEAR_COMPLETED')(null)
).resolves.toEqual({ success: true });
expect(mockRemoveJournaledPartial).toHaveBeenCalledWith(
'/downloads/resume.mp4',
proof,
expect.any(Function),
true
);
expect(mockRecordArchiveCleanupPath).toHaveBeenCalledWith(
expect.anything(),
1,
proof,
'/downloads/.iptvnator-cleanup-test/entry',
'partial'
);
expect(mockRemovePartialDownloadFile).not.toHaveBeenCalled();
});
it('removes queued resumed partial files before deleting the row', async () => {
const { deleteWhere } = mockDownloadRow(createDownloadRow('queued'));
@@ -1,5 +1,9 @@
import {
clearCompletedDownloadsRequest,
removeDownloadRequest,
} from './download-removal-requests';
import type { DownloadMetadataSnapshot } from '@iptvnator/shared/interfaces';
import { and, eq, inArray } from 'drizzle-orm';
import { eq } from 'drizzle-orm';
import { app, dialog, ipcMain, shell } from 'electron';
import { mkdir, readFile, rename, writeFile } from 'node:fs/promises';
import { join } from 'node:path';
@@ -10,7 +14,6 @@ import {
decorateDownloadItemAsync,
isAvailableDownloadFile,
} from './download-file-availability';
import { removePartialDownloadFile } from './download-file-path';
import { updateDownloadMetadataRequest } from './download-metadata-update';
import {
resumeDownloadRequest,
@@ -24,18 +27,9 @@ import {
broadcastDownloadUpdate,
cancelDownload,
pauseDownload,
removeDownloadFromRuntime,
setMainWindow,
} from './download-runtime';
const removablePartialStatuses = new Set([
'queued',
'paused',
'completed',
'failed',
'canceled',
]);
function getDownloadAuthorizationPath(): string {
return join(
app.getPath('userData'),
@@ -176,49 +170,9 @@ ipcMain.handle(
}
);
ipcMain.handle('DOWNLOADS_REMOVE', async (_event, downloadId: number) => {
try {
console.log('[Downloads] Remove download:', downloadId);
const db = await getDatabase();
const rows = await db
.select({
filePath: schema.downloads.filePath,
status: schema.downloads.status,
})
.from(schema.downloads)
.where(eq(schema.downloads.id, downloadId))
.limit(1);
const row = rows[0];
if (row?.filePath && removablePartialStatuses.has(row.status)) {
try {
removePartialDownloadFile(row.filePath);
} catch (cleanupError) {
// Keep the row (and its runtime entry) so the .part is never
// orphaned, but answer with a structured failure the UI can
// surface instead of an opaque IPC rejection. Retrying the
// remove re-attempts the deletion.
console.error(
'[Downloads] Failed to delete partial file on remove:',
row.filePath,
cleanupError
);
return {
error: 'Could not delete the partial file',
success: false,
};
}
}
removeDownloadFromRuntime(downloadId);
await db
.delete(schema.downloads)
.where(eq(schema.downloads.id, downloadId));
broadcastDownloadUpdate();
return { success: true };
} catch (error) {
console.error('[Downloads] Error removing download:', error);
throw error;
}
});
ipcMain.handle('DOWNLOADS_REMOVE', (_event, downloadId: number) =>
removeDownloadRequest(downloadId)
);
ipcMain.handle('DOWNLOADS_GET_LIST', async (_event, playlistId?: string) => {
try {
@@ -305,62 +259,8 @@ ipcMain.handle('DOWNLOADS_PLAY_FILE', async (_event, filePath: string) => {
: { error: 'File not found', success: false };
});
ipcMain.handle(
'DOWNLOADS_CLEAR_COMPLETED',
async (_event, playlistId?: string) => {
try {
const db = await getDatabase();
const terminalStatus = inArray(schema.downloads.status, [
'completed',
'failed',
'canceled',
]);
const terminalFilter = playlistId
? and(
eq(schema.downloads.playlistId, playlistId),
terminalStatus
)
: terminalStatus;
const rows = await db
.select({
id: schema.downloads.id,
filePath: schema.downloads.filePath,
status: schema.downloads.status,
})
.from(schema.downloads)
.where(terminalFilter);
const downloadIdsToDelete: number[] = [];
for (const row of rows) {
if (row.filePath && removablePartialStatuses.has(row.status)) {
try {
removePartialDownloadFile(row.filePath);
} catch (error) {
console.error(
'[Downloads] Retaining download after partial cleanup failed:',
error
);
continue;
}
}
downloadIdsToDelete.push(row.id);
}
if (downloadIdsToDelete.length > 0) {
await db
.delete(schema.downloads)
.where(
and(
terminalFilter,
inArray(schema.downloads.id, downloadIdsToDelete)
)
);
broadcastDownloadUpdate();
}
return { success: true };
} catch (error) {
console.error('[Downloads] Error clearing completed:', error);
throw error;
}
}
ipcMain.handle('DOWNLOADS_CLEAR_COMPLETED', (_event, playlistId?: string) =>
clearCompletedDownloadsRequest(playlistId)
);
export { resetStaleDownloads, setMainWindow };
@@ -11,6 +11,13 @@ type IpcHandler = (_event: unknown, ...args: unknown[]) => Promise<unknown>;
export const mockRegisteredHandlers = new Map<string, IpcHandler>();
export const mockGetDatabase = jest.fn();
export const mockRemoveDownloadFromRuntime = jest.fn();
export const mockIsDownloadCommitting = jest.fn();
export const mockHasRuntimeDownload = jest.fn();
export const mockPrepareArchiveRemoval = jest.fn();
export const mockArchiveProofs = jest.fn();
export const mockVerifiedArchiveSize = jest.fn();
export const mockRecordArchiveCleanupPath = jest.fn();
export const mockRemoveJournaledPartial = jest.fn();
export const mockBroadcastDownloadUpdate = jest.fn();
export const mockRemovePartialDownloadFile = jest.fn();
export const mockPauseDownload = jest.fn();
@@ -53,6 +60,13 @@ export async function setupDownloadsEventsHarness(): Promise<void> {
mockRegisteredHandlers.clear();
mockGetDatabase.mockReset();
mockRemoveDownloadFromRuntime.mockReset();
mockIsDownloadCommitting.mockReset().mockReturnValue(false);
mockHasRuntimeDownload.mockReset().mockReturnValue(false);
mockPrepareArchiveRemoval.mockReset().mockResolvedValue(true);
mockVerifiedArchiveSize.mockReset().mockReturnValue(null);
mockArchiveProofs.mockReset().mockResolvedValue(new Map());
mockRecordArchiveCleanupPath.mockReset();
mockRemoveJournaledPartial.mockReset();
mockBroadcastDownloadUpdate.mockReset();
mockRemovePartialDownloadFile.mockReset();
mockPauseDownload.mockReset();
@@ -111,9 +125,20 @@ export async function setupDownloadsEventsHarness(): Promise<void> {
jest.doMock('./download-redownload', () => ({
redownloadMissingRequest: mockRedownloadMissingRequest,
}));
jest.doMock('./download-catchup-journal', () => ({
readArchiveFinalizations: mockArchiveProofs,
verifiedArchiveSize: mockVerifiedArchiveSize,
recordArchiveCleanupPath: mockRecordArchiveCleanupPath,
}));
jest.doMock('./download-catchup-removal', () => ({
removeJournaledCatchupPartial: mockRemoveJournaledPartial,
}));
jest.doMock('./download-runtime', () => ({
broadcastDownloadUpdate: mockBroadcastDownloadUpdate,
cancelDownload: jest.fn(),
isDownloadCommitting: mockIsDownloadCommitting,
hasRuntimeDownload: mockHasRuntimeDownload,
prepareArchiveRemoval: mockPrepareArchiveRemoval,
pauseDownload: mockPauseDownload,
removeDownloadFromRuntime: mockRemoveDownloadFromRuntime,
setMainWindow: jest.fn(),
@@ -168,6 +193,7 @@ export function expectManagedPathLookup(
}
export function mockDownloadRow(row: {
contentType?: string;
filePath: string | null;
status: string;
}) {
@@ -187,7 +213,11 @@ export function mockDownloadRow(row: {
}
export function mockTerminalRows(
rows: Array<{ filePath: string | null; status: string }>
rows: Array<{
filePath: string | null;
status: string;
contentType?: string;
}>
) {
const deleteWhere = jest.fn().mockResolvedValue(undefined);
const selectWhere = jest
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "نسخ رابط الأرشيف",
"ARCHIVE_URL_COPIED": "تم نسخ رابط الأرشيف",
"ARCHIVE_URL_FAILED": "تعذّر نسخ رابط الأرشيف",
"ARCHIVE_URL_NOTICE": "قد يحتوي الرابط على بيانات الدخول. قد تحتاج المشغلات الخارجية أيضًا إلى ترويسات HTTP الخاصة بالمصدر."
"ARCHIVE_URL_NOTICE": "قد يحتوي الرابط على بيانات الدخول. قد تحتاج المشغلات الخارجية أيضًا إلى ترويسات HTTP الخاصة بالمصدر.",
"DOWNLOAD_ARCHIVE": "تنزيل البرنامج (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "أُضيف البرنامج إلى التنزيلات",
"ARCHIVE_DOWNLOAD_FAILED": "تعذر تنزيل البرنامج. قد لا يكون متاحًا بعد الآن.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "تنزيلات الأرشيف تدعم حاليًا تدفقات TS فقط.",
"ARCHIVE_DOWNLOAD_EXISTS": "هذا البرنامج موجود بالفعل في التنزيلات."
},
"FETCH_SUCCESS": "تم جلب الدليل بنجاح.",
"IMPORT_PROGRESS": "استيراد EPG",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "معلومات",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "يلزم استعادة الملف",
"MESSAGE": "تم الاحتفاظ بملف غير مرتبط بهذا التنزيل في: {{path}}. استعد المحتوى الذي تحتاجه، ثم احذف نسخة الاستعادة هذه وأعد المحاولة.",
"COPY_PATH": "نسخ مسار الاستعادة"
},
"TITLE": "التنزيلات",
"DOWNLOAD": "تنزيل",
"PLAY_LOCAL": "تشغيل محلي",
@@ -1395,9 +1405,12 @@
"RETRY": "إعادة محاولة {{title}}",
"MORE_ACTIONS": "إجراءات أخرى لـ {{title}}",
"COPY_URL": "نسخ رابط تنزيل {{title}}",
"DOWNLOAD_AGAIN": "تنزيل {{title}} مرة أخرى"
"DOWNLOAD_AGAIN": "تنزيل {{title}} مرة أخرى",
"RESTART_ARCHIVE": "تنزيل {{title}} من البداية"
},
"DOWNLOAD_AGAIN": "تنزيل مرة أخرى"
"DOWNLOAD_AGAIN": "تنزيل مرة أخرى",
"CATCHUP": "برنامج تلفزيوني",
"ARCHIVE_RESTART_NOTE": "بعد الانقطاع، يبدأ تنزيل الأرشيف من البداية."
},
"REMOTE_CONTROL": {
"TITLE": "التحكم عن بعد",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "نسخ رابط الأرشيف",
"ARCHIVE_URL_COPIED": "تنسخ رابط الأرشيف",
"ARCHIVE_URL_FAILED": "ما قدرناش ننسخو رابط الأرشيف",
"ARCHIVE_URL_NOTICE": "الرابط يقدر يكون فيه معلومات الدخول. المشغلات الخارجية تقدر تحتاج حتى ترويسات HTTP ديال المصدر."
"ARCHIVE_URL_NOTICE": "الرابط يقدر يكون فيه معلومات الدخول. المشغلات الخارجية تقدر تحتاج حتى ترويسات HTTP ديال المصدر.",
"DOWNLOAD_ARCHIVE": "حمّل البرنامج (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "تزاد البرنامج للتنزيلات",
"ARCHIVE_DOWNLOAD_FAILED": "ما قدرناش نحمّلو البرنامج. يمكن ما بقاش متوفر.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "تنزيلات الأرشيف كتدعم دابا غير تدفقات TS.",
"ARCHIVE_DOWNLOAD_EXISTS": "هاد البرنامج راه كاين فالتنزيلات."
},
"FETCH_SUCCESS": "EPG تجلب بنجاح",
"IMPORT_PROGRESS": "استيراد EPG",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "معلومات",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "خاص استرجاع الملف",
"MESSAGE": "تحفظ واحد الملف ما عندوش علاقة بهاد التحميل فـ: {{path}}. رجّع المحتوى اللي محتاج، مسح هاد النسخة الاحتياطية، وعاود جرّب.",
"COPY_PATH": "نسخ مسار الاسترجاع"
},
"TITLE": "التحميلات",
"DOWNLOAD": "تحميل",
"PLAY_LOCAL": "شغل محلياً",
@@ -1395,9 +1405,12 @@
"RETRY": "عاود جرّب {{title}}",
"MORE_ACTIONS": "إجراءات أخرى لـ {{title}}",
"COPY_URL": "نسخ رابط تحميل {{title}}",
"DOWNLOAD_AGAIN": "عاود حمّل {{title}}"
"DOWNLOAD_AGAIN": "عاود حمّل {{title}}",
"RESTART_ARCHIVE": "حمّل {{title}} من الأول"
},
"DOWNLOAD_AGAIN": "عاود حمّل"
"DOWNLOAD_AGAIN": "عاود حمّل",
"CATCHUP": "برنامج تلفزيوني",
"ARCHIVE_RESTART_NOTE": "من بعد الانقطاع، تنزيل الأرشيف كيبدا من الأول."
},
"REMOTE_CONTROL": {
"TITLE": "التحكم عن بعد",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "Капіяваць спасылку на архіў",
"ARCHIVE_URL_COPIED": "Спасылка на архіў скапіявана",
"ARCHIVE_URL_FAILED": "Не ўдалося скапіяваць спасылку на архіў",
"ARCHIVE_URL_NOTICE": "Спасылка можа змяшчаць даныя ўліковага запісу. Знешняму прайгравальніку таксама могуць спатрэбіцца HTTP-загалоўкі крыніцы."
"ARCHIVE_URL_NOTICE": "Спасылка можа змяшчаць даныя ўліковага запісу. Знешняму прайгравальніку таксама могуць спатрэбіцца HTTP-загалоўкі крыніцы.",
"DOWNLOAD_ARCHIVE": "Спампаваць перадачу (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "Перадача дададзена ў спампоўкі",
"ARCHIVE_DOWNLOAD_FAILED": "Не атрымалася спампаваць перадачу. Магчыма, яна ўжо недаступная.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "Спампоўка архіва пакуль падтрымлівае толькі TS-патокі.",
"ARCHIVE_DOWNLOAD_EXISTS": "Гэтая перадача ўжо ёсць у спампоўках."
},
"FETCH_SUCCESS": "EPG быў паспяхова загружаны",
"IMPORT_PROGRESS": "Імпарт EPG",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "Інфармацыя",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "Патрабуецца аднаўленне файла",
"MESSAGE": "Файл, не звязаны з гэтай загрузкай, захаваны па шляху: {{path}}. Захавайце патрэбнае змесціва, выдаліце гэтую рэзервовую копію і паўтарыце дзеянне.",
"COPY_PATH": "Капіяваць шлях аднаўлення"
},
"TITLE": "Спампаванні",
"DOWNLOAD": "Спампаваць",
"PLAY_LOCAL": "Прайграць лакальна",
@@ -1395,9 +1405,12 @@
"RETRY": "Паўтарыць {{title}}",
"MORE_ACTIONS": "Дадатковыя дзеянні для {{title}}",
"COPY_URL": "Скапіяваць URL спампоўкі для {{title}}",
"DOWNLOAD_AGAIN": "Спампаваць {{title}} яшчэ раз"
"DOWNLOAD_AGAIN": "Спампаваць {{title}} яшчэ раз",
"RESTART_ARCHIVE": "Спампаваць {{title}} нанова"
},
"DOWNLOAD_AGAIN": "Спампаваць яшчэ раз"
"DOWNLOAD_AGAIN": "Спампаваць яшчэ раз",
"CATCHUP": "Тэлеперадача",
"ARCHIVE_RESTART_NOTE": "Пасля перапынення спампоўка перадачы пачынаецца нанова."
},
"REMOTE_CONTROL": {
"TITLE": "Аддаленае кіраванне",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "Archiv-URL kopieren",
"ARCHIVE_URL_COPIED": "Archiv-URL kopiert",
"ARCHIVE_URL_FAILED": "Archiv-URL konnte nicht kopiert werden",
"ARCHIVE_URL_NOTICE": "Die URL kann Zugangsdaten enthalten. Externe Player benötigen möglicherweise auch die HTTP-Header der Quelle."
"ARCHIVE_URL_NOTICE": "Die URL kann Zugangsdaten enthalten. Externe Player benötigen möglicherweise auch die HTTP-Header der Quelle.",
"DOWNLOAD_ARCHIVE": "Sendung herunterladen (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "Sendung zu Downloads hinzugefügt",
"ARCHIVE_DOWNLOAD_FAILED": "Die Sendung konnte nicht heruntergeladen werden. Sie ist möglicherweise nicht mehr verfügbar.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "Archiv-Downloads unterstützen derzeit nur TS-Streams.",
"ARCHIVE_DOWNLOAD_EXISTS": "Diese Sendung ist bereits in Downloads."
},
"FETCH_SUCCESS": "EPG wurde erfolgreich abgerufen",
"IMPORT_PROGRESS": "EPG-Import",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "Information",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "Dateiwiederherstellung erforderlich",
"MESSAGE": "Eine fremde Datei wurde hier gesichert: {{path}}. Sichern Sie benötigte Inhalte, löschen Sie diese Wiederherstellungskopie und versuchen Sie es erneut.",
"COPY_PATH": "Wiederherstellungspfad kopieren"
},
"TITLE": "Downloads",
"DOWNLOAD": "Herunterladen",
"PLAY_LOCAL": "Lokal abspielen",
@@ -1395,9 +1405,12 @@
"RETRY": "{{title}} erneut versuchen",
"MORE_ACTIONS": "Weitere Aktionen für {{title}}",
"COPY_URL": "Download-URL für {{title}} kopieren",
"DOWNLOAD_AGAIN": "{{title}} erneut herunterladen"
"DOWNLOAD_AGAIN": "{{title}} erneut herunterladen",
"RESTART_ARCHIVE": "{{title}} von vorn herunterladen"
},
"DOWNLOAD_AGAIN": "Erneut herunterladen"
"DOWNLOAD_AGAIN": "Erneut herunterladen",
"CATCHUP": "TV-Sendung",
"ARCHIVE_RESTART_NOTE": "Nach einer Unterbrechung beginnt der Archiv-Download von vorn."
},
"REMOTE_CONTROL": {
"TITLE": "Fernbedienung",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "Αντιγραφή URL αρχείου",
"ARCHIVE_URL_COPIED": "Το URL αρχείου αντιγράφηκε",
"ARCHIVE_URL_FAILED": "Αδυναμία αντιγραφής του URL αρχείου",
"ARCHIVE_URL_NOTICE": "Το URL μπορεί να περιέχει στοιχεία σύνδεσης. Τα εξωτερικά προγράμματα αναπαραγωγής μπορεί να χρειάζονται και τις κεφαλίδες HTTP της πηγής."
"ARCHIVE_URL_NOTICE": "Το URL μπορεί να περιέχει στοιχεία σύνδεσης. Τα εξωτερικά προγράμματα αναπαραγωγής μπορεί να χρειάζονται και τις κεφαλίδες HTTP της πηγής.",
"DOWNLOAD_ARCHIVE": "Λήψη εκπομπής (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "Η εκπομπή προστέθηκε στις λήψεις",
"ARCHIVE_DOWNLOAD_FAILED": "Η λήψη απέτυχε. Η εκπομπή ίσως δεν είναι πλέον διαθέσιμη.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "Οι λήψεις από το αρχείο υποστηρίζουν προς το παρόν μόνο ροές TS.",
"ARCHIVE_DOWNLOAD_EXISTS": "Αυτή η εκπομπή υπάρχει ήδη στις λήψεις."
},
"FETCH_SUCCESS": "Το EPG ανακτήθηκε με επιτυχία",
"IMPORT_PROGRESS": "Εισαγωγή Ηλεκτρονικού Οδηγού Προγράμματος (EPG)",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "Πληροφορίες",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "Απαιτείται ανάκτηση αρχείου",
"MESSAGE": "Ένα αρχείο άσχετο με αυτή τη λήψη διατηρήθηκε στη διαδρομή: {{path}}. Ανακτήστε ό,τι χρειάζεστε, διαγράψτε αυτό το αντίγραφο ανάκτησης και δοκιμάστε ξανά.",
"COPY_PATH": "Αντιγραφή διαδρομής ανάκτησης"
},
"TITLE": "Λήψεις",
"DOWNLOAD": "Λήψη",
"PLAY_LOCAL": "Τοπική αναπαραγωγή",
@@ -1395,9 +1405,12 @@
"RETRY": "Επανάληψη {{title}}",
"MORE_ACTIONS": "Περισσότερες ενέργειες για {{title}}",
"COPY_URL": "Αντιγραφή URL λήψης για {{title}}",
"DOWNLOAD_AGAIN": "Λήψη του {{title}} ξανά"
"DOWNLOAD_AGAIN": "Λήψη του {{title}} ξανά",
"RESTART_ARCHIVE": "Λήψη του {{title}} από την αρχή"
},
"DOWNLOAD_AGAIN": "Λήψη ξανά"
"DOWNLOAD_AGAIN": "Λήψη ξανά",
"CATCHUP": "Τηλεοπτική εκπομπή",
"ARCHIVE_RESTART_NOTE": "Μετά από διακοπή, η λήψη από το αρχείο ξεκινά από την αρχή."
},
"REMOTE_CONTROL": {
"TITLE": "Τηλεχειριστήριο",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "Copy archive URL",
"ARCHIVE_URL_COPIED": "Archive URL copied",
"ARCHIVE_URL_FAILED": "Could not copy the archive URL",
"ARCHIVE_URL_NOTICE": "The URL may contain account credentials. External players may also need your source’s HTTP headers."
"ARCHIVE_URL_NOTICE": "The URL may contain account credentials. External players may also need your source’s HTTP headers.",
"DOWNLOAD_ARCHIVE": "Download programme (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "Programme added to Downloads",
"ARCHIVE_DOWNLOAD_FAILED": "Could not download this programme. It may no longer be available.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "Archive downloads currently support TS streams only.",
"ARCHIVE_DOWNLOAD_EXISTS": "This programme is already in Downloads."
},
"FETCH_SUCCESS": "EPG was fetched successfully",
"IMPORT_PROGRESS": "EPG Import",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "Information",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "File recovery needed",
"MESSAGE": "A file unrelated to this download was preserved at: {{path}}. Recover any content you need, delete this recovery copy, then try again.",
"COPY_PATH": "Copy recovery path"
},
"TITLE": "Downloads",
"DOWNLOAD": "Download",
"PLAY_LOCAL": "Play Local",
@@ -1395,9 +1405,12 @@
"RETRY": "Retry {{title}}",
"MORE_ACTIONS": "More actions for {{title}}",
"COPY_URL": "Copy download URL for {{title}}",
"DOWNLOAD_AGAIN": "Download {{title}} again"
"DOWNLOAD_AGAIN": "Download {{title}} again",
"RESTART_ARCHIVE": "Restart {{title}} from the beginning"
},
"DOWNLOAD_AGAIN": "Download again"
"DOWNLOAD_AGAIN": "Download again",
"CATCHUP": "TV programme",
"ARCHIVE_RESTART_NOTE": "Archive downloads restart from the beginning after an interruption."
},
"REMOTE_CONTROL": {
"TITLE": "Remote control",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "Copiar URL del archivo",
"ARCHIVE_URL_COPIED": "URL del archivo copiada",
"ARCHIVE_URL_FAILED": "No se pudo copiar la URL del archivo",
"ARCHIVE_URL_NOTICE": "La URL puede contener credenciales. Los reproductores externos también pueden necesitar las cabeceras HTTP de la fuente."
"ARCHIVE_URL_NOTICE": "La URL puede contener credenciales. Los reproductores externos también pueden necesitar las cabeceras HTTP de la fuente.",
"DOWNLOAD_ARCHIVE": "Descargar programa (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "Programa añadido a Descargas",
"ARCHIVE_DOWNLOAD_FAILED": "No se pudo descargar el programa. Puede que ya no esté disponible.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "Las descargas del archivo solo admiten flujos TS por ahora.",
"ARCHIVE_DOWNLOAD_EXISTS": "Este programa ya está en Descargas."
},
"FETCH_SUCCESS": "EPG obtenida correctamente",
"IMPORT_PROGRESS": "Importación de EPG",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "Información",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "Se requiere recuperar un archivo",
"MESSAGE": "Se ha conservado un archivo ajeno a esta descarga en: {{path}}. Recupera el contenido que necesites, elimina esta copia de recuperación y vuelve a intentarlo.",
"COPY_PATH": "Copiar ruta de recuperación"
},
"TITLE": "Descargas",
"DOWNLOAD": "Descargar",
"PLAY_LOCAL": "Reproducir localmente",
@@ -1395,9 +1405,12 @@
"RETRY": "Reintentar {{title}}",
"MORE_ACTIONS": "Más acciones para {{title}}",
"COPY_URL": "Copiar la URL de descarga de {{title}}",
"DOWNLOAD_AGAIN": "Volver a descargar {{title}}"
"DOWNLOAD_AGAIN": "Volver a descargar {{title}}",
"RESTART_ARCHIVE": "Descargar {{title}} desde el principio"
},
"DOWNLOAD_AGAIN": "Volver a descargar"
"DOWNLOAD_AGAIN": "Volver a descargar",
"CATCHUP": "Programa de TV",
"ARCHIVE_RESTART_NOTE": "Tras una interrupción, la descarga del archivo empieza desde el principio."
},
"REMOTE_CONTROL": {
"TITLE": "Control remoto",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "Copier le lien de rattrapage",
"ARCHIVE_URL_COPIED": "Lien de rattrapage copié",
"ARCHIVE_URL_FAILED": "Impossible de copier le lien de rattrapage",
"ARCHIVE_URL_NOTICE": "Le lien peut contenir des identifiants. Les lecteurs externes peuvent aussi nécessiter les en-têtes HTTP de la source."
"ARCHIVE_URL_NOTICE": "Le lien peut contenir des identifiants. Les lecteurs externes peuvent aussi nécessiter les en-têtes HTTP de la source.",
"DOWNLOAD_ARCHIVE": "Télécharger l’émission (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "Émission ajoutée aux téléchargements",
"ARCHIVE_DOWNLOAD_FAILED": "Impossible de télécharger cette émission. Elle n’est peut-être plus disponible.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "Seuls les flux TS sont pris en charge pour le téléchargement des archives.",
"ARCHIVE_DOWNLOAD_EXISTS": "Cette émission figure déjà dans les téléchargements."
},
"FETCH_SUCCESS": "L'EPG a été récupéré avec succès",
"IMPORT_PROGRESS": "Importation EPG",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "Information",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "Récupération du fichier nécessaire",
"MESSAGE": "Un fichier sans rapport avec ce téléchargement a été conservé ici : {{path}}. Récupérez le contenu utile, supprimez cette copie de récupération, puis réessayez.",
"COPY_PATH": "Copier le chemin de récupération"
},
"TITLE": "Téléchargements",
"DOWNLOAD": "Télécharger",
"PLAY_LOCAL": "Lire localement",
@@ -1395,9 +1405,12 @@
"RETRY": "Réessayer {{title}}",
"MORE_ACTIONS": "Plus d’actions pour {{title}}",
"COPY_URL": "Copier l’URL de téléchargement de {{title}}",
"DOWNLOAD_AGAIN": "Télécharger à nouveau {{title}}"
"DOWNLOAD_AGAIN": "Télécharger à nouveau {{title}}",
"RESTART_ARCHIVE": "Retélécharger {{title}} depuis le début"
},
"DOWNLOAD_AGAIN": "Télécharger à nouveau"
"DOWNLOAD_AGAIN": "Télécharger à nouveau",
"CATCHUP": "Émission TV",
"ARCHIVE_RESTART_NOTE": "Après une interruption, le téléchargement de l’archive reprend depuis le début."
},
"REMOTE_CONTROL": {
"TITLE": "Télécommande",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "Archív URL másolása",
"ARCHIVE_URL_COPIED": "Archív URL másolva",
"ARCHIVE_URL_FAILED": "Az archív URL másolása sikertelen",
"ARCHIVE_URL_NOTICE": "Az URL bejelentkezési adatokat tartalmazhat. A külső lejátszóknak a forrás HTTP-fejléceire is szükségük lehet."
"ARCHIVE_URL_NOTICE": "Az URL bejelentkezési adatokat tartalmazhat. A külső lejátszóknak a forrás HTTP-fejléceire is szükségük lehet.",
"DOWNLOAD_ARCHIVE": "Műsor letöltése (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "Műsor hozzáadva a letöltésekhez",
"ARCHIVE_DOWNLOAD_FAILED": "A műsor nem tölthető le. Lehet, hogy már nem érhető el.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "Az archívum letöltése jelenleg csak TS adatfolyamokkal működik.",
"ARCHIVE_DOWNLOAD_EXISTS": "Ez a műsor már szerepel a letöltések között."
},
"FETCH_SUCCESS": "Az EPG lekérése sikerült",
"IMPORT_PROGRESS": "EPG importálása",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "Információk",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "Fájlhelyreállítás szükséges",
"MESSAGE": "A letöltéshez nem tartozó fájlt megőriztük itt: {{path}}. Mentse a szükséges tartalmat, törölje ezt a helyreállítási másolatot, majd próbálja újra.",
"COPY_PATH": "Helyreállítási útvonal másolása"
},
"TITLE": "Letöltések",
"DOWNLOAD": "Letöltés",
"PLAY_LOCAL": "Helyi fájl lejátszása",
@@ -1395,9 +1405,12 @@
"RETRY": "{{title}} újrapróbálása",
"MORE_ACTIONS": "További műveletek: {{title}}",
"COPY_URL": "{{title}} letöltési URL-jének másolása",
"DOWNLOAD_AGAIN": "{{title}} letöltése újra"
"DOWNLOAD_AGAIN": "{{title}} letöltése újra",
"RESTART_ARCHIVE": "{{title}} letöltése elölről"
},
"DOWNLOAD_AGAIN": "Letöltés újra"
"DOWNLOAD_AGAIN": "Letöltés újra",
"CATCHUP": "TV-műsor",
"ARCHIVE_RESTART_NOTE": "Megszakítás után az archív műsor letöltése elölről indul."
},
"REMOTE_CONTROL": {
"TITLE": "Távvezérlés",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "Copia URL archivio",
"ARCHIVE_URL_COPIED": "URL archivio copiato",
"ARCHIVE_URL_FAILED": "Impossibile copiare URL archivio",
"ARCHIVE_URL_NOTICE": "L’URL può contenere credenziali. I lettori esterni potrebbero richiedere anche le intestazioni HTTP della sorgente."
"ARCHIVE_URL_NOTICE": "L’URL può contenere credenziali. I lettori esterni potrebbero richiedere anche le intestazioni HTTP della sorgente.",
"DOWNLOAD_ARCHIVE": "Scarica programma (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "Programma aggiunto ai download",
"ARCHIVE_DOWNLOAD_FAILED": "Impossibile scaricare il programma. Potrebbe non essere più disponibile.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "I download dall’archivio supportano solo flussi TS.",
"ARCHIVE_DOWNLOAD_EXISTS": "Questo programma è già nei download."
},
"FETCH_SUCCESS": "EPG scaricato correttamente",
"IMPORT_PROGRESS": "Importazione EPG",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "Informazioni",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "Recupero del file necessario",
"MESSAGE": "Un file estraneo a questo download è stato conservato in: {{path}}. Recupera i contenuti necessari, elimina questa copia di recupero e riprova.",
"COPY_PATH": "Copia percorso di recupero"
},
"TITLE": "Download",
"DOWNLOAD": "Scarica",
"PLAY_LOCAL": "Riproduci localmente",
@@ -1395,9 +1405,12 @@
"RETRY": "Riprova {{title}}",
"MORE_ACTIONS": "Altre azioni per {{title}}",
"COPY_URL": "Copia URL di download per {{title}}",
"DOWNLOAD_AGAIN": "Scarica di nuovo {{title}}"
"DOWNLOAD_AGAIN": "Scarica di nuovo {{title}}",
"RESTART_ARCHIVE": "Scarica {{title}} dall’inizio"
},
"DOWNLOAD_AGAIN": "Scarica di nuovo"
"DOWNLOAD_AGAIN": "Scarica di nuovo",
"CATCHUP": "Programma TV",
"ARCHIVE_RESTART_NOTE": "Dopo un’interruzione, il download dall’archivio ricomincia dall’inizio."
},
"REMOTE_CONTROL": {
"TITLE": "Controllo remoto",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "アーカイブURLをコピー",
"ARCHIVE_URL_COPIED": "アーカイブURLをコピーしました",
"ARCHIVE_URL_FAILED": "アーカイブURLをコピーできませんでした",
"ARCHIVE_URL_NOTICE": "URLにはアカウントの認証情報が含まれる場合があります。外部プレーヤーではソースのHTTPヘッダーも必要になることがあります。"
"ARCHIVE_URL_NOTICE": "URLにはアカウントの認証情報が含まれる場合があります。外部プレーヤーではソースのHTTPヘッダーも必要になることがあります。",
"DOWNLOAD_ARCHIVE": "番組をダウンロード (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "番組をダウンロードに追加しました",
"ARCHIVE_DOWNLOAD_FAILED": "番組をダウンロードできませんでした。配信が終了している可能性があります。",
"ARCHIVE_DOWNLOAD_TS_ONLY": "アーカイブのダウンロードは現在TSストリームのみ対応しています。",
"ARCHIVE_DOWNLOAD_EXISTS": "この番組はすでにダウンロードにあります。"
},
"FETCH_SUCCESS": "EPGの取得に成功しました",
"IMPORT_PROGRESS": "EPGインポート",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "情報",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "ファイルの復元が必要です",
"MESSAGE": "このダウンロードとは無関係なファイルが {{path}} に保存されています。必要な内容を取り出し、この復元用コピーを削除してから、もう一度お試しください。",
"COPY_PATH": "復元パスをコピー"
},
"TITLE": "ダウンロード",
"DOWNLOAD": "ダウンロード",
"PLAY_LOCAL": "ローカル再生",
@@ -1395,9 +1405,12 @@
"RETRY": "{{title}} を再試行",
"MORE_ACTIONS": "{{title}} のその他の操作",
"COPY_URL": "{{title}} のダウンロード URL をコピー",
"DOWNLOAD_AGAIN": "{{title}} を再ダウンロード"
"DOWNLOAD_AGAIN": "{{title}} を再ダウンロード",
"RESTART_ARCHIVE": "{{title}}を最初からダウンロード"
},
"DOWNLOAD_AGAIN": "再ダウンロード"
"DOWNLOAD_AGAIN": "再ダウンロード",
"CATCHUP": "テレビ番組",
"ARCHIVE_RESTART_NOTE": "中断後はアーカイブのダウンロードを最初からやり直します。"
},
"REMOTE_CONTROL": {
"TITLE": "リモートコントロール",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "다시보기 URL 복사",
"ARCHIVE_URL_COPIED": "다시보기 URL이 복사되었습니다",
"ARCHIVE_URL_FAILED": "다시보기 URL을 복사하지 못했습니다",
"ARCHIVE_URL_NOTICE": "URL에 계정 인증 정보가 포함될 수 있습니다. 외부 플레이어에는 소스의 HTTP 헤더도 필요할 수 있습니다."
"ARCHIVE_URL_NOTICE": "URL에 계정 인증 정보가 포함될 수 있습니다. 외부 플레이어에는 소스의 HTTP 헤더도 필요할 수 있습니다.",
"DOWNLOAD_ARCHIVE": "프로그램 다운로드 (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "프로그램을 다운로드에 추가했습니다",
"ARCHIVE_DOWNLOAD_FAILED": "프로그램을 다운로드할 수 없습니다. 더 이상 제공되지 않을 수 있습니다.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "보관 프로그램 다운로드는 현재 TS 스트림만 지원합니다.",
"ARCHIVE_DOWNLOAD_EXISTS": "이 프로그램은 이미 다운로드 목록에 있습니다."
},
"FETCH_SUCCESS": "EPG를 성공적으로 가져왔습니다",
"IMPORT_PROGRESS": "EPG 가져오기",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "정보",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "파일 복구 필요",
"MESSAGE": "이 다운로드와 관련 없는 파일이 {{path}}에 보관되었습니다. 필요한 내용을 복구하고 이 복구용 사본을 삭제한 후 다시 시도하세요.",
"COPY_PATH": "복구 경로 복사"
},
"TITLE": "다운로드",
"DOWNLOAD": "다운로드",
"PLAY_LOCAL": "로컬 재생",
@@ -1395,9 +1405,12 @@
"RETRY": "{{title}} 다시 시도",
"MORE_ACTIONS": "{{title}} 추가 작업",
"COPY_URL": "{{title}} 다운로드 URL 복사",
"DOWNLOAD_AGAIN": "{{title}} 다시 다운로드"
"DOWNLOAD_AGAIN": "{{title}} 다시 다운로드",
"RESTART_ARCHIVE": "{{title}} 처음부터 다운로드"
},
"DOWNLOAD_AGAIN": "다시 다운로드"
"DOWNLOAD_AGAIN": "다시 다운로드",
"CATCHUP": "TV 프로그램",
"ARCHIVE_RESTART_NOTE": "중단 후에는 보관 프로그램 다운로드를 처음부터 다시 시작합니다."
},
"REMOTE_CONTROL": {
"TITLE": "원격 제어",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "Archief-URL kopiëren",
"ARCHIVE_URL_COPIED": "Archief-URL gekopieerd",
"ARCHIVE_URL_FAILED": "Kan archief-URL niet kopiëren",
"ARCHIVE_URL_NOTICE": "De URL kan inloggegevens bevatten. Externe spelers hebben mogelijk ook de HTTP-headers van de bron nodig."
"ARCHIVE_URL_NOTICE": "De URL kan inloggegevens bevatten. Externe spelers hebben mogelijk ook de HTTP-headers van de bron nodig.",
"DOWNLOAD_ARCHIVE": "Programma downloaden (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "Programma toegevoegd aan Downloads",
"ARCHIVE_DOWNLOAD_FAILED": "Het programma kon niet worden gedownload. Het is mogelijk niet meer beschikbaar.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "Archiefdownloads ondersteunen momenteel alleen TS-streams.",
"ARCHIVE_DOWNLOAD_EXISTS": "Dit programma staat al in Downloads."
},
"FETCH_SUCCESS": "TV-Gids bijgewerkt",
"IMPORT_PROGRESS": "EPG-import",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "Informatie",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "Bestandsherstel nodig",
"MESSAGE": "Een bestand dat niet bij deze download hoort, is bewaard op: {{path}}. Bewaar de benodigde inhoud, verwijder deze herstelkopie en probeer het opnieuw.",
"COPY_PATH": "Herstelpad kopiëren"
},
"TITLE": "Downloads",
"DOWNLOAD": "Downloaden",
"PLAY_LOCAL": "Lokaal bestand afspelen",
@@ -1395,9 +1405,12 @@
"RETRY": "{{title}} opnieuw proberen",
"MORE_ACTIONS": "Meer acties voor {{title}}",
"COPY_URL": "Download-URL voor {{title}} kopiëren",
"DOWNLOAD_AGAIN": "{{title}} opnieuw downloaden"
"DOWNLOAD_AGAIN": "{{title}} opnieuw downloaden",
"RESTART_ARCHIVE": "{{title}} vanaf het begin downloaden"
},
"DOWNLOAD_AGAIN": "Opnieuw downloaden"
"DOWNLOAD_AGAIN": "Opnieuw downloaden",
"CATCHUP": "TV-programma",
"ARCHIVE_RESTART_NOTE": "Na een onderbreking begint de archiefdownload opnieuw."
},
"REMOTE_CONTROL": {
"TITLE": "Afstandsbediening",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "Kopiuj adres archiwum",
"ARCHIVE_URL_COPIED": "Skopiowano adres archiwum",
"ARCHIVE_URL_FAILED": "Nie udało się skopiować adresu archiwum",
"ARCHIVE_URL_NOTICE": "Adres może zawierać dane logowania. Odtwarzacze zewnętrzne mogą też wymagać nagłówków HTTP źródła."
"ARCHIVE_URL_NOTICE": "Adres może zawierać dane logowania. Odtwarzacze zewnętrzne mogą też wymagać nagłówków HTTP źródła.",
"DOWNLOAD_ARCHIVE": "Pobierz program (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "Program dodany do pobierania",
"ARCHIVE_DOWNLOAD_FAILED": "Nie udało się pobrać programu. Może już być niedostępny.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "Pobieranie z archiwum obsługuje obecnie tylko strumienie TS.",
"ARCHIVE_DOWNLOAD_EXISTS": "Ten program jest już na liście pobierania."
},
"FETCH_SUCCESS": "EPG zostało pomyślnie pobrane",
"IMPORT_PROGRESS": "Import EPG",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "Informacje",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "Wymagane odzyskanie pliku",
"MESSAGE": "Plik niezwiązany z tym pobieraniem zachowano pod ścieżką: {{path}}. Odzyskaj potrzebną zawartość, usuń tę kopię odzyskiwania i spróbuj ponownie.",
"COPY_PATH": "Kopiuj ścieżkę odzyskiwania"
},
"TITLE": "Pobrane",
"DOWNLOAD": "Pobierz",
"PLAY_LOCAL": "Odtwórz plik lokalny",
@@ -1395,9 +1405,12 @@
"RETRY": "Ponów {{title}}",
"MORE_ACTIONS": "Więcej akcji dla {{title}}",
"COPY_URL": "Kopiuj URL pobierania dla {{title}}",
"DOWNLOAD_AGAIN": "Pobierz {{title}} ponownie"
"DOWNLOAD_AGAIN": "Pobierz {{title}} ponownie",
"RESTART_ARCHIVE": "Pobierz {{title}} od początku"
},
"DOWNLOAD_AGAIN": "Pobierz ponownie"
"DOWNLOAD_AGAIN": "Pobierz ponownie",
"CATCHUP": "Program TV",
"ARCHIVE_RESTART_NOTE": "Po przerwaniu pobieranie z archiwum zaczyna się od początku."
},
"REMOTE_CONTROL": {
"TITLE": "Zdalne sterowanie",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "Copiar URL do arquivo",
"ARCHIVE_URL_COPIED": "URL do arquivo copiada",
"ARCHIVE_URL_FAILED": "Não foi possível copiar a URL do arquivo",
"ARCHIVE_URL_NOTICE": "A URL pode conter credenciais. Os leitores externos também podem precisar dos cabeçalhos HTTP da fonte."
"ARCHIVE_URL_NOTICE": "A URL pode conter credenciais. Os leitores externos também podem precisar dos cabeçalhos HTTP da fonte.",
"DOWNLOAD_ARCHIVE": "Baixar programa (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "Programa adicionado aos downloads",
"ARCHIVE_DOWNLOAD_FAILED": "Não foi possível baixar o programa. Ele pode não estar mais disponível.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "Os downloads do arquivo aceitam apenas fluxos TS por enquanto.",
"ARCHIVE_DOWNLOAD_EXISTS": "Este programa já está nos downloads."
},
"FETCH_SUCCESS": "EPG foi buscado com sucesso",
"IMPORT_PROGRESS": "Importação de EPG",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "Informação",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "É necessário recuperar um ficheiro",
"MESSAGE": "Um ficheiro alheio a esta transferência foi preservado em: {{path}}. Recupere o conteúdo necessário, elimine esta cópia de recuperação e tente novamente.",
"COPY_PATH": "Copiar caminho de recuperação"
},
"TITLE": "Downloads",
"DOWNLOAD": "Baixar",
"PLAY_LOCAL": "Reproduzir local",
@@ -1395,9 +1405,12 @@
"RETRY": "Tentar {{title}} novamente",
"MORE_ACTIONS": "Mais ações para {{title}}",
"COPY_URL": "Copiar URL de download de {{title}}",
"DOWNLOAD_AGAIN": "Baixar {{title}} novamente"
"DOWNLOAD_AGAIN": "Baixar {{title}} novamente",
"RESTART_ARCHIVE": "Baixar {{title}} do início"
},
"DOWNLOAD_AGAIN": "Baixar novamente"
"DOWNLOAD_AGAIN": "Baixar novamente",
"CATCHUP": "Programa de TV",
"ARCHIVE_RESTART_NOTE": "Após uma interrupção, o download do arquivo começa do início."
},
"REMOTE_CONTROL": {
"TITLE": "Controle remoto",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "Копировать ссылку на архив",
"ARCHIVE_URL_COPIED": "Ссылка на архив скопирована",
"ARCHIVE_URL_FAILED": "Не удалось скопировать ссылку на архив",
"ARCHIVE_URL_NOTICE": "Ссылка может содержать данные аккаунта. Внешнему плееру также могут понадобиться HTTP-заголовки источника."
"ARCHIVE_URL_NOTICE": "Ссылка может содержать данные аккаунта. Внешнему плееру также могут понадобиться HTTP-заголовки источника.",
"DOWNLOAD_ARCHIVE": "Скачать передачу (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "Передача добавлена в загрузки",
"ARCHIVE_DOWNLOAD_FAILED": "Не удалось скачать передачу. Возможно, она уже недоступна в архиве.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "Скачивание архива пока поддерживает только TS-потоки.",
"ARCHIVE_DOWNLOAD_EXISTS": "Эта передача уже есть в загрузках."
},
"FETCH_SUCCESS": "Программа каналов (EPG) была успешно получена",
"IMPORT_PROGRESS": "Импорт EPG",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "Инфо",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "Требуется восстановление файла",
"MESSAGE": "Файл, не относящийся к этой загрузке, сохранён по пути: {{path}}. Сохраните нужное содержимое, удалите эту резервную копию и повторите действие.",
"COPY_PATH": "Копировать путь восстановления"
},
"TITLE": "Загрузки",
"DOWNLOAD": "Скачать",
"PLAY_LOCAL": "Воспроизвести локально",
@@ -1395,9 +1405,12 @@
"RETRY": "Повторить {{title}}",
"MORE_ACTIONS": "Дополнительные действия для {{title}}",
"COPY_URL": "Скопировать URL загрузки для {{title}}",
"DOWNLOAD_AGAIN": "Скачать {{title}} снова"
"DOWNLOAD_AGAIN": "Скачать {{title}} снова",
"RESTART_ARCHIVE": "Скачать {{title}} заново"
},
"DOWNLOAD_AGAIN": "Скачать снова"
"DOWNLOAD_AGAIN": "Скачать снова",
"CATCHUP": "Передача",
"ARCHIVE_RESTART_NOTE": "После прерывания скачивание передачи начинается заново."
},
"REMOTE_CONTROL": {
"TITLE": "Дистанционное управление",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "Arşiv URL’sini kopyala",
"ARCHIVE_URL_COPIED": "Arşiv URL’si kopyalandı",
"ARCHIVE_URL_FAILED": "Arşiv URL’si kopyalanamadı",
"ARCHIVE_URL_NOTICE": "URL hesap bilgileri içerebilir. Harici oynatıcılar kaynağın HTTP başlıklarına da ihtiyaç duyabilir."
"ARCHIVE_URL_NOTICE": "URL hesap bilgileri içerebilir. Harici oynatıcılar kaynağın HTTP başlıklarına da ihtiyaç duyabilir.",
"DOWNLOAD_ARCHIVE": "Programı indir (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "Program indirmelere eklendi",
"ARCHIVE_DOWNLOAD_FAILED": "Program indirilemedi. Artık mevcut olmayabilir.",
"ARCHIVE_DOWNLOAD_TS_ONLY": "Arşiv indirmeleri şu anda yalnızca TS akışlarını destekler.",
"ARCHIVE_DOWNLOAD_EXISTS": "Bu program zaten indirmelerde."
},
"FETCH_SUCCESS": "EPG verileri başarıyla alındı",
"IMPORT_PROGRESS": "EPG İçe Aktarımı",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "Bilgi",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "Dosya kurtarma gerekiyor",
"MESSAGE": "Bu indirmeyle ilgisi olmayan bir dosya şurada korundu: {{path}}. Gerekli içeriği kurtarın, bu kurtarma kopyasını silin ve yeniden deneyin.",
"COPY_PATH": "Kurtarma yolunu kopyala"
},
"TITLE": "İndirilenler",
"DOWNLOAD": "İndir",
"PLAY_LOCAL": "Yerel olarak oynat",
@@ -1395,9 +1405,12 @@
"RETRY": "{{title}} için yeniden dene",
"MORE_ACTIONS": "{{title}} için daha fazla eylem",
"COPY_URL": "{{title}} için indirme URL’sini kopyala",
"DOWNLOAD_AGAIN": "{{title}} öğesini yeniden indir"
"DOWNLOAD_AGAIN": "{{title}} öğesini yeniden indir",
"RESTART_ARCHIVE": "{{title}} programını baştan indir"
},
"DOWNLOAD_AGAIN": "Yeniden indir"
"DOWNLOAD_AGAIN": "Yeniden indir",
"CATCHUP": "TV programı",
"ARCHIVE_RESTART_NOTE": "Kesintiden sonra arşiv indirmesi baştan başlar."
},
"REMOTE_CONTROL": {
"TITLE": "Uzaktan kontrol",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "复制回看链接",
"ARCHIVE_URL_COPIED": "已复制回看链接",
"ARCHIVE_URL_FAILED": "无法复制回看链接",
"ARCHIVE_URL_NOTICE": "链接可能包含账户凭据。外部播放器可能还需要此来源的 HTTP 请求头。"
"ARCHIVE_URL_NOTICE": "链接可能包含账户凭据。外部播放器可能还需要此来源的 HTTP 请求头。",
"DOWNLOAD_ARCHIVE": "下载节目 (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "节目已加入下载列表",
"ARCHIVE_DOWNLOAD_FAILED": "无法下载此节目,它可能已不可用。",
"ARCHIVE_DOWNLOAD_TS_ONLY": "回看下载目前仅支持 TS 流。",
"ARCHIVE_DOWNLOAD_EXISTS": "此节目已在下载列表中。"
},
"FETCH_SUCCESS": "EPG 获取成功",
"IMPORT_PROGRESS": "EPG 导入",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "信息",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "需要恢复文件",
"MESSAGE": "与此下载无关的文件已保存在:{{path}}。请保存所需内容,删除此恢复副本,然后重试。",
"COPY_PATH": "复制恢复路径"
},
"TITLE": "下载",
"DOWNLOAD": "下载",
"PLAY_LOCAL": "播放本地文件",
@@ -1395,9 +1405,12 @@
"RETRY": "重试 {{title}}",
"MORE_ACTIONS": "{{title}} 的更多操作",
"COPY_URL": "复制 {{title}} 的下载 URL",
"DOWNLOAD_AGAIN": "再次下载 {{title}}"
"DOWNLOAD_AGAIN": "再次下载 {{title}}",
"RESTART_ARCHIVE": "从头下载 {{title}}"
},
"DOWNLOAD_AGAIN": "再次下载"
"DOWNLOAD_AGAIN": "再次下载",
"CATCHUP": "电视节目",
"ARCHIVE_RESTART_NOTE": "中断后,回看节目的下载将从头开始。"
},
"REMOTE_CONTROL": {
"TITLE": "远程控制",
+16 -3
View File
@@ -816,7 +816,12 @@
"COPY_ARCHIVE_URL": "複製回看連結",
"ARCHIVE_URL_COPIED": "已複製回看連結",
"ARCHIVE_URL_FAILED": "無法複製回看連結",
"ARCHIVE_URL_NOTICE": "連結可能包含帳戶憑證。外部播放器可能還需要此來源的 HTTP 標頭。"
"ARCHIVE_URL_NOTICE": "連結可能包含帳戶憑證。外部播放器可能還需要此來源的 HTTP 標頭。",
"DOWNLOAD_ARCHIVE": "下載節目 (TS)",
"ARCHIVE_DOWNLOAD_QUEUED": "節目已加入下載清單",
"ARCHIVE_DOWNLOAD_FAILED": "無法下載此節目,它可能已無法使用。",
"ARCHIVE_DOWNLOAD_TS_ONLY": "回看下載目前僅支援 TS 串流。",
"ARCHIVE_DOWNLOAD_EXISTS": "此節目已在下載清單中。"
},
"FETCH_SUCCESS": "EPG 已成功擷取",
"IMPORT_PROGRESS": "EPG 匯入",
@@ -1261,6 +1266,11 @@
},
"INFORMATION": "資訊",
"DOWNLOADS": {
"RECOVERY": {
"TITLE": "需要復原檔案",
"MESSAGE": "與此下載無關的檔案已保存在:{{path}}。請儲存所需內容,刪除此復原副本,然後重試。",
"COPY_PATH": "複製復原路徑"
},
"TITLE": "下載",
"DOWNLOAD": "下載",
"PLAY_LOCAL": "播放本機檔案",
@@ -1395,9 +1405,12 @@
"RETRY": "重試 {{title}}",
"MORE_ACTIONS": "{{title}} 的更多操作",
"COPY_URL": "複製 {{title}} 的下載 URL",
"DOWNLOAD_AGAIN": "再次下載 {{title}}"
"DOWNLOAD_AGAIN": "再次下載 {{title}}",
"RESTART_ARCHIVE": "從頭下載 {{title}}"
},
"DOWNLOAD_AGAIN": "再次下載"
"DOWNLOAD_AGAIN": "再次下載",
"CATCHUP": "電視節目",
"ARCHIVE_RESTART_NOTE": "中斷後,回看節目的下載將從頭開始。"
},
"REMOTE_CONTROL": {
"TITLE": "遠端控制",
+16 -1
View File
@@ -322,7 +322,22 @@ function installStreamRoutes(
app.get('/series/:username/:password/:streamId.:ext', seriesResponse);
app.all(
'/timeshift/:username/:password/:duration/:start/:streamId.ts',
streamResponse
(request, response) => {
if (request.params['username'] === 'epg') {
response
.type('video/mp2t')
.send(
readFileSync(
join(
process.cwd(),
'apps/xtream-mock-server/src/fixtures/live.mpegts'
)
)
);
return;
}
streamResponse(request, response);
}
);
app.all('/streaming/timeshift.php', streamResponse);
}
+169 -6
View File
@@ -7,10 +7,173 @@ views. Backend work is handled in the Electron process while the Angular
renderer exposes the global `/workspace/downloads` page, source-scoped route
variants, contextual buttons, and theme-aware styling.
## Xtream archive downloads
The desktop Xtream Live TV programme dialog offers **Download programme (TS)**
for completed, available catch-up programmes in both timeline and list views.
The host captures the playlist, channel and original programme timestamps before
resolving the existing timeshift URL (including the panel timezone). A channel
change while the dialog is open invalidates its action. Downloading does not
select the programme or change playback. Collection views retain archive URL
copying; archive downloads are initially exposed through Xtream Live TV only.
M3U, Stalker and PWA archive downloads, HLS assembly and recording future broadcasts
are outside this feature.
`EpgArchiveDownloadService` submits the resolved URL and the playlist's playback
header allowlist to the existing desktop queue. Pending submission suppression
is per programme identity, so slow resolution does not discard a different
programme's request. HLS URLs are refused explicitly;
the backend also checks the response type and every 188-byte MPEG-TS packet.
No transcoder or media helper is required. The file always uses `.ts`.
The `catchup` download type stores `programme_start` and JSON metadata containing
channel name, start/stop Unix seconds and the known archive expiry. Its unique
identity is `(playlist_id, xtream_id, programme_start)`; movies and episodes keep
their existing identity. `download-schema.ts` widens the SQLite CHECK in a
transaction after older additive migrations, preserving row IDs, files, headers,
resume state and metadata. It also replaces the former global unique index with
separate catch-up and non-catch-up indexes.
Only ended programmes can be enqueued. Known expiry is checked on enqueue,
retry, resume, missing-file recovery and when the queued transfer starts.
Pause keeps its owned partial file, but **resume/retry restarts from byte zero**,
without Range, If-Range, or automatic reconnect append. The queue explains this.
Before truncating, `download-catchup-output.ts` rejects symlinks and hardlinks,
opens without truncation (with O_NOFOLLOW where available), checks descriptor
identity against lstat, then truncates and writes through that same descriptor.
An absent partial is created exclusively, so a replaced path cannot redirect writes.
The task retains that descriptor's device/inode identity through promotion:
`download-catchup-finalize.ts` verifies the source and published file, rejects a
replaced partial, and copies from the verified descriptor on filesystems without
hardlinks. Existing destination files are never overwritten. Cleanup atomically
moves the public pathname into a private temporary directory before checking its
identity and removing it. A captured replacement is restored with no-clobber
linking; if restoration is unavailable or the original pathname is occupied,
the file is retained in `.iptvnator-cleanup-*/entry` and its recovery location
is logged. Cleanup never recursively deletes a nonempty quarantine. This closes
the predictable-path check/unlink window; it does not isolate files from
same-user processes that deliberately enter the private temporary directory.
Active failure and cancellation use the same captured transfer identity; a
partial that was never safely opened is preserved instead of being deleted.
`download_archive_finalizations` is a write-ahead SQLite journal keyed by download
ID (cascade-deleted with the download). It records the path, size, source identity
and expected final identity **before hardlink promotion**, or before the first
byte is written to an exclusively created copy destination. Thus even a completed
unknown-length file has durable proof before the completion-status write. Startup
requires that proof and a matching regular file, identity and size to recover an
archive; termination before promotion leaves the verified partial paused. An
owned incomplete copy is removed by journal identity before the source resumes. The
journal also makes startup partial cleanup identity-aware. A journaled partial
whose identity changed is preserved and detached from the failed row; Retry
reserves a fresh path instead of adopting or truncating it. The journal remains with
completed archives until they are removed. An explicitly restarted transfer
carries the journaled source identity to output opening and checks
it against the opened descriptor. Only then does it replace the previous proof,
before truncation or writes, replacing the old finalization proof with a
transfer-phase journal containing the newly opened identity. That identity
survives pause, failure and restart without claiming the file is complete. A
rejected replacement is detached from the failed row without deletion, so Retry
reserves a new path. Retained catch-up paths without any durable proof are also
preserved and redirected to a fresh reservation. On a final-path collision,
archives never relocate a retained partial: only a verified owned partial is
discarded before reserving a new filename, because archive retries restart at zero. The transfer proof is upgraded to finalization proof only
after validated EOF. Process-local proof allows immediate
recovery after a transient completion DB error without waiting for a restart.
Fallback copying observes pause/cancel between bounded 64 KiB reads/writes and
before publication completes. Interruption removes only the owned copy and
leaves the source for the runtime pause/cancel handler. Once publication identity
and size pass the final check, the task synchronously enters completion commit
before awaited partial cleanup and the SQLite completion write. Pause/cancel then
return false without setting flags; a command is never accepted and subsequently
overwritten by completion. Remove rejects this committing row before partial
cleanup or deletion, and Clear completed skips it, preserving the cascading
journal until completion finishes. Remove waits for an accepted active archive
cancellation to settle before reading/deleting its row; queued archives are
canceled first, and a concurrent new runtime attempt blocks removal. A settled
archive still marked downloading after a failed status write also retries
journal cleanup before row deletion. Remove/Clear preserve a final file whose
journaled device/inode/creation-time identity and full size prove completed promotion, even when completion
status writes failed and its stored status is stale. Repeat submissions, Retry
and Resume restore such a journal-proven completion in place before any new
transfer or ownership reset, before expiry, provider DNS and new-folder checks
that apply only to another remote transfer; retained cleanup failures keep their
journal. Remove, Clear completed and missing-file
re-download and repeated programme submissions use journal-backed private
capture for archive partial cleanup;
unknown or replaced entries are preserved. Ownership reads device/inode as BigInt and journals decimal strings without
losing 64-bit Windows file references, alongside a positive file creation timestamp, so inode reuse after unlink cannot
bless a new entry; proofs lacking creation time remain untrusted. Fresh
reservations capture this identity from their exclusive creation descriptor and
commit it together with the downloads row path/name in one SQLite transaction
before the HTTP wait. A committed reservation is therefore recoverable even if
the process exits before transfer setup. Existing partials without matching expected
ownership are never truncated, including before the first response. Before capture, a synchronous SQLite
write records `partialCleanupPath` (or `finalCleanupPath` for failed promotion)
in the existing ownership proof. Active cancellation/failure, promotion and
startup recovery use the same journal-backed cleanup as manual actions. Cancel,
failure and removal of an unfinished attempt also clean its journaled final target;
removing a completed row preserves its media. Retry cleans an incomplete owned
final before reserving a destination. A failed
unlink or replacement restoration keeps that durable pointer and blocks journal
deletion/replacement. Later cleanup retries no-clobber restoration of captured
foreign entries; an occupied public path or unsupported hardlinks preserves both
the capture and its journal for recovery. Even after a foreign entry is restored,
its private recovery copy is never automatically unlinked: another process could
remove the public link first. Remove/Clear return a structured recovery path and open a persistent, localized
dialog with the full path, Copy recovery path and manual recovery instructions.
Copy keeps the dialog open, including on clipboard failure; Close dismisses it.
After the user recovers it and explicitly removes that private copy, cleanup may release the
journal. Ordinary owned-file cleanup remains automatic. Remove/Clear, Retry/Resume and fresh
reservations retry identity-verified cleanup, including after restart and on
filesystems without hardlinks. Cleanup remains synchronous after the
runtime guard, so a completion transition cannot interleave with unlink.
Missing archive re-downloads claim a fresh reservation instead of inheriting the
completed file's identity.
If the initial reservation journal write fails, persistence is retried once
before journal-backed cleanup. If SQLite remains unavailable, the empty
reservation stays at its public path; no unjournaled entry is relocated. An
empty fallback-copy target whose final proof cannot commit is likewise preserved.
A kill between exclusive reservation/copy-file creation and its identity journal
commit, or persistent SQLite failure before that commit, can leave an unowned
**empty** destination: no bytes are written before the commit.
Recovery preserves that file rather than guessing ownership; Retry uses a
numbered free destination. SQLite and filesystem creation cannot commit
atomically, and portable rename cannot guarantee no-clobber publication on the
filesystems that need this fallback. This bounded orphan is preferred to deleting
or overwriting an unrelated file.
Cancellation of queued/paused archives uses the same journal-backed cleanup as
Remove; unproven or replaced entries, including symlinks, are preserved.
A retained archive cannot use the VOD byte-count completion shortcut. Transfers
have a 30-second idle timeout and a total deadline of twice programme duration
plus ten minutes, capped at 24 hours. Transfers also stop at the smallest of a
100 Mbit/s budget for the programme duration plus one minute, 64 GiB, and
half of the initial available space after subtracting a 1 GiB reserve (reserving a second
copy for filesystems without hardlinks). The retained partial is safely truncated
through its verified descriptor before computing this budget, so Resume/Retry
can reuse its released space. Known Content-Length values above
that budget are rejected before writing; unknown-length responses are counted
before forwarding chunks. Free space is rechecked every 16 MiB to account for
other disk activity, and copy headroom is checked again against the final byte
count after the output stream closes, including short tails below that interval. These safety limits apply to TS archives only. Failure never promotes the partial to the
library, and errors omit credential-bearing URLs.
Completion means clean HTTP EOF, matching Content-Length when supplied, and
valid nonempty TS packet framing. **It does not verify media duration or the
provider's programme boundaries.** A provider can return a shorter valid clip
with clean EOF; this version cannot identify that without demuxing duration.
Unknown-length responses show indeterminate progress until EOF.
Completed archive cards appear under All with programme title, channel and
broadcast date. Clicking a card plays the local file through the download action;
it does not navigate to a movie or series detail route. Files and metadata remain
available after restart and after the source archive expires.
## Backend responsibilities
- **Queue control (`apps/electron-backend/src/app/events/database/download-runtime.ts`)**
`DownloadTask` mirrors a row of the shared `downloads` table (type `Download` in `libs/shared/database/src/lib/schema.ts`) plus transient cancel/pause/progress helpers (shared task types live in `download-task.ts`). Request validation and row creation live in `download-requests.ts`, while `downloads.events.ts` stays focused on IPC registration. `enqueueDownload()` pushes the task onto `downloadQueue` and triggers `processQueue()`. `processQueue()` keeps one active download, updates the row to `downloading`, and calls `startDownload()`. The byte transfer itself lives in `download-transfer.ts`, finalization and retained-partial persistence in `download-finalize.ts`, and the renderer update broadcast in `download-broadcast.ts`.
`DownloadTask` mirrors a row of the shared `downloads` table (type `Download` in `libs/shared/database/src/lib/schema.ts`) plus transient cancel/pause/progress helpers (shared task types live in `download-task.ts`). Request validation and row creation live in `download-requests.ts`, retry/resume flows in `download-resume-requests.ts`, removal/terminal cleanup in
`download-removal-requests.ts`, while `downloads.events.ts` stays focused on IPC registration. `enqueueDownload()` pushes the task onto `downloadQueue` and triggers `processQueue()`. `processQueue()` keeps one active download, updates the row to `downloading`, and calls `startDownload()`. The byte transfer itself lives in `download-transfer.ts`, finalization and retained-partial persistence in `download-finalize.ts` (ordinary file promotion in `download-file-finalize.ts`, archive completion in
`download-catchup-completion.ts`, proven completion recovery in `download-catchup-recover-completion.ts`, target reservation in `download-runtime-reservation.ts`), cancellation/pause persistence in `download-runtime-persistence.ts`, and the renderer update broadcast in `download-broadcast.ts`.
- **Range-aware transfer (`download-transfer.ts`)**
The transfer streams the response through the backend's validated Axios redirect helper instead of `electron-dl`, and always requests `Accept-Encoding: identity`: Range offsets, totals, and the persisted `.part` must describe the same representation, and Axios's transparent gzip/brotli decoding would put decoded bytes on disk while every counter speaks encoded bytes. Headers (user agent, referer, origin) are persisted in `request_headers` and re-applied through the same allowlist when read back on retry/resume. Fresh Xtream movie and series-episode downloads propagate the playlist's configured headers, using its User-Agent when present and otherwise sharing the provider-compatible `XTREAM_CLIENT_USER_AGENT` used by Xtream API requests and stream probes. Retry, resume, and missing-file recovery resolve the owning playlist type and add that fallback to legacy Xtream rows without a stored User-Agent; known Stalker rows are left unchanged. Download rows deliberately outlive individually deleted playlists, so a headerless legacy row whose source no longer exists receives the same IPTV-player fallback because its original provider type cannot be recovered. Active pause/cancel operations abort the current request with `AbortController`; pause keeps the partial file and cancel removes it. Resume checks the existing `.part` size (rejecting anything that is not a regular file, so a symlink planted while paused is never followed). The first response's strong `ETag` (or `Last-Modified`) is persisted in `resume_validator`; a partial carrying that validator resumes with `Range: bytes=<offset>-` plus `If-Range`, so the server itself proves the entity is unchanged. A retained partial **without** a validator resumes through overlap verification instead (`download-overlap.ts`): the `Range` request rewinds by up to 256 KiB (`OVERLAP_VERIFICATION_BYTES`) and a transform stream compares that replayed window byte-for-byte against the partial's tail before anything is appended — a self-made validator for the many Xtream panels that send neither header. A mismatching overlap truncates the `.part` and restarts the transfer from byte zero (`OverlapMismatchError`); a partial smaller than the overlap window is verified in full from byte zero over a plain request and appended to — never rewritten in place, so a reconnect that dies early can only grow the file. Success requires the verifier to have consumed its ENTIRE window: a response that ends inside the overlap is an ordinary retained interruption when the stream died early, but a response that delivered its complete AUTHORITATIVE total inside the window — whether it then closed cleanly or reset — proves the remote entity shrank and restarts from scratch; the old suffix is never finalized as a completed file. An HTTP 416 answer to a resume request is classified by `classifyRangeNotSatisfiable()`: it COMPLETES only an exact-EOF request with identity proof (`If-Range`-backed, or the EOF probe that follows a fully verified overlap replay) whose stated `bytes */N` equals the partial — a bare length match on a rewound request proves nothing about whose bytes are on disk; it RESTARTS only when a STATED total proves the entity shrank — the total sits below a rewound request's first byte, or at it (the rewound range beginning exactly at the new EOF), or below the partial at an exact-EOF request; every length-less, ambiguous, or contradictory 416 RETAINS the partial, and none of these paths ever reaches generic cleanup. A validator promoted by a complete overlap match survives mid-append failures too: the promotion also runs on the error path, and retained-failure and pause persistence write `resume_validator` from the task, so later attempts resume via `If-Range` instead of replaying the window — without this, a server whose per-connection cap barely exceeds the window would stall out on sub-threshold progress. A verify-append attempt promotes the response's `ETag`/`Last-Modified` onto the row only after the complete overlap matched; until then the retained bytes are unproven and blessing them with a validator would let the next resume `If-Range`-append onto a foreign prefix. The response's TOTAL stays equally uncommitted (task and row) until the overlap matched — a persisted total equal to the unverified partial's size would let the completed-partial shortcut finalize unproven bytes after a pause, crash, or retained failure. Retained-interruption persistence keeps the live task in sync with the row (a stale falsified total would make the next reconnect's resume-offset guard reject the partial). Overlap replay re-counts bytes from the rewound offset, so reported progress is floored at the partial's retained size whenever the transfer appends — a response that ends inside the overlap can never move displayed progress backwards. A `206 Partial Content` answer must start at the requested offset (`Content-Range` is verified) before bytes are appended; any other 2xx answer Line truncated
- **Destination collision policy**
@@ -294,12 +457,12 @@ display snapshot via `playlistDisplayLabel`).
- **Lifecycle tracking** is owned by `EmbeddedMpvRecordingTracker`
(`apps/electron-backend/src/app/services/embedded-mpv-recording-tracker.ts`):
explicit start/stop hooks in `EmbeddedMpvNativeService` plus a
session-snapshot observer. The stop hook is a *request*, not an outcome —
session-snapshot observer. The stop hook is a _request_, not an outcome —
`addon.stopRecording()` only dispatches (async mpv property set, or a
command written to the frame-copy helper), so finalization always waits for
the snapshot reporting the recording inactive; statting or unlinking
earlier would report a short recording as failed and could delete bytes mpv
is still flushing. macOS native-view clears `recordingActive` *before*
is still flushing. macOS native-view clears `recordingActive` _before_
dispatching the async property set and restores it if that request is
rejected, so an inactive snapshot must additionally survive a 1.5 s settle
window (three poll cycles) before it counts as an acknowledgement; a revived
@@ -355,7 +518,7 @@ display snapshot via `playlistDisplayLabel`).
`RecordingStartMetadata` (channel name/logo, playlist id + display-label
snapshot, source type, EPG key, current program) that flows
`WebPlayerViewComponent → EmbeddedMpvPlayerComponent →
EmbeddedMpvControlsAdapter → EmbeddedMpvRecordingStartOptions.metadata`.
EmbeddedMpvControlsAdapter → EmbeddedMpvRecordingStartOptions.metadata`.
`EmbeddedMpvPlayerComponent` watches the session snapshot for the
active→inactive recording edge and emits `recordingStopped` — one owner for
every trigger, including a Stop clicked in the download manager, which
@@ -377,14 +540,14 @@ display snapshot via `playlistDisplayLabel`).
target path — that is how a recording spanning a program boundary lists
every covered show. The handler is deliberately independent of finalization: it
looks the row up in **any** status (the newest for that path — `openSync
('wx')` keeps a reserved path exclusive while its recording owns it) and
('wx')` keeps a reserved path exclusive while its recording owns it) and
`finalize()` never touches `programs_json`, so the two writes commit in
either order. The only wait is the tracker's queue drain, which guarantees
the row's INSERT exists — no deadline, and therefore no way for a one-shot
enrichment to be dropped by a clock. A recording stopped while no player is mounted on that
channel keeps its start snapshot.
- **IPC surface** (`recordings.events.ts`): `RECORDINGS_GET_LIST/GET/STOP/
REMOVE/UPDATE_PROGRAMS/REVEAL_FILE/PLAY_FILE` plus the dedicated
REMOVE/UPDATE_PROGRAMS/REVEAL_FILE/PLAY_FILE` plus the dedicated
`RECORDINGS_UPDATE_EVENT` bare ping (not shared with downloads, so
recording transitions do not force availability-probed download refetches).
Active rows are decorated with a live `fs.stat` size — `file_size_bytes` is
@@ -316,3 +316,11 @@ or without, the account-info check. The same timezone lets
clock the panel wrote them in (`parseXtreamServerLocalDateTime`);
`start_timestamp` still wins whenever it is present. Formatting uses
`hourCycle: 'h23'`, so server midnight renders as `00`, never `24`.
### Downloading completed catch-up programmes
Desktop Xtream Live TV exposes a TS-only download action in programme details.
It reuses the canonical timeshift resolver and original timestamps, preserves
playback headers and does not change playback. See
[Download Manager](download-manager.md#xtream-archive-downloads) for identity,
restart, expiry and transport-completion limits.
@@ -32,6 +32,7 @@ export class DownloadLibraryNavigationService {
availablePlaylistIds: ReadonlySet<string>
): boolean {
return (
item.contentType !== 'catchup' &&
availablePlaylistIds.has(item.playlistId) &&
this.targetId(item) !== null
);
@@ -47,6 +48,7 @@ export class DownloadLibraryNavigationService {
async resolveProviderTarget(
item: DownloadItem
): Promise<WorkspaceNavigationTarget | null> {
if (item.contentType === 'catchup') return null;
const targetId = this.targetId(item);
if (targetId === null) {
return null;
@@ -198,7 +200,7 @@ export class DownloadLibraryNavigationService {
const matched = findMatchingStalkerDownloadRecent(
recent,
targetId,
item.contentType
item.contentType === 'episode' ? 'episode' : 'vod'
);
if (matched) {
@@ -29,8 +29,10 @@
"
[disabled]="isPending(representativeOf(entity))"
[attr.aria-label]="
'DOWNLOADS.OPEN_ARTWORK'
| translate: { title: entityTitle(entity) }
(entity.kind === 'catchup'
? 'DOWNLOADS.PLAY'
: 'DOWNLOADS.OPEN_ARTWORK'
) | translate: { title: entityTitle(entity) }
"
(click)="openDetails(representativeOf(entity))"
>
@@ -199,7 +201,10 @@
"
[disabled]="isPending(representativeOf(entity))"
[attr.aria-label]="
('DOWNLOADS.OPEN_DETAILS' | translate) +
((entity.kind === 'catchup'
? 'DOWNLOADS.PLAY'
: 'DOWNLOADS.OPEN_DETAILS'
) | translate) +
': ' +
entityTitle(entity)
"
@@ -260,6 +265,18 @@
}
</div>
}
@if (
entity.kind === 'catchup' && entity.item.catchup;
as archive
) {
<div class="download-library__facts">
{{ archive.channelName }} ·
{{
archive.startTimestamp * 1000
| date: 'medium'
}}
</div>
}
@if (entity.kind === 'episode' && entity.episodeLabel) {
<span class="download-library__episode-label">
{{ entity.episodeLabel }}
@@ -135,6 +135,38 @@ describe('DownloadLibraryComponent', () => {
let fixture: ComponentFixture<DownloadLibraryComponent>;
let component: DownloadLibraryComponent;
it('plays an archive card locally instead of opening provider details', async () => {
const archive = item({
contentType: 'catchup',
catchup: {
channelName: 'News TV',
startTimestamp: 1788800000,
stopTimestamp: 1788803600,
},
});
fixture.componentRef.setInput('entities', [
{
kind: 'catchup',
key: 'catchup:1',
item: archive,
newestTimestamp: 0,
sourceName: 'Source',
trackedBytes: 0,
},
]);
await fixture.whenStable();
const actions = jest.fn(),
open = jest.fn();
component.itemAction.subscribe(actions);
component.openRequested.subscribe(open);
fixture.nativeElement
.querySelector('.download-library__title-button')
.click();
expect(actions).toHaveBeenCalledWith({ type: 'play', item: archive });
expect(open).not.toHaveBeenCalled();
expect(fixture.nativeElement.textContent).toContain('News TV');
});
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [
@@ -1,3 +1,4 @@
import { DatePipe } from '@angular/common';
import {
ChangeDetectionStrategy,
Component,
@@ -25,6 +26,7 @@ import { DownloadSourceMenuHeaderComponent } from './download-source-menu-header
selector: 'app-download-library',
standalone: true,
imports: [
DatePipe,
MatButtonModule,
MatIconModule,
MatMenuModule,
@@ -46,6 +48,8 @@ export class DownloadLibraryComponent {
protected libraryTestId(entity: DownloadLibraryEntity): string {
switch (entity.kind) {
case 'catchup':
return `download-library-catchup-${entity.item.id}`;
case 'movie':
return `download-library-movie-${entity.item.id}`;
case 'episode':
@@ -65,6 +69,8 @@ export class DownloadLibraryComponent {
protected typeKey(entity: DownloadLibraryEntity): string {
switch (entity.kind) {
case 'catchup':
return 'DOWNLOADS.CATCHUP';
case 'movie':
return 'DOWNLOADS.MOVIE';
case 'episode':
@@ -78,6 +84,7 @@ export class DownloadLibraryComponent {
switch (entity.kind) {
case 'movie':
return 'movie';
case 'catchup':
case 'episode':
return 'live_tv';
case 'series':
@@ -129,7 +136,9 @@ export class DownloadLibraryComponent {
protected openDetails(item: DownloadItem): void {
if (!this.isPending(item)) {
this.openRequested.emit(item);
if (item.contentType === 'catchup')
this.itemAction.emit({ type: 'play', item });
else this.openRequested.emit(item);
}
}
@@ -21,6 +21,10 @@ export interface DownloadMovieCardViewModel extends DownloadItemLibraryEntityBas
readonly kind: 'movie';
}
export interface DownloadCatchupCardViewModel extends DownloadItemLibraryEntityBase {
readonly kind: 'catchup';
}
export interface DownloadEpisodeCardViewModel extends DownloadItemLibraryEntityBase {
readonly kind: 'episode';
readonly episodeLabel: string;
@@ -39,6 +43,7 @@ export interface DownloadSeriesCardViewModel extends DownloadLibraryEntityBase {
export type DownloadLibraryEntity =
| DownloadMovieCardViewModel
| DownloadCatchupCardViewModel
| DownloadEpisodeCardViewModel
| DownloadSeriesCardViewModel;
@@ -216,10 +221,10 @@ export function buildDownloadLibrary(
for (const row of rows) {
const { item } = row;
if (item.contentType === 'vod') {
if (item.contentType === 'vod' || item.contentType === 'catchup') {
entities.push({
kind: 'movie',
key: `movie:${item.id}`,
kind: item.contentType === 'catchup' ? 'catchup' : 'movie',
key: `${item.contentType === 'catchup' ? 'catchup' : 'movie'}:${item.id}`,
item,
newestTimestamp: normalizedDownloadTimestamp(item.createdAt),
sourceName: row.sourceName,
@@ -11,6 +11,7 @@ import type {
interface DownloadOperationResult {
readonly success: boolean;
readonly error?: string;
readonly recoveryPath?: string;
}
@Injectable()
@@ -104,7 +105,8 @@ export class DownloadManagerActionsService {
try {
const result = await this.downloads.clearCompleted(scopePlaylistId);
if (!result.success) {
this.showActionError();
if (result.recoveryPath) this.showRecovery(result.recoveryPath);
else this.showActionError();
}
} catch (error) {
this.showActionError(
@@ -172,6 +174,10 @@ export class DownloadManagerActionsService {
let failed = false;
try {
const result = await operation();
if (result.recoveryPath) {
this.showRecovery(result.recoveryPath);
return 'failed';
}
failed = !result.success;
failure = result.error;
} catch (error) {
@@ -199,6 +205,28 @@ export class DownloadManagerActionsService {
}
}
private showRecovery(path: string): void {
this.dialogs.openConfirmDialog({
title: this.translate.instant('DOWNLOADS.RECOVERY.TITLE'),
keepOpenOnConfirm: true,
cancelLabel: this.translate.instant('CLOSE'),
width: '600px',
message: this.translate.instant('DOWNLOADS.RECOVERY.MESSAGE', {
path,
}),
confirmLabel: this.translate.instant(
'DOWNLOADS.RECOVERY.COPY_PATH'
),
onConfirm: async () => {
try {
await navigator.clipboard.writeText(path);
} catch {
this.showActionError();
}
},
});
}
private showFileActionError(error?: string): void {
this.showMessage(
error === 'File not found'
@@ -3,6 +3,7 @@ import type { Playlist } from '@iptvnator/shared/interfaces';
import {
buildDownloadManagerViewModel,
normalizeDownloadFilter,
type DownloadCatchupCardViewModel,
type DownloadEpisodeCardViewModel,
type DownloadLibraryEntity,
type DownloadManagerViewModel,
@@ -13,6 +14,7 @@ import {
type PlaylistSummary = Pick<Playlist, '_id' | 'title'>;
type ExpectedDownloadLibraryEntity =
| DownloadMovieCardViewModel
| DownloadCatchupCardViewModel
| DownloadEpisodeCardViewModel
| DownloadSeriesCardViewModel;
@@ -1092,3 +1094,18 @@ describe('buildDownloadManagerViewModel', () => {
}
});
});
it('keeps separate archive programmes out of movie and series counts', () => {
const items = [
download(1, { contentType: 'catchup', xtreamId: 100 }),
download(2, { contentType: 'catchup', xtreamId: 100 }),
];
const model = build(items);
expect(model.library.map(({ kind }) => kind)).toEqual([
'catchup',
'catchup',
]);
expect(model.counts).toMatchObject({ all: 2, movie: 0, series: 0 });
expect(build(items, { filter: 'movie' }).library).toEqual([]);
expect(build(items, { filter: 'series' }).library).toEqual([]);
});
@@ -11,6 +11,7 @@ import {
} from './download-library.viewmodel';
export type {
DownloadCatchupCardViewModel,
DownloadEpisodeCardViewModel,
DownloadLibraryEntity,
DownloadMovieCardViewModel,
@@ -18,11 +19,7 @@ export type {
} from './download-library.viewmodel';
export type DownloadFilterId =
| 'all'
| 'movie'
| 'series'
| 'in-progress'
| 'recording';
'all' | 'movie' | 'series' | 'in-progress' | 'recording';
export type DownloadAttentionReason = 'file-missing' | 'transfer';
@@ -231,7 +228,9 @@ export function buildDownloadManagerViewModel({
series:
countContentType(scopedActive, 'episode') +
countContentType(scopedAttention, 'episode') +
scopedLibrary.filter(({ kind }) => kind !== 'movie').length,
scopedLibrary.filter(
({ kind }) => kind === 'series' || kind === 'episode'
).length,
inProgress: scopedActive.length,
},
activeCount: scopedItems.filter(
@@ -33,7 +33,7 @@
'DOWNLOADS.ARIA.OPEN'
| translate: { title: row.item.title }
"
[disabled]="isPending(row.item.id)"
[disabled]="isPending(row.item.id) || row.item.contentType === 'catchup'"
(click)="requestOpen(row.item)"
>
@if (hasPoster(row.item)) {
@@ -55,9 +55,9 @@
>
<mat-icon aria-hidden="true">
{{
row.item.contentType === 'episode'
? 'tv'
: 'movie'
row.item.contentType === 'vod'
? 'movie'
: 'tv'
}}
</mat-icon>
</span>
@@ -74,7 +74,7 @@
'DOWNLOADS.ARIA.OPEN'
| translate: { title: row.item.title }
"
[disabled]="isPending(row.item.id)"
[disabled]="isPending(row.item.id) || row.item.contentType === 'catchup'"
(click)="requestOpen(row.item)"
>
{{ row.item.title }}
@@ -86,6 +86,13 @@
}
</div>
@if (row.item.contentType === 'catchup') {
<p class="download-queue__episode">
{{
'DOWNLOADS.ARCHIVE_RESTART_NOTE' | translate
}}
</p>
}
<div class="download-queue__metadata">
<span
class="download-queue__status"
@@ -178,12 +185,18 @@
data-test-action="resume"
[disabled]="isPending(row.item.id)"
[attr.aria-label]="
'DOWNLOADS.ARIA.RESUME'
(row.item.contentType === 'catchup'
? 'DOWNLOADS.ARIA.RESTART_ARCHIVE'
: 'DOWNLOADS.ARIA.RESUME'
)
| translate
: { title: row.item.title }
"
[matTooltip]="
'DOWNLOADS.ARIA.RESUME'
(row.item.contentType === 'catchup'
? 'DOWNLOADS.ARIA.RESTART_ARCHIVE'
: 'DOWNLOADS.ARIA.RESUME'
)
| translate
: { title: row.item.title }
"
@@ -114,6 +114,7 @@ export class DownloadQueueComponent {
}
requestOpen(item: DownloadItem): void {
if (item.contentType === 'catchup') return;
if (!this.isPending(item.id)) {
this.openRequested.emit(item);
}
@@ -39,6 +39,8 @@ interface Deferred<T> {
}
interface ConfirmConfig {
readonly confirmLabel?: string;
readonly cancelLabel?: string;
readonly title: string;
readonly message: string;
readonly onConfirm: () => Promise<void> | void;
@@ -335,8 +337,15 @@ describe('DownloadsComponent', () => {
const translate = TestBed.inject(TranslateService);
translate.setTranslation('en', {
CLOSE: 'Close',
DOWNLOADS: {
TITLE: 'Downloads',
RECOVERY: {
TITLE: 'File recovery needed',
MESSAGE:
'Recover any needed content at {{path}}, remove that recovery copy, then try again.',
COPY_PATH: 'Copy recovery path',
},
ACTIVE_COUNT: '{{count}} active',
TRACKED_DOWNLOADS: 'Tracked downloads',
CHANGE_FOLDER: 'Change folder',
@@ -640,6 +649,51 @@ describe('DownloadsComponent', () => {
}
);
it.each(['remove', 'clear'])(
'shows a persistent recovery dialog with a copyable path on %s',
async (action) => {
const recoveryPath = '/downloads/.iptvnator-cleanup-example/entry';
const result = { success: false, recoveryPath };
downloadsService.removeDownload.mockResolvedValueOnce(result);
downloadsService.clearCompleted.mockResolvedValueOnce(result);
if (action === 'remove')
component.runAction({
type: 'remove',
item: download(7, { status: 'failed' }),
});
else component.clearFinished();
await dialogConfigs[0].onConfirm();
expect(dialogConfigs[1]).toEqual(
expect.objectContaining({
title: 'File recovery needed',
message: expect.stringContaining(recoveryPath),
confirmLabel: 'Copy recovery path',
keepOpenOnConfirm: true,
cancelLabel: 'Close',
})
);
expect(snackBar.open).not.toHaveBeenCalled();
expect(component.pendingIds().size).toBe(0);
const original = Object.getOwnPropertyDescriptor(
navigator,
'clipboard'
);
const writeText = jest.fn().mockResolvedValue(undefined);
Object.defineProperty(navigator, 'clipboard', {
configurable: true,
value: { writeText },
});
try {
await dialogConfigs[1].onConfirm();
expect(writeText).toHaveBeenCalledWith(recoveryPath);
} finally {
if (original)
Object.defineProperty(navigator, 'clipboard', original);
else Reflect.deleteProperty(navigator, 'clipboard');
}
}
);
it('describes both finalized and retained-partial outcomes when clearing', async () => {
component.clearFinished();
@@ -328,6 +328,10 @@ export class DownloadsComponent {
}
openOfflineDetail(item: DownloadItem): void {
if (item.contentType === 'catchup') {
void this.actions.run({ type: 'play', item });
return;
}
if (this.pendingIds().has(item.id)) {
return;
}
@@ -338,6 +342,7 @@ export class DownloadsComponent {
}
async openInLibrary(item: DownloadItem): Promise<void> {
if (item.contentType === 'catchup') return;
if (this.pendingIds().has(item.id)) {
return;
}
@@ -374,7 +374,11 @@ export function buildDownloadOfflineDetail({
}
const representative = downloads.find(({ id }) => id === downloadId);
if (!representative || !isLocallyAvailable(representative)) {
if (
!representative ||
representative.contentType === 'catchup' ||
!isLocallyAvailable(representative)
) {
return undefined;
}
@@ -238,7 +238,7 @@ export class DownloadOfflineMetadataService {
provider: findMatchingStalkerDownloadRecent(
recent,
targetId(detail),
item.contentType
item.contentType === 'episode' ? 'episode' : 'vod'
),
};
} catch {
@@ -1,7 +1,7 @@
import type {
ElectronBridgeDownloadStatus,
ElectronBridgeEpisodeIdentityScope,
ElectronBridgePlaybackContentType,
DownloadContentType,
ElectronDownloadFileAvailability,
} from '@iptvnator/shared/interfaces';
@@ -18,7 +18,7 @@ export interface EpisodeDownloadIdentity {
export interface EpisodeDownloadRecord {
readonly id: number;
readonly playlistId: string;
readonly contentType: ElectronBridgePlaybackContentType;
readonly contentType: DownloadContentType;
readonly xtreamId: number;
readonly seriesXtreamId?: number | null;
readonly seasonNumber?: number | null;
@@ -1,3 +1,4 @@
import { EpgArchiveDownloadService } from '@iptvnator/ui/epg';
import { EpgArchiveCopyService } from '@iptvnator/ui/epg';
import { computed, signal } from '@angular/core';
import { ComponentFixture, TestBed } from '@angular/core/testing';
@@ -90,6 +91,10 @@ describe('Xtream live auto-open playback queue', () => {
await TestBed.configureTestingModule({
imports: [LiveStreamLayoutComponent],
providers: [
{
provide: EpgArchiveDownloadService,
useValue: { start: jest.fn() },
},
{
provide: EpgArchiveCopyService,
useValue: { copy: jest.fn() },
@@ -73,6 +73,7 @@ export class StubEpgTimelineComponent {
readonly channelLogo = input('');
readonly sourceLabel = input('');
readonly archivePlaybackAvailable = input(false);
readonly archiveDownloadAvailable = input(false);
readonly archiveContextKey = input<string | null>(null);
readonly archiveDays = input(0);
readonly activeProgram = input<EpgProgram | null>(null);
@@ -94,3 +95,17 @@ export class StubEpgTimelineComponent {
standalone: true,
})
export class StubResizableDirective {}
export const sampleChannel = {
xtream_id: 101,
name: 'Channel 101',
stream_icon: 'channel-101.png',
tv_archive: 1,
tv_archive_duration: 3,
};
export const playlist = {
id: 'playlist-1',
serverUrl: 'http://demo.example',
username: 'demo',
password: 'secret',
};
@@ -230,6 +230,7 @@
[channelLogo]="activePlayback()?.thumbnail ?? ''"
[sourceLabel]="'Xtream'"
[archiveContextKey]="archiveContextKey()"
[archiveDownloadAvailable]="archiveDownloadsAvailable()"
[archivePlaybackAvailable]="archivePlaybackAvailable()"
[archiveDays]="controlledArchiveDays()"
[activeProgram]="activeCatchupProgram()"
@@ -252,6 +253,7 @@
[channelLogo]="activePlayback()?.thumbnail ?? ''"
[sourceLabel]="'Xtream'"
[archiveContextKey]="archiveContextKey()"
[archiveDownloadAvailable]="archiveDownloadsAvailable()"
[archivePlaybackAvailable]="archivePlaybackAvailable()"
[archiveDays]="controlledArchiveDays()"
[activeProgram]="activeCatchupProgram()"
@@ -24,6 +24,7 @@ import {
XtreamUrlService,
} from '@iptvnator/portal/xtream/data-access';
import {
EpgArchiveDownloadService,
EpgArchiveCopyService,
EpgListViewComponent,
EpgTimelineComponent,
@@ -44,6 +45,8 @@ import { RuntimeCapabilitiesService, SettingsStore } from '@iptvnator/services';
import { createPlaybackSessionKey } from '@iptvnator/playback/util';
import {
sampleChannel,
playlist,
StubEpgTimelineComponent,
StubGridListComponent,
StubPortalChannelsListComponent,
@@ -61,20 +64,6 @@ describe('LiveStreamLayoutComponent', () => {
>;
const fixedNow = new Date('2026-04-05T12:00:00.000Z');
const sampleChannel = {
xtream_id: 101,
name: 'Channel 101',
stream_icon: 'channel-101.png',
tv_archive: 1,
tv_archive_duration: 3,
};
const playlist = {
id: 'playlist-1',
serverUrl: 'http://demo.example',
username: 'demo',
password: 'secret',
};
const categories = signal([{ category_id: 1, category_name: 'News' }]);
const categoryItemCounts = signal(new Map<number, number>([[1, 1]]));
const epgItems = signal<EpgItem[]>([]);
@@ -186,6 +175,14 @@ describe('LiveStreamLayoutComponent', () => {
await TestBed.configureTestingModule({
imports: [LiveStreamLayoutComponent, NoopAnimationsModule],
providers: [
{
provide: EpgArchiveDownloadService,
useValue: {
start: jest.fn(async (_input, resolve) => {
await resolve();
}),
},
},
{
provide: EpgArchiveCopyService,
useValue: {
@@ -226,6 +223,7 @@ describe('LiveStreamLayoutComponent', () => {
{
provide: RuntimeCapabilitiesService,
useValue: {
supportsDownloads: true,
get supportsEpg() {
return Boolean(window.electron);
},
@@ -785,6 +783,42 @@ describe('LiveStreamLayoutComponent', () => {
);
});
it('queues a completed programme with its channel identity and headers without changing playback', async () => {
const before = component.activePlayback();
const start = Math.floor(Date.now() / 1000) - 7200;
await component.onProgramActivated({
type: 'download-catchup',
program: {
start: new Date(start * 1000).toISOString(),
stop: new Date((start + 3600) * 1000).toISOString(),
startTimestamp: start,
stopTimestamp: start + 3600,
channel: 'channel-101',
title: 'Archive',
desc: null,
category: null,
},
});
expect(
TestBed.inject(EpgArchiveDownloadService).start
).toHaveBeenCalledWith(
expect.objectContaining({
playlistId: playlist.id,
xtreamId: 101,
title: 'Archive',
catchup: expect.objectContaining({
startTimestamp: start,
stopTimestamp: start + 3600,
}),
headers: expect.objectContaining({
userAgent: expect.any(String),
}),
}),
expect.any(Function)
);
expect(component.activePlayback()).toEqual(before);
});
it('copies the programme URL without changing playback or channel selection', async () => {
const before = component.activePlayback();
xtreamStore.openPlayer.mockClear();
@@ -1,3 +1,8 @@
import {
activateXtreamArchiveAction,
getProgramTimestampSeconds,
} from './xtream-live-archive-actions';
import { EpgArchiveDownloadService } from '@iptvnator/ui/epg';
import { NgTemplateOutlet } from '@angular/common';
import {
ChangeDetectionStrategy,
@@ -153,6 +158,10 @@ export class LiveStreamLayoutComponent
this.selectedLiveItem()?.xtream_id,
])
);
private readonly archiveDownloads = inject(EpgArchiveDownloadService);
readonly archiveDownloadsAvailable = computed(
() => this.runtime.supportsDownloads && this.archivePlaybackAvailable()
);
private readonly archiveCopy = inject(EpgArchiveCopyService);
private readonly xtreamUrlService = inject(XtreamUrlService);
private readonly runtime = inject(RuntimeCapabilitiesService);
@@ -619,28 +628,21 @@ export class LiveStreamLayoutComponent
return;
}
if (event.type === 'copy-catchup-url') {
const playlist = this.xtreamStore.currentPlaylist();
await this.archiveCopy.copy(() => {
const start = this.getProgramTimestampSeconds(
event.program.start,
event.program.startTimestamp
);
const stop = this.getProgramTimestampSeconds(
event.program.stop,
event.program.stopTimestamp
);
return playlist && start && stop && stop > start
? this.xtreamUrlService.resolveCatchupUrl(
playlist.id,
playlist,
selectedItem.xtream_id,
start,
stop,
playlist.serverTimezone
)
: null;
});
if (
event.type === 'download-catchup' ||
event.type === 'copy-catchup-url'
) {
await activateXtreamArchiveAction(
event,
this.xtreamStore.currentPlaylist(),
selectedItem,
this.archiveDownloadsAvailable(),
{
copy: this.archiveCopy,
downloads: this.archiveDownloads,
urls: this.xtreamUrlService,
}
);
return;
}
if (event.type === 'live') {
@@ -795,11 +797,11 @@ export class LiveStreamLayoutComponent
return;
}
const startTimestamp = this.getProgramTimestampSeconds(
const startTimestamp = getProgramTimestampSeconds(
program.start,
program.startTimestamp
);
const stopTimestamp = this.getProgramTimestampSeconds(
const stopTimestamp = getProgramTimestampSeconds(
program.stop,
program.stopTimestamp
);
@@ -858,11 +860,11 @@ export class LiveStreamLayoutComponent
title: program.title,
desc: program.description ?? null,
category: null,
startTimestamp: this.getProgramTimestampSeconds(
startTimestamp: getProgramTimestampSeconds(
program.start,
program.start_timestamp
),
stopTimestamp: this.getProgramTimestampSeconds(
stopTimestamp: getProgramTimestampSeconds(
program.stop ?? program.end,
program.stop_timestamp
),
@@ -900,29 +902,11 @@ export class LiveStreamLayoutComponent
return this.runtime.supportsRemoteControl ? window.electron : undefined;
}
private getProgramTimestampSeconds(
dateValue: string,
unixTimestampValue?: number | string | null
): number | null {
const unixTimestamp = Number.parseInt(
String(unixTimestampValue ?? ''),
10
);
if (Number.isFinite(unixTimestamp) && unixTimestamp > 0) {
return unixTimestamp;
}
const parsedDate = Date.parse(dateValue);
return Number.isFinite(parsedDate)
? Math.floor(parsedDate / 1000)
: null;
}
private getProgramTimestampMilliseconds(
dateValue: string,
unixTimestampValue?: number | string | null
): number | null {
const unixTimestamp = this.getProgramTimestampSeconds(
const unixTimestamp = getProgramTimestampSeconds(
dateValue,
unixTimestampValue
);
@@ -1,3 +1,4 @@
import { EpgArchiveDownloadService } from '@iptvnator/ui/epg';
import { EpgArchiveCopyService } from '@iptvnator/ui/epg';
import { Component, Directive, input, output, signal } from '@angular/core';
import { ComponentFixture, TestBed } from '@angular/core/testing';
@@ -131,6 +132,10 @@ describe('LiveStreamLayoutComponent sidebar levels', () => {
await TestBed.configureTestingModule({
imports: [LiveStreamLayoutComponent, NoopAnimationsModule],
providers: [
{
provide: EpgArchiveDownloadService,
useValue: { start: jest.fn() },
},
{
provide: EpgArchiveCopyService,
useValue: { copy: jest.fn() },
@@ -0,0 +1,105 @@
import {
XtreamPlaylistData,
XtreamUrlService,
} from '@iptvnator/portal/xtream/data-access';
import { XTREAM_CLIENT_USER_AGENT } from '@iptvnator/shared/interfaces';
import {
EpgArchiveCopyService,
EpgArchiveDownloadService,
EpgProgramActivationEvent,
} from '@iptvnator/ui/epg';
import { XtreamLiveChannelItem } from './xtream-live-channel-navigation.service';
/** Archive actions capture their source without changing the playback session. */
export async function activateXtreamArchiveAction(
event: EpgProgramActivationEvent,
playlist: XtreamPlaylistData | null | undefined,
item: XtreamLiveChannelItem,
downloadsAvailable: boolean,
services: {
copy: EpgArchiveCopyService;
downloads: EpgArchiveDownloadService;
urls: XtreamUrlService;
}
): Promise<void> {
const start = getProgramTimestampSeconds(
event.program.start,
event.program.startTimestamp
);
const stop = getProgramTimestampSeconds(
event.program.stop,
event.program.stopTimestamp
);
const resolve = () =>
playlist && start && stop && stop > start
? services.urls.resolveCatchupUrl(
playlist.id,
playlist,
item.xtream_id,
start,
stop,
playlist.serverTimezone
)
: null;
if (event.type === 'copy-catchup-url') {
await services.copy.copy(resolve);
return;
}
if (
event.type !== 'download-catchup' ||
!downloadsAvailable ||
!playlist ||
!start ||
!stop ||
stop > Date.now() / 1000 ||
stop <= start
)
return;
const days = Number(item.tv_archive_duration ?? 0);
await services.downloads.start(
{
playlistId: playlist.id,
xtreamId: item.xtream_id,
playlistType: 'xtream',
serverUrl: playlist.serverUrl,
title: event.program.title,
posterUrl: item.poster_url ?? item.stream_icon ?? undefined,
catchup: {
channelName: item.title ?? item.name ?? '',
startTimestamp: start,
stopTimestamp: stop,
...(days > 0
? { expiresAt: Math.floor(start + days * 86400) }
: {}),
},
headers: {
userAgent:
playlist.userAgent?.trim() || XTREAM_CLIENT_USER_AGENT,
referer: playlist.referrer,
origin: playlist.origin,
},
},
() =>
services.urls.resolveCatchupUrl(
playlist.id,
playlist,
item.xtream_id,
start,
stop,
playlist.serverTimezone
)
);
}
export function getProgramTimestampSeconds(
dateValue: string,
unixTimestampValue?: number | string | null
): number | null {
const unixTimestamp = Number.parseInt(String(unixTimestampValue ?? ''), 10);
if (Number.isFinite(unixTimestamp) && unixTimestamp > 0) {
return unixTimestamp;
}
const parsedDate = Date.parse(dateValue);
return Number.isFinite(parsedDate) ? Math.floor(parsedDate / 1000) : null;
}
+4 -1
View File
@@ -1,4 +1,6 @@
import type {
CatchupDownloadMetadata,
DownloadContentType,
DownloadMetadataSnapshot,
ElectronBridgeDownloadStatus,
ElectronBridgeDownloadStartPayload,
@@ -16,7 +18,8 @@ export interface DownloadItem {
id: number;
playlistId: string;
xtreamId: number;
contentType: 'vod' | 'episode';
contentType: DownloadContentType;
catchup?: CatchupDownloadMetadata | null;
seriesXtreamId?: number;
seasonNumber?: number;
episodeNumber?: number;
+3 -4
View File
@@ -1,3 +1,4 @@
import type { DownloadRecoveryResult } from '@iptvnator/shared/interfaces';
import { computed, inject, Injectable, OnDestroy, signal } from '@angular/core';
import type { DownloadMetadataSnapshot } from '@iptvnator/shared/interfaces';
import type { ElectronBridgeDownloadStartResult } from '@iptvnator/shared/interfaces';
@@ -288,9 +289,7 @@ export class DownloadsService implements OnDestroy {
/**
* Remove a download from the list
*/
async removeDownload(
downloadId: number
): Promise<{ success: boolean; error?: string }> {
async removeDownload(downloadId: number): Promise<DownloadRecoveryResult> {
if (!this.isAvailable()) {
return { success: false, error: 'Downloads not available' };
}
@@ -379,7 +378,7 @@ export class DownloadsService implements OnDestroy {
/**
* Clear completed/failed downloads
*/
async clearCompleted(playlistId?: string): Promise<{ success: boolean }> {
async clearCompleted(playlistId?: string): Promise<DownloadRecoveryResult> {
if (!this.isAvailable()) {
return { success: false };
}
@@ -56,7 +56,7 @@ const completedMigrationStateRule: HandlerRule = [
/** The live title index, already carrying the folding tokenizer. */
const foldedIndexRule: HandlerRule = [
'SELECT sql FROM sqlite_master',
"name = 'content_title_fts'",
{
get: () => ({
sql: "CREATE VIRTUAL TABLE content_title_fts USING fts5(title, tokenize='trigram remove_diacritics 1')",
+6 -29
View File
@@ -1,3 +1,8 @@
import {
DOWNLOADS_TABLE_SQL,
DOWNLOADS_INDEX_STATEMENTS,
ensureDownloadsCatchupSchema,
} from './download-schema';
/**
* Database connection and initialization for IPTVnator
* Uses Drizzle ORM with better-sqlite3
@@ -90,35 +95,6 @@ const TMDB_METADATA_TABLE_SQL = `CREATE TABLE IF NOT EXISTS tmdb_metadata (
fetched_at TEXT DEFAULT (datetime('now'))
)`;
const TMDB_METADATA_INDEX_SQL = `CREATE UNIQUE INDEX IF NOT EXISTS tmdb_metadata_lookup_unique ON tmdb_metadata(media_type, lookup_key, language)`;
const DOWNLOADS_TABLE_SQL = `CREATE TABLE IF NOT EXISTS downloads (
id INTEGER PRIMARY KEY AUTOINCREMENT,
playlist_id TEXT NOT NULL,
xtream_id INTEGER NOT NULL,
content_type TEXT NOT NULL CHECK (content_type IN ('vod', 'episode')),
series_xtream_id INTEGER,
season_number INTEGER,
episode_number INTEGER,
episode_identity_scope TEXT,
title TEXT NOT NULL,
url TEXT NOT NULL,
file_name TEXT,
file_path TEXT,
poster_url TEXT,
request_headers TEXT,
resume_validator TEXT,
metadata_snapshot TEXT,
status TEXT NOT NULL DEFAULT 'queued' CHECK (status IN ('queued', 'downloading', 'paused', 'completed', 'failed', 'canceled')),
bytes_downloaded INTEGER DEFAULT 0,
total_bytes INTEGER,
error_message TEXT,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
)`;
const DOWNLOADS_INDEX_STATEMENTS = [
`CREATE UNIQUE INDEX IF NOT EXISTS downloads_xtream_playlist_unique ON downloads(xtream_id, playlist_id, content_type)`,
`CREATE INDEX IF NOT EXISTS downloads_playlist_idx ON downloads(playlist_id)`,
`CREATE INDEX IF NOT EXISTS downloads_status_idx ON downloads(status)`,
];
// No unique index: re-recording the same channel is a normal workflow, and
// playlist_id carries no FK so recordings survive source deletion.
const RECORDINGS_TABLE_SQL = `CREATE TABLE IF NOT EXISTS recordings (
@@ -1165,6 +1141,7 @@ function runMigrations(sqliteDb: Database.Database): void {
cleanupLegacyTmdbSearchCache(sqliteDb);
ensureDownloadsPauseResumeSchema(sqliteDb);
runMigrationStatements(sqliteDb, COLUMN_MIGRATION_STATEMENTS);
ensureDownloadsCatchupSchema(sqliteDb);
// The tokenizer upgrade recreates and rebuilds the index itself, so the
// plain rebuild below would only repeat work it just did.
if (!upgradeContentTitleFtsTokenizer(sqliteDb)) {
@@ -0,0 +1,65 @@
import { execFileSync } from 'node:child_process';
import { createRequire } from 'node:module';
import { resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
it('migrates existing downloads without losing files and separates programme identities', () => {
const electron = createRequire(__filename)('electron') as string;
const moduleUrl = pathToFileURL(
resolve(__dirname, 'download-schema.ts')
).href;
const result = execFileSync(
electron,
[
'--import',
'tsx',
'--eval',
`
const { default: Database } = await import('better-sqlite3');
const { DOWNLOADS_TABLE_SQL, ensureDownloadsCatchupSchema } = await import(${JSON.stringify(moduleUrl)});
const db = new Database(':memory:');
db.pragma('foreign_keys = ON');
db.exec(DOWNLOADS_TABLE_SQL.replace(", 'catchup'", '').replace('programme_start INTEGER NOT NULL DEFAULT 0,', '').replace('catchup TEXT,', ''));
db.exec("CREATE UNIQUE INDEX downloads_xtream_playlist_unique ON downloads(xtream_id, playlist_id, content_type)");
db.prepare("INSERT INTO downloads (id,playlist_id,xtream_id,content_type,title,url,status,file_path,bytes_downloaded,resume_validator,request_headers) VALUES (42,'p',1,'vod','Movie','https://host/movie','paused','/safe/movie.mp4',123,'etag','headers')").run();
ensureDownloadsCatchupSchema(db);
ensureDownloadsCatchupSchema(db);
const insert = db.prepare("INSERT INTO downloads (playlist_id,xtream_id,content_type,programme_start,title,url) VALUES ('p',1,?,?, 'Show','https://host/archive')");
insert.run('catchup', 1000); insert.run('catchup', 2000);
let duplicateArchive = false, duplicateMovie = false;
try { insert.run('catchup', 1000); } catch { duplicateArchive = true; }
try { insert.run('vod', 0); } catch { duplicateMovie = true; }
db.prepare("INSERT INTO downloads (id,playlist_id,xtream_id,content_type,title,url) VALUES (999,'p',99,'catchup','Proof','https://host/archive')").run();
db.prepare("INSERT INTO download_archive_finalizations (download_id,proof) VALUES (999,'{}')").run();
db.prepare('DELETE FROM downloads WHERE id=999').run();
const journalCount = db.prepare('SELECT count(*) AS n FROM download_archive_finalizations').get().n;
process.stdout.write(JSON.stringify({journalCount,row:db.prepare('SELECT * FROM downloads WHERE id=42').get(), count:db.prepare('SELECT count(*) AS n FROM downloads').get().n, duplicateArchive,duplicateMovie}));
`,
],
{
cwd: process.cwd(),
encoding: 'utf8',
env: {
...process.env,
ELECTRON_RUN_AS_NODE: '1',
TSX_TSCONFIG_PATH: resolve(process.cwd(), 'tsconfig.base.json'),
},
}
);
expect(JSON.parse(result)).toMatchObject({
count: 3,
journalCount: 0,
duplicateArchive: true,
duplicateMovie: true,
row: {
id: 42,
status: 'paused',
file_path: '/safe/movie.mp4',
bytes_downloaded: 123,
resume_validator: 'etag',
request_headers: 'headers',
programme_start: 0,
catchup: null,
},
});
});
@@ -0,0 +1,94 @@
import type Database from 'better-sqlite3';
export const DOWNLOADS_TABLE_SQL = `CREATE TABLE IF NOT EXISTS downloads (
id INTEGER PRIMARY KEY AUTOINCREMENT,
playlist_id TEXT NOT NULL,
xtream_id INTEGER NOT NULL,
content_type TEXT NOT NULL CHECK (content_type IN ('vod', 'episode', 'catchup')),
programme_start INTEGER NOT NULL DEFAULT 0,
catchup TEXT,
series_xtream_id INTEGER,
season_number INTEGER,
episode_number INTEGER,
episode_identity_scope TEXT,
title TEXT NOT NULL,
url TEXT NOT NULL,
file_name TEXT,
file_path TEXT,
poster_url TEXT,
request_headers TEXT,
resume_validator TEXT,
metadata_snapshot TEXT,
status TEXT NOT NULL DEFAULT 'queued' CHECK (status IN ('queued', 'downloading', 'paused', 'completed', 'failed', 'canceled')),
bytes_downloaded INTEGER DEFAULT 0,
total_bytes INTEGER,
error_message TEXT,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
)`;
export const DOWNLOADS_INDEX_STATEMENTS = [
`CREATE UNIQUE INDEX IF NOT EXISTS downloads_xtream_playlist_unique ON downloads(xtream_id, playlist_id, content_type) WHERE content_type != 'catchup'`,
`CREATE INDEX IF NOT EXISTS downloads_playlist_idx ON downloads(playlist_id)`,
`CREATE INDEX IF NOT EXISTS downloads_status_idx ON downloads(status)`,
];
const ARCHIVE_FINALIZATIONS_SQL = `CREATE TABLE IF NOT EXISTS download_archive_finalizations (
download_id INTEGER PRIMARY KEY REFERENCES downloads(id) ON DELETE CASCADE,
proof TEXT NOT NULL
)`;
const CATCHUP_INDEX = `CREATE UNIQUE INDEX IF NOT EXISTS downloads_catchup_unique
ON downloads(xtream_id, playlist_id, programme_start) WHERE content_type = 'catchup'`;
/** Widen the CHECK transactionally; existing ids, files and resume state survive. */
export function ensureDownloadsCatchupSchema(db: Database.Database): void {
const row = db
.prepare(
"SELECT sql FROM sqlite_master WHERE type = 'table' AND name = 'downloads'"
)
.get() as { sql: string } | undefined;
if (!row) return;
if (row.sql.includes("'catchup'")) {
db.exec(CATCHUP_INDEX);
db.exec(ARCHIVE_FINALIZATIONS_SQL);
return;
}
const columns = [
'id',
'playlist_id',
'xtream_id',
'content_type',
'series_xtream_id',
'season_number',
'episode_number',
'episode_identity_scope',
'title',
'url',
'file_name',
'file_path',
'poster_url',
'request_headers',
'resume_validator',
'metadata_snapshot',
'status',
'bytes_downloaded',
'total_bytes',
'error_message',
'created_at',
'updated_at',
];
db.transaction(() => {
db.exec('DROP INDEX IF EXISTS downloads_xtream_playlist_unique');
db.exec('DROP INDEX IF EXISTS downloads_playlist_idx');
db.exec('DROP INDEX IF EXISTS downloads_status_idx');
db.exec('ALTER TABLE downloads RENAME TO downloads_catchup_legacy');
db.exec(DOWNLOADS_TABLE_SQL);
db.exec(
`INSERT INTO downloads (${columns.join(',')}) SELECT ${columns.join(',')} FROM downloads_catchup_legacy`
);
db.exec('DROP TABLE downloads_catchup_legacy');
for (const statement of DOWNLOADS_INDEX_STATEMENTS) db.exec(statement);
db.exec(CATCHUP_INDEX);
db.exec(ARCHIVE_FINALIZATIONS_SQL);
})();
}
@@ -0,0 +1,84 @@
import type { CatchupDownloadMetadata } from '@iptvnator/shared/interfaces';
import { sql } from 'drizzle-orm';
import {
index,
integer,
sqliteTable,
text,
uniqueIndex,
} from 'drizzle-orm/sqlite-core';
// Downloads table
export const downloads = sqliteTable(
'downloads',
{
id: integer('id').primaryKey({ autoIncrement: true }),
playlistId: text('playlist_id').notNull(),
// Content identifiers
xtreamId: integer('xtream_id').notNull(),
contentType: text('content_type', {
enum: ['vod', 'episode', 'catchup'],
}).notNull(),
programmeStart: integer('programme_start').notNull().default(0),
catchup: text('catchup', {
mode: 'json',
}).$type<CatchupDownloadMetadata>(),
// For episodes: store series info
seriesXtreamId: integer('series_xtream_id'),
seasonNumber: integer('season_number'),
episodeNumber: integer('episode_number'),
episodeIdentityScope: text('episode_identity_scope'),
// Download metadata
title: text('title').notNull(),
url: text('url').notNull(),
fileName: text('file_name'),
filePath: text('file_path'),
posterUrl: text('poster_url'),
requestHeaders: text('request_headers'),
resumeValidator: text('resume_validator'),
metadataSnapshot: text('metadata_snapshot'),
// Download progress
status: text('status', {
enum: [
'queued',
'downloading',
'paused',
'completed',
'failed',
'canceled',
],
})
.notNull()
.default('queued'),
bytesDownloaded: integer('bytes_downloaded').default(0),
totalBytes: integer('total_bytes'),
errorMessage: text('error_message'),
// Timestamps
createdAt: text('created_at').default(sql`CURRENT_TIMESTAMP`),
updatedAt: text('updated_at').default(sql`CURRENT_TIMESTAMP`),
},
(table) => ({
playlistIdx: index('downloads_playlist_idx').on(table.playlistId),
statusIdx: index('downloads_status_idx').on(table.status),
xtreamPlaylistUnique: uniqueIndex('downloads_xtream_playlist_unique')
.on(table.xtreamId, table.playlistId, table.contentType)
.where(sql`${table.contentType} != 'catchup'`),
catchupUnique: uniqueIndex('downloads_catchup_unique')
.on(table.xtreamId, table.playlistId, table.programmeStart)
.where(sql`${table.contentType} = 'catchup'`),
})
);
export type Download = typeof downloads.$inferSelect;
export type NewDownload = typeof downloads.$inferInsert;
// Write-ahead archive promotion proof. It outlives process-local DownloadTask.
export const downloadArchiveFinalizations = sqliteTable(
'download_archive_finalizations',
{
downloadId: integer('download_id')
.primaryKey()
.references(() => downloads.id, { onDelete: 'cascade' }),
proof: text('proof').notNull(),
}
);
+1 -56
View File
@@ -335,62 +335,7 @@ export type NewEpgProgramDb = typeof epgPrograms.$inferInsert;
export type PlaybackPosition = typeof playbackPositions.$inferSelect;
export type NewPlaybackPosition = typeof playbackPositions.$inferInsert;
// Downloads table
export const downloads = sqliteTable(
'downloads',
{
id: integer('id').primaryKey({ autoIncrement: true }),
playlistId: text('playlist_id').notNull(),
// Content identifiers
xtreamId: integer('xtream_id').notNull(),
contentType: text('content_type', {
enum: ['vod', 'episode'],
}).notNull(),
// For episodes: store series info
seriesXtreamId: integer('series_xtream_id'),
seasonNumber: integer('season_number'),
episodeNumber: integer('episode_number'),
episodeIdentityScope: text('episode_identity_scope'),
// Download metadata
title: text('title').notNull(),
url: text('url').notNull(),
fileName: text('file_name'),
filePath: text('file_path'),
posterUrl: text('poster_url'),
requestHeaders: text('request_headers'),
resumeValidator: text('resume_validator'),
metadataSnapshot: text('metadata_snapshot'),
// Download progress
status: text('status', {
enum: [
'queued',
'downloading',
'paused',
'completed',
'failed',
'canceled',
],
})
.notNull()
.default('queued'),
bytesDownloaded: integer('bytes_downloaded').default(0),
totalBytes: integer('total_bytes'),
errorMessage: text('error_message'),
// Timestamps
createdAt: text('created_at').default(sql`CURRENT_TIMESTAMP`),
updatedAt: text('updated_at').default(sql`CURRENT_TIMESTAMP`),
},
(table) => ({
playlistIdx: index('downloads_playlist_idx').on(table.playlistId),
statusIdx: index('downloads_status_idx').on(table.status),
xtreamPlaylistUnique: uniqueIndex(
'downloads_xtream_playlist_unique'
).on(table.xtreamId, table.playlistId, table.contentType),
})
);
export type Download = typeof downloads.$inferSelect;
export type NewDownload = typeof downloads.$inferInsert;
export * from './download-tables';
// Live-TV recordings table. Rows are created by the embedded-MPV recording
// tracker, not by the download queue: a recording has no source URL to
Loaded 100 of 117 files, more files were not shown because too many files have changed in this diff. Show more