fix(playlist): bound playlist downloads and isolate startup auto-update (#1233)

Playlist downloads in the main process ran without an axios timeout, so a
host that accepted the connection and then went silent kept the request
pending forever. The startup auto-update refreshed playlists sequentially,
so that one unresponsive source also withheld every playlist that did
refresh, and the URL import dialog could spin with no way out.

- Add PLAYLIST_FETCH_TIMEOUT_MS (30s) to the main-process fetch and reuse it
  in the refresh worker instead of its duplicated literal. Redirects are
  followed one hop at a time, so each hop is bounded separately.
- Move auto-update into playlist-auto-update.ts and refresh at most
  AUTO_UPDATE_CONCURRENCY (3) playlists at once, isolating each failure while
  preserving the requested order. An unbounded fan-out would download and
  parse arbitrarily many large M3U files in the main process at once.
- Redact refresh log URLs, which routinely carry Xtream username/password
  query parameters.

The existing auto-update spec handed out fixtures by call order, which no
longer holds once refreshes overlap; it now keys them by source type.

Fixes #931

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
4grayandClaude Opus 5 authored and GitHub committed 2026-07-25 10:37:11 +02:00
1 parent b4c0cce741
commit fc81f4f941
7 files changed
+422 -73

No files matched your search

@@ -0,0 +1,255 @@
import type { Playlist } from '@iptvnator/shared/interfaces';
import {
AUTO_UPDATE_CONCURRENCY,
autoUpdatePlaylists,
} from './playlist-auto-update';
const mockFetchPlaylistFromFile = jest.fn();
const mockFetchPlaylistFromUrl = jest.fn();
jest.mock('./playlist-source', () => ({
...jest.requireActual('./playlist-source'),
fetchPlaylistFromFile: (...args: unknown[]) =>
mockFetchPlaylistFromFile(...args),
fetchPlaylistFromUrl: (...args: unknown[]) =>
mockFetchPlaylistFromUrl(...args),
}));
function createPlaylist(overrides: Partial<Playlist> = {}): Playlist {
return {
_id: 'playlist-1',
autoRefresh: true,
count: 1,
favorites: [],
filename: 'Playlist',
importDate: '2026-06-02T00:00:00.000Z',
lastUsage: '2026-06-02T00:00:00.000Z',
playlist: { items: [] },
title: 'Playlist',
...overrides,
};
}
function createUrlPlaylist(id: string, url: string): Playlist {
return createPlaylist({ _id: id, title: id, url });
}
/** Lets every pending promise chain settle without advancing timers. */
function flushMicrotasks(): Promise<void> {
return new Promise((resolve) => setImmediate(resolve));
}
/** A promise that never settles on its own, plus its resolver. */
function createPendingFetch(): {
promise: Promise<Playlist>;
resolve: (playlist: Playlist) => void;
} {
let resolve!: (playlist: Playlist) => void;
const promise = new Promise<Playlist>((resolvePromise) => {
resolve = resolvePromise;
});
return { promise, resolve };
}
describe('autoUpdatePlaylists', () => {
let consoleErrorSpy: jest.SpyInstance;
let consoleLogSpy: jest.SpyInstance;
let consoleWarnSpy: jest.SpyInstance;
beforeEach(() => {
mockFetchPlaylistFromFile.mockReset();
mockFetchPlaylistFromUrl.mockReset();
consoleErrorSpy = jest.spyOn(console, 'error').mockImplementation();
consoleLogSpy = jest.spyOn(console, 'log').mockImplementation();
consoleWarnSpy = jest.spyOn(console, 'warn').mockImplementation();
});
afterEach(() => {
consoleErrorSpy.mockRestore();
consoleLogSpy.mockRestore();
consoleWarnSpy.mockRestore();
});
it('refreshes the remaining playlists while an unresponsive source is still pending', async () => {
const unresponsive = createPendingFetch();
mockFetchPlaylistFromUrl.mockImplementation((url: string) =>
url === 'https://unresponsive.test/list.m3u'
? unresponsive.promise
: Promise.resolve(createPlaylist({ title: 'Refreshed' }))
);
const updatePromise = autoUpdatePlaylists([
createUrlPlaylist(
'unresponsive',
'https://unresponsive.test/list.m3u'
),
createUrlPlaylist('reachable', 'https://reachable.test/list.m3u'),
]);
await flushMicrotasks();
expect(mockFetchPlaylistFromUrl).toHaveBeenCalledWith(
'https://reachable.test/list.m3u',
'reachable',
{}
);
unresponsive.resolve(createPlaylist({ title: 'Slow but refreshed' }));
await expect(updatePromise).resolves.toEqual([
expect.objectContaining({ _id: 'unresponsive' }),
expect.objectContaining({ _id: 'reachable' }),
]);
});
it('keeps the playlists that succeeded when another source fails', async () => {
mockFetchPlaylistFromUrl.mockImplementation((url: string) =>
url === 'https://broken.test/list.m3u'
? Promise.reject(new Error('timeout of 30000ms exceeded'))
: Promise.resolve(createPlaylist({ title: 'Refreshed' }))
);
const result = await autoUpdatePlaylists([
createUrlPlaylist('broken', 'https://broken.test/list.m3u'),
createUrlPlaylist('reachable', 'https://reachable.test/list.m3u'),
]);
expect(result).toEqual([expect.objectContaining({ _id: 'reachable' })]);
expect(consoleErrorSpy).toHaveBeenCalledWith(
'Failed to update playlist "broken":',
expect.objectContaining({
message: 'timeout of 30000ms exceeded',
})
);
});
it('never runs more refreshes at once than the configured concurrency', async () => {
const pendingFetches = Array.from(
{ length: AUTO_UPDATE_CONCURRENCY + 2 },
createPendingFetch
);
let startedFetches = 0;
mockFetchPlaylistFromUrl.mockImplementation(
() => pendingFetches[startedFetches++].promise
);
const updatePromise = autoUpdatePlaylists(
pendingFetches.map((_, index) =>
createUrlPlaylist(
`playlist-${index}`,
`https://example.test/${index}.m3u`
)
)
);
await flushMicrotasks();
expect(startedFetches).toBe(AUTO_UPDATE_CONCURRENCY);
pendingFetches[0].resolve(createPlaylist());
await flushMicrotasks();
expect(startedFetches).toBe(AUTO_UPDATE_CONCURRENCY + 1);
for (const pendingFetch of pendingFetches.slice(1)) {
pendingFetch.resolve(createPlaylist());
}
await expect(updatePromise).resolves.toHaveLength(
pendingFetches.length
);
});
it('preserves user-owned fields and refreshes file-based playlists', async () => {
mockFetchPlaylistFromFile.mockResolvedValue(
createPlaylist({
_id: 'regenerated',
autoRefresh: false,
favorites: [],
title: 'Refreshed from file',
userAgent: undefined,
})
);
const result = await autoUpdatePlaylists([
createPlaylist({
_id: 'file-playlist',
autoRefresh: true,
favorites: ['fav-channel'],
filePath: '/playlists/local.m3u',
importDate: '',
title: 'File playlist',
userAgent: 'PlaylistAgent/1.0',
}),
]);
expect(mockFetchPlaylistFromFile).toHaveBeenCalledWith(
'/playlists/local.m3u',
'File playlist'
);
expect(result).toEqual([
expect.objectContaining({
_id: 'file-playlist',
autoRefresh: true,
favorites: ['fav-channel'],
title: 'Refreshed from file',
userAgent: 'PlaylistAgent/1.0',
}),
]);
});
it('skips playlists without a usable source and forwards trust options', async () => {
mockFetchPlaylistFromUrl.mockResolvedValue(createPlaylist());
const result = await autoUpdatePlaylists(
[
createPlaylist({
_id: 'no-source',
filePath: undefined,
title: 'No source',
url: undefined,
}),
createUrlPlaylist(
'reachable',
'https://reachable.test/list.m3u'
),
],
{ trustedInsecureTlsHosts: ['self-signed.test'] }
);
expect(result).toHaveLength(1);
expect(consoleWarnSpy).toHaveBeenCalledWith(
'Skipping playlist "No source": no URL or file path found'
);
expect(mockFetchPlaylistFromUrl).toHaveBeenCalledWith(
'https://reachable.test/list.m3u',
'reachable',
{ trustedInsecureTlsHosts: ['self-signed.test'] }
);
});
it('redacts playlist credentials from the refresh log', async () => {
mockFetchPlaylistFromUrl.mockResolvedValue(createPlaylist());
await autoUpdatePlaylists([
createUrlPlaylist(
'xtream',
'https://portal.test/get.php?username=alice&password=hunter2'
),
]);
const loggedMessages = consoleLogSpy.mock.calls.map(([message]) =>
String(message)
);
expect(
loggedMessages.some((message) => message.includes('hunter2'))
).toBe(false);
expect(
loggedMessages.some((message) => message.includes('alice'))
).toBe(false);
expect(
loggedMessages.some((message) =>
message.includes('Updating playlist "xtream" from URL:')
)
).toBe(true);
});
});
@@ -0,0 +1,97 @@
import type { Playlist } from '@iptvnator/shared/interfaces';
import { redactSensitiveData } from '@iptvnator/shared/logging';
import {
fetchPlaylistFromFile,
fetchPlaylistFromUrl,
preserveAutoUpdatedPlaylistFields,
type PlaylistFetchOptions,
} from './playlist-source';
/**
* How many playlists the startup auto-update refreshes at the same time.
* Refreshing sequentially let a single slow or unreachable host delay every
* remaining playlist (issue #931), while an unbounded fan-out would download
* and parse arbitrarily many large M3U files in the main process at once.
*/
export const AUTO_UPDATE_CONCURRENCY = 3;
/**
* Refreshes a single playlist from its original source.
* Returns `null` when the playlist carries neither a URL nor a file path.
*/
async function refreshPlaylist(
playlist: Playlist,
options: PlaylistFetchOptions
): Promise<Playlist | null> {
let playlistObject: Playlist;
if (playlist.importDate && playlist.url) {
console.log(
`Updating playlist "${playlist.title}" from URL: ${redactSensitiveData(playlist.url)}`
);
playlistObject = await fetchPlaylistFromUrl(
playlist.url,
playlist.title,
options
);
} else if (playlist.filePath) {
console.log(
`Updating playlist "${playlist.title}" from file: ${playlist.filePath}`
);
playlistObject = await fetchPlaylistFromFile(
playlist.filePath,
playlist.title
);
} else {
console.warn(
`Skipping playlist "${playlist.title}": no URL or file path found`
);
return null;
}
console.log(`Successfully updated playlist "${playlist.title}"`);
return preserveAutoUpdatedPlaylistFields(playlistObject, playlist);
}
/**
* Refreshes the given playlists with bounded concurrency, isolating every
* failure so one broken source never withholds the playlists that did update.
* The returned playlists keep the order of the requested ones.
*/
export async function autoUpdatePlaylists(
playlists: readonly Playlist[],
options: PlaylistFetchOptions = {}
): Promise<Playlist[]> {
const results: (Playlist | null)[] = new Array(playlists.length).fill(null);
let nextIndex = 0;
const refreshNextPlaylist = async (): Promise<void> => {
for (
let index = nextIndex++;
index < playlists.length;
index = nextIndex++
) {
const playlist = playlists[index];
try {
results[index] = await refreshPlaylist(playlist, options);
} catch (error) {
console.error(
`Failed to update playlist "${playlist.title}":`,
error
);
}
}
};
await Promise.all(
Array.from(
{ length: Math.min(AUTO_UPDATE_CONCURRENCY, playlists.length) },
refreshNextPlaylist
)
);
return results.filter(
(playlist): playlist is Playlist => playlist !== null
);
}
@@ -18,6 +18,14 @@ export interface PlaylistFetchOptions {
trustedInsecureTlsHosts?: readonly string[];
}
/**
* Idle timeout for every playlist download hop. Without it a host that accepts
* the connection and then goes silent keeps the request pending forever, which
* stalled the unattended startup auto-update indefinitely (issue #931).
* Mirrored by the playlist refresh worker.
*/
export const PLAYLIST_FETCH_TIMEOUT_MS = 30000;
export async function fetchPlaylistFromUrl(
url: string,
title?: string,
@@ -32,6 +40,7 @@ export async function fetchPlaylistFromUrl(
trustedInsecureTlsHosts: options.trustedInsecureTlsHosts,
}),
method: 'GET',
timeout: PLAYLIST_FETCH_TIMEOUT_MS,
},
{ allowPrivateNetworks: true }
);
@@ -195,6 +195,7 @@ describe('playlist IPC events', () => {
httpsAgent: expect.any(Object),
maxRedirects: 0,
method: 'GET',
timeout: 30000,
url: 'https://example.test/remote.m3u',
})
);
@@ -288,24 +289,25 @@ describe('playlist IPC events', () => {
mockReadFile.mockResolvedValue('#EXTM3U file');
mockParse.mockReturnValue(parsedPlaylist);
mockGetFilenameFromUrl.mockReturnValue('list.m3u');
mockCreatePlaylistObject
.mockReturnValueOnce(
createPlaylist({
_id: 'new-url-playlist',
autoRefresh: false,
favorites: [],
title: 'Updated URL playlist',
url: 'https://example.test/list.m3u',
})
)
.mockReturnValueOnce(
createPlaylist({
_id: 'new-file-playlist',
autoRefresh: true,
filePath: '/playlists/local.m3u',
title: 'Updated file playlist',
})
);
// Auto-update refreshes playlists concurrently, so the fixtures are
// keyed by source type instead of by call order.
mockCreatePlaylistObject.mockImplementation(
(_title, _parsed, _source, type) =>
type === 'URL'
? createPlaylist({
_id: 'new-url-playlist',
autoRefresh: false,
favorites: [],
title: 'Updated URL playlist',
url: 'https://example.test/list.m3u',
})
: createPlaylist({
_id: 'new-file-playlist',
autoRefresh: true,
filePath: '/playlists/local.m3u',
title: 'Updated file playlist',
})
);
const result = await getHandler(AUTO_UPDATE_PLAYLISTS)(
createIpcEvent(),
@@ -332,6 +334,7 @@ describe('playlist IPC events', () => {
httpsAgent: expect.any(Object),
maxRedirects: 0,
method: 'GET',
timeout: 30000,
url: 'https://example.test/list.m3u',
})
);
@@ -22,11 +22,11 @@ import type {
PlaylistRefreshWorkerMessage,
PlaylistRefreshWorkerResponseMessage,
} from '../workers/playlist-refresh.worker.types';
import { autoUpdatePlaylists } from './playlist-auto-update';
import {
derivePlaylistTitleFromFilePath,
fetchPlaylistFromFile,
fetchPlaylistFromUrl,
preserveAutoUpdatedPlaylistFields,
} from './playlist-source';
import { PlaylistWriteAuthorizer } from './playlist-write-authorization';
@@ -152,56 +152,9 @@ ipcMain.handle(
) => {
console.log(`Auto-updating ${playlists.length} playlist(s)...`);
const updatedPlaylists: Playlist[] = [];
for (const playlist of playlists) {
try {
let playlistObject;
if (playlist.importDate && playlist.url) {
// Update from URL
console.log(
`Updating playlist "${playlist.title}" from URL: ${playlist.url}`
);
playlistObject = await fetchPlaylistFromUrl(
playlist.url,
playlist.title,
{
trustedInsecureTlsHosts:
options?.trustedInsecureTlsHosts,
}
);
} else if (playlist.filePath) {
// Update from file path
console.log(
`Updating playlist "${playlist.title}" from file: ${playlist.filePath}`
);
playlistObject = await fetchPlaylistFromFile(
playlist.filePath,
playlist.title
);
} else {
console.warn(
`Skipping playlist "${playlist.title}": no URL or file path found`
);
continue;
}
updatedPlaylists.push(
preserveAutoUpdatedPlaylistFields(playlistObject, playlist)
);
console.log(
`Successfully updated playlist "${playlist.title}"`
);
} catch (error) {
console.error(
`Failed to update playlist "${playlist.title}":`,
error
);
// Continue with other playlists even if one fails
}
}
const updatedPlaylists = await autoUpdatePlaylists(playlists, {
trustedInsecureTlsHosts: options?.trustedInsecureTlsHosts,
});
console.log(
`Auto-update completed: ${updatedPlaylists.length} updated`
@@ -21,6 +21,7 @@ import {
isInvalidTlsCertificateError,
} from '../util/security-errors';
import { requestWithValidatedRedirects } from '../util/validated-axios';
import { PLAYLIST_FETCH_TIMEOUT_MS } from '../events/playlist-source';
type ActiveRefreshState = {
cancelled: boolean;
@@ -97,7 +98,7 @@ async function fetchPlaylistFromUrl(
}),
method: 'GET',
signal: controller.signal,
timeout: 30000,
timeout: PLAYLIST_FETCH_TIMEOUT_MS,
},
{ allowPrivateNetworks: true }
);
+34 -3
View File
@@ -55,6 +55,37 @@ There is intentionally **no URL validation** (upstream removed it in 0.15.0): an
The behavioral contract is guarded by `apps/web/src/app/iptv-playlist-parser.contract.spec.ts` (jest maps the module to the real parser source) and by the fork's own test suite.
## Playlist Refresh And Startup Auto-Update (Electron)
Two paths re-download an M3U playlist from its original source:
- **Explicit refresh** — `PLAYLIST_REFRESH` runs in `playlist-refresh.worker.ts`, reports
progress through `PLAYLIST_REFRESH_EVENT`, and is cancellable via
`PLAYLIST_CANCEL_REFRESH`.
- **Startup auto-update** — after `loadPlaylistsSuccess`, `AppComponent` sends
`AUTO_UPDATE_PLAYLISTS` for every playlist with `autoRefresh === true`. The main
process fulfils it in `playlist-auto-update.ts` on top of `playlist-source.ts`.
Both share one download contract, because this path is unattended and a hostile or
dead source must never stall startup (issue #931):
- Every HTTP hop uses the idle timeout `PLAYLIST_FETCH_TIMEOUT_MS` (30s, exported by
`playlist-source.ts`). Without it a host that accepts the connection and then goes
silent keeps the request pending forever. Redirects are followed one hop at a time,
so each hop is bounded separately.
- Auto-update refreshes at most `AUTO_UPDATE_CONCURRENCY` (3) playlists at a time.
Sequential refreshes let one slow host delay every remaining playlist; an unbounded
fan-out would download and parse arbitrarily many large M3U files in the main
process at once.
- Each playlist's failure is isolated and logged; the successful ones are still
returned, in the order they were requested. Playlists with neither a URL nor a file
path are skipped with a warning.
- `preserveAutoUpdatedPlaylistFields()` re-applies the user-owned fields (`_id`,
`autoRefresh`, `favorites`, `userAgent`) onto the freshly parsed playlist.
- Playlist URLs frequently carry Xtream-style `username`/`password` query parameters,
so refresh logging goes through `redactSensitiveData()` from
`@iptvnator/shared/logging`.
## State Management (libs/m3u-state/)
### State Structure
@@ -283,7 +314,7 @@ per-tab EPG logic:
The programme guide under the player renders in one of **two interchangeable
views**, chosen by the **`epgViewMode`** setting (`'timeline'` default, or
`'list'`; Settings → EPG → *Guide view*):
`'list'`; Settings → EPG → _Guide view_):
- **Timeline** — a horizontal **ribbon** (`app-epg-timeline`,
`libs/ui/epg/src/lib/epg-timeline/`).
@@ -679,7 +710,7 @@ class EpgService {
### DASH + ClearKey Playback
MPEG-DASH (`.mpd`) channels play through a Shaka Player *source engine* inside
MPEG-DASH (`.mpd`) channels play through a Shaka Player _source engine_ inside
the existing built-in players — exactly like hls.js/mpegts.js. There is no new
player in settings.
@@ -687,7 +718,7 @@ player in settings.
1. The playlist parser fork does not understand `#KODIPROP:` lines, but keeps
every unknown line between `#EXTINF` and the stream URL in `item.raw` (the
dominant Kodi/TiviMate layout; `#KODIPROP` lines *before* `#EXTINF` are
dominant Kodi/TiviMate layout; `#KODIPROP` lines _before_ `#EXTINF` are
dropped by the parser — fixing that requires a parser-fork patch and is
deferred).
2. `extractDrmFromRaw()` (`libs/shared/m3u-utils/src/lib/kodiprop.utils.ts`)