mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-08 17:06:15 -08:00
feat(tmdb): metadata cache panel with a clear button in settings (#1244)
* feat(tmdb): metadata cache panel with a clear button in settings Adds "Metadata cache — N entries · X MB" with a Clear button to Settings > Metadata (TMDB), next to the API key it belongs to. Three things it is good for: dropping stale or wrong metadata so the next open refetches it, seeing what the cache actually costs on disk, and reclaiming rows that a lookup-key version bump has orphaned — a bump makes rows unreachable, not deleted, so nothing else would ever collect them. Sizing the cache is a full table scan (LENGTH() on TEXT counts characters, so the SUM casts to BLOB to get bytes), which is why stats load lazily and only once the TMDB section is the active one rather than on every settings open. Clearing is always safe: enrichment refetches on demand, so the only cost is the next few requests. Works in both environments — the PWA has no bridge, so the service reports and clears its session-scoped in-memory map instead. i18n: 4 keys across all 19 locales via the tools/i18n workflow; placeholder integrity verified. Contract fixtures updated for both the preload bridge and the DB-worker payload shapes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tmdb): make cache clearing durable and stop reporting failures as empty Four review findings, all real: - A metadata write already in flight when the user cleared would land afterwards and silently restore what they removed. Writes now carry the generation they started in; a write that outlives a clear is dropped (PWA) or undone (Electron). - The PWA byte count used String.length, i.e. UTF-16 code units, so localized payloads under-reported and disagreed with the SQLite BLOB byte count. TextEncoder now measures actual bytes. - A failed stats read returned a valid zero-entry result, so the panel claimed an empty cache and disabled Clear while rows were still there. getStats/clear now return null on failure and the panel says so instead of inventing state. - No behavioural coverage existed for either side. Tests: SQL ops (entry/byte reporting, empty table, missing row, delete count) and the service (encoded bytes, clear count, and a write racing a clear). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tmdb): make the cache clear precise and version skew visible Review follow-ups on the cache panel: - A write that was in flight when the user cleared used to trigger a second full-table clear once it landed, which also deleted anything written in between. clear() now waits for the writes issued before it and lets the single clear take them; later writes survive. - An Electron shell without the maintenance ops fell through to the renderer map, which is always empty there — it reported an empty cache and disabled the Clear button while SQLite was full. Both operations now report unsupported instead. - Component coverage for the panel (deferred scan, clear + re-read, failed clear, failed read) and Electron-path service coverage. - The canonical IPC and settings sections of the enrichment doc, plus the matching CLAUDE.md lines, now list the maintenance ops. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tmdb): drop Promise.allSettled from the cache clear The web target compiles against lib es2018, so allSettled broke the Windows frontend build (TS2550). The pending writes swallow their own errors, so a plain Promise.all over neutralized promises does the job. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tmdb): keep a synchronous bridge throw inside the cache write Moving the write into a tracked promise dropped the try/catch that used to cover the call itself, so a bridge that threw synchronously would escape set(). Wrap it in an async IIFE, which turns that back into a rejection the same handler swallows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tmdb): retry the cache size read when the section is reopened The effect skipped the read once cacheError was set, so one transient IPC failure left the panel showing "could not read the cache" for the life of the settings page — and the only enabled control that could shift it was the destructive Clear button. Gate on the stats signal alone: reopening the section retries. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tmdb): queue writes that start while the cache is being cleared Awaiting the in-flight writes closed one side of the race and left the other open: a set() that started during that wait dispatched its IPC immediately, was absent from the snapshot, and could reach SQLite just before the delete — so a row written after the user clicked Clear was removed anyway. clear() now holds its own promise for the whole operation and set() waits on it, which puts such a write on the far side of the delete. Rows are stamped when they are dispatched rather than when set() was called, since a write may have waited. Covered by a test that fails without the guard. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(tmdb): add the release note for the cache panel Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(tmdb): cover the cache panel with an Electron E2E The panel drives IPC and SQLite, and nothing exercised that path end to end. The new test seeds a row through the preload bridge — enrichment itself needs a TMDB key that CI does not have — then opens the section, asserts the reported size, clears, and reads the database back to confirm the row is gone rather than merely hidden. Verified both ways: dropping the DELETE from clearTmdbMetadata fails it. Settings nav buttons gained a data-test-id so the section can be opened without matching translated labels. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
4grayandClaude Opus 4.8 authored and GitHub committed
1 parent
d76b2d2a57
commit
0b967d66d4
39 files changed
+888
-24
No files matched your search
@@ -50,4 +50,145 @@ describe('TmdbCacheService (in-memory LRU)', () => {
|
||||
const cached = await service.get('movie', 'id:same', 'en-US');
|
||||
expect(cached?.lookupKey).toBe('id:same');
|
||||
});
|
||||
|
||||
describe('stats and clearing (PWA path)', () => {
|
||||
it('counts entries and ENCODED payload bytes', async () => {
|
||||
// String.length would report 3 for a 6-byte Cyrillic payload
|
||||
await service.set({ ...entry('id:1'), payload: '"да"' });
|
||||
|
||||
const stats = await service.getStats();
|
||||
expect(stats?.entries).toBe(1);
|
||||
expect(stats?.bytes).toBe(
|
||||
new TextEncoder().encode('"да"').length
|
||||
);
|
||||
});
|
||||
|
||||
it('clears the map and reports how many entries went', async () => {
|
||||
await service.set(entry('id:1'));
|
||||
await service.set(entry('id:2'));
|
||||
|
||||
await expect(service.clear()).resolves.toBe(2);
|
||||
await expect(service.getStats()).resolves.toEqual({
|
||||
entries: 0,
|
||||
bytes: 0,
|
||||
});
|
||||
});
|
||||
|
||||
it('drops a write that was in flight when the cache was cleared', async () => {
|
||||
// Otherwise the late write silently restores exactly the data
|
||||
// the user just asked to remove
|
||||
const pending = service.set(entry('id:late'));
|
||||
await service.clear();
|
||||
await pending;
|
||||
|
||||
await expect(
|
||||
service.get('movie', 'id:late', 'en-US')
|
||||
).resolves.toBeNull();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('TmdbCacheService (Electron bridge)', () => {
|
||||
const entry = (lookupKey: string): TmdbCacheEntry => ({
|
||||
mediaType: 'movie',
|
||||
lookupKey,
|
||||
language: 'en-US',
|
||||
tmdbId: 1,
|
||||
payload: '{}',
|
||||
});
|
||||
|
||||
interface ElectronStub {
|
||||
dbGetTmdbMetadata: jest.Mock;
|
||||
dbSetTmdbMetadata: jest.Mock;
|
||||
dbGetTmdbCacheStats?: jest.Mock;
|
||||
dbClearTmdbMetadata?: jest.Mock;
|
||||
}
|
||||
|
||||
let electron: ElectronStub;
|
||||
|
||||
/** Resolves the next dbSetTmdbMetadata call on demand */
|
||||
let releaseWrite: () => void;
|
||||
|
||||
beforeEach(() => {
|
||||
releaseWrite = () => undefined;
|
||||
electron = {
|
||||
dbGetTmdbMetadata: jest.fn().mockResolvedValue(null),
|
||||
dbSetTmdbMetadata: jest.fn(
|
||||
() =>
|
||||
new Promise<void>((resolve) => {
|
||||
releaseWrite = resolve;
|
||||
})
|
||||
),
|
||||
dbGetTmdbCacheStats: jest
|
||||
.fn()
|
||||
.mockResolvedValue({ entries: 3, bytes: 90 }),
|
||||
dbClearTmdbMetadata: jest.fn().mockResolvedValue({ deleted: 3 }),
|
||||
};
|
||||
(window as unknown as { electron: unknown }).electron = electron;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
delete (window as unknown as { electron?: unknown }).electron;
|
||||
});
|
||||
|
||||
it('waits for an in-flight write instead of re-clearing after it', async () => {
|
||||
const service = new TmdbCacheService();
|
||||
const pending = service.set(entry('id:late'));
|
||||
|
||||
const cleared = service.clear();
|
||||
// The clear must not reach SQLite before the write it has to remove
|
||||
await Promise.resolve();
|
||||
expect(electron.dbClearTmdbMetadata).not.toHaveBeenCalled();
|
||||
|
||||
releaseWrite();
|
||||
await pending;
|
||||
await expect(cleared).resolves.toBe(3);
|
||||
expect(electron.dbClearTmdbMetadata).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('keeps rows written after the clear', async () => {
|
||||
const service = new TmdbCacheService();
|
||||
await service.clear();
|
||||
|
||||
const pending = service.set(entry('id:fresh'));
|
||||
releaseWrite();
|
||||
await pending;
|
||||
|
||||
// A second clear here would delete data the user never asked to lose
|
||||
expect(electron.dbClearTmdbMetadata).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('lands a write that starts during the clear after the delete', async () => {
|
||||
const service = new TmdbCacheService();
|
||||
const order: string[] = [];
|
||||
electron.dbClearTmdbMetadata = jest.fn(async () => {
|
||||
order.push('clear');
|
||||
return { deleted: 3 };
|
||||
});
|
||||
|
||||
const first = service.set(entry('id:before'));
|
||||
const cleared = service.clear();
|
||||
|
||||
// Starts while the clear is waiting on the first write — without
|
||||
// serialization its row would reach SQLite before the delete
|
||||
const second = service.set(entry('id:during'));
|
||||
electron.dbSetTmdbMetadata.mockImplementation(async () => {
|
||||
order.push('write');
|
||||
});
|
||||
|
||||
releaseWrite();
|
||||
await Promise.all([first, cleared, second]);
|
||||
|
||||
expect(order).toEqual(['clear', 'write']);
|
||||
});
|
||||
|
||||
it('reports an unsupported shell instead of the empty renderer map', async () => {
|
||||
// Newer renderer, older preload: rows are in SQLite but out of reach
|
||||
delete electron.dbGetTmdbCacheStats;
|
||||
delete electron.dbClearTmdbMetadata;
|
||||
const service = new TmdbCacheService();
|
||||
|
||||
await expect(service.getStats()).resolves.toBeNull();
|
||||
await expect(service.clear()).resolves.toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -1,5 +1,9 @@
|
||||
import { Injectable } from '@angular/core';
|
||||
import { TmdbCacheEntry, TmdbCacheMediaType } from '@iptvnator/shared/interfaces';
|
||||
import {
|
||||
TmdbCacheEntry,
|
||||
TmdbCacheMediaType,
|
||||
TmdbCacheStats,
|
||||
} from '@iptvnator/shared/interfaces';
|
||||
|
||||
/** PWA in-memory cache ceiling — details payloads are a few KB each */
|
||||
const MEMORY_CACHE_MAX_ENTRIES = 300;
|
||||
@@ -18,6 +22,17 @@ const MEMORY_CACHE_MAX_ENTRIES = 300;
|
||||
export class TmdbCacheService {
|
||||
private readonly memoryCache = new Map<string, TmdbCacheEntry>();
|
||||
|
||||
/**
|
||||
* Electron writes that have not landed yet. {@link clear} waits for
|
||||
* them, so a write already in flight cannot outlive the clear that
|
||||
* followed it — and, unlike re-clearing after the late write lands,
|
||||
* waiting cannot take rows written *after* the user cleared with it.
|
||||
*/
|
||||
private readonly pendingWrites = new Set<Promise<unknown>>();
|
||||
|
||||
/** Set for the duration of {@link clear}; writes queue behind it */
|
||||
private clearInFlight: Promise<unknown> | null = null;
|
||||
|
||||
private get bridge() {
|
||||
// typeof checks guard against version skew: an older Electron shell
|
||||
// may not expose the TMDB cache methods yet
|
||||
@@ -59,17 +74,29 @@ export class TmdbCacheService {
|
||||
}
|
||||
|
||||
async set(entry: TmdbCacheEntry): Promise<void> {
|
||||
const stamped: TmdbCacheEntry = {
|
||||
...entry,
|
||||
fetchedAt: new Date().toISOString(),
|
||||
};
|
||||
|
||||
const bridge = this.bridge;
|
||||
if (bridge) {
|
||||
// A clear owns the table until it finishes. Dispatching now
|
||||
// would race the delete and lose a row the user never asked to
|
||||
// lose, so this write waits it out and lands after it.
|
||||
while (this.clearInFlight) {
|
||||
await this.clearInFlight;
|
||||
}
|
||||
|
||||
// The IIFE also converts a synchronous throw from the bridge
|
||||
// into a rejection, so `set` never throws at its caller
|
||||
const write = (async () => {
|
||||
try {
|
||||
await bridge.dbSetTmdbMetadata(this.stamp(entry));
|
||||
} catch (error) {
|
||||
console.warn('TMDB cache write failed:', error);
|
||||
}
|
||||
})();
|
||||
this.pendingWrites.add(write);
|
||||
try {
|
||||
await bridge.dbSetTmdbMetadata(stamped);
|
||||
} catch (error) {
|
||||
console.warn('TMDB cache write failed:', error);
|
||||
await write;
|
||||
} finally {
|
||||
this.pendingWrites.delete(write);
|
||||
}
|
||||
return;
|
||||
}
|
||||
@@ -80,7 +107,7 @@ export class TmdbCacheService {
|
||||
entry.language
|
||||
);
|
||||
this.memoryCache.delete(key);
|
||||
this.memoryCache.set(key, stamped);
|
||||
this.memoryCache.set(key, this.stamp(entry));
|
||||
// delete-then-reinsert above means at most one entry over the cap
|
||||
if (this.memoryCache.size > MEMORY_CACHE_MAX_ENTRIES) {
|
||||
const oldest = this.memoryCache.keys().next().value;
|
||||
@@ -90,6 +117,11 @@ export class TmdbCacheService {
|
||||
}
|
||||
}
|
||||
|
||||
/** Stamped at write time, not at call time — a write may have waited */
|
||||
private stamp(entry: TmdbCacheEntry): TmdbCacheEntry {
|
||||
return { ...entry, fetchedAt: new Date().toISOString() };
|
||||
}
|
||||
|
||||
isFresh(entry: TmdbCacheEntry | null, ttlMs: number): boolean {
|
||||
if (!entry?.fetchedAt) {
|
||||
return false;
|
||||
@@ -106,4 +138,102 @@ export class TmdbCacheService {
|
||||
): string {
|
||||
return `${mediaType}:${language}:${lookupKey}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Cache size for the settings panel. In the PWA the cache is the
|
||||
* session-scoped map, so the numbers describe that instead.
|
||||
*/
|
||||
async getStats(): Promise<TmdbCacheStats | null> {
|
||||
const bridge = this.bridge;
|
||||
if (bridge) {
|
||||
if (!bridge.dbGetTmdbCacheStats) {
|
||||
// Version skew: this shell persists to SQLite but predates
|
||||
// the maintenance ops. The renderer map is empty in
|
||||
// Electron, so falling back to it would report a cache that
|
||||
// is anything but empty as having nothing in it.
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return await bridge.dbGetTmdbCacheStats();
|
||||
} catch (error) {
|
||||
// `null`, not zeros: reporting a failed read as an empty
|
||||
// cache would hide real rows and disable the Clear button
|
||||
console.warn('TMDB cache stats failed:', error);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// TextEncoder measures encoded bytes; String.length counts UTF-16
|
||||
// code units, which understates non-ASCII payloads and would
|
||||
// disagree with the SQLite BLOB byte count.
|
||||
const encoder = new TextEncoder();
|
||||
let bytes = 0;
|
||||
for (const entry of this.memoryCache.values()) {
|
||||
bytes += entry.payload ? encoder.encode(entry.payload).length : 0;
|
||||
}
|
||||
return { entries: this.memoryCache.size, bytes };
|
||||
}
|
||||
|
||||
/**
|
||||
* Drops everything. Enrichment refetches on demand, so this only
|
||||
* costs the next few requests.
|
||||
*/
|
||||
async clear(): Promise<number | null> {
|
||||
const bridge = this.bridge;
|
||||
const clearedFromMemory = this.memoryCache.size;
|
||||
this.memoryCache.clear();
|
||||
|
||||
if (!bridge) {
|
||||
return clearedFromMemory;
|
||||
}
|
||||
|
||||
const clearMetadata = bridge.dbClearTmdbMetadata;
|
||||
if (!clearMetadata) {
|
||||
// Same version skew as getStats(): the rows are in SQLite and
|
||||
// stay there, so reporting a successful clear would be a lie
|
||||
return null;
|
||||
}
|
||||
|
||||
// Called through the bridge so the preload keeps its receiver
|
||||
const run = this.runClear(() => clearMetadata.call(bridge));
|
||||
this.clearInFlight = run;
|
||||
try {
|
||||
return await run;
|
||||
} finally {
|
||||
this.clearInFlight = null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Everything between the writes already in flight and the delete. Held
|
||||
* in `clearInFlight` for its duration, which is what puts writes that
|
||||
* start meanwhile on the far side of the delete rather than under it.
|
||||
*/
|
||||
private async runClear(
|
||||
clearMetadata: () => Promise<{ deleted: number } | undefined>
|
||||
): Promise<number | null> {
|
||||
// Writes issued before the clear land first, so the delete takes
|
||||
// them too.
|
||||
if (this.pendingWrites.size > 0) {
|
||||
// Not Promise.allSettled: the web target compiles against
|
||||
// lib es2018. The writes swallow their own errors anyway.
|
||||
await Promise.all(
|
||||
[...this.pendingWrites].map((write) =>
|
||||
write.then(
|
||||
() => undefined,
|
||||
() => undefined
|
||||
)
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
try {
|
||||
const result = await clearMetadata();
|
||||
return result?.deleted ?? 0;
|
||||
} catch (error) {
|
||||
// `null` so the panel can say it failed and stay actionable
|
||||
console.warn('TMDB cache clear failed:', error);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user