mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-14 13:16:16 -08:00
docs(tmdb): replace catalog titles in matching examples with stand-ins
The Cyrillic and Arabic examples that document title folding were taken straight from a user's own portal catalog while the folding bugs were being diagnosed, and they spread from there into comments, fixtures, assertions and the architecture doc. A public repository is not the place for someone's viewing inventory, and the TMDB ids pinned alongside them identify the exact shows. Swap in stand-ins that reproduce the property under test rather than the title: a word whose "й" folds away, one whose "ё" does, a two-word title carrying a season marker, an Arabic phrase whose initial hamza decomposes into a separate word. Every replacement was run through the real `normalizeTitle` and `cleanTitleForSearch` before being written, so the folded keys, the wire queries and the cache lookup keys are the same shape as before — "Лейка" and "Леика" still meet on one key while staying two different searches, and "AR| أمثلة تجريبية" still keys as a hamza split into a space. Real TMDB ids become synthetic ones. One channel fixture that read as a film title becomes a plain channel name. Deliberately left alone: the leading-tag rule's "Akira | 1988" / "Момо | Momo" examples in `title-normalization.util.ts`. Those are not an inventory — they are the evidence for a regex decision measured over 1.27M catalog titles, and inventing replacements would document a measurement that never happened. No release note: comments, fixtures and docs only, with no behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
11358caa7a
commit
1a1f5f30db
10 files changed
+91
-91
No files matched your search
+2
-2
@@ -30,7 +30,7 @@ const TITLES = [
|
||||
'Şan Ve Şeref',
|
||||
'Первый канал HD',
|
||||
'ПЕРВЫЙ КАНАЛ',
|
||||
'Ёлки 1914',
|
||||
'Ёлки ТВ',
|
||||
'Йога для всех',
|
||||
'Матч ТВ',
|
||||
'Ελλάδα Σήμερα',
|
||||
@@ -179,7 +179,7 @@ describe('content-search.util against SQLite', () => {
|
||||
it.each([
|
||||
['Первый', 'первый', ['Первый канал HD', 'ПЕРВЫЙ КАНАЛ']],
|
||||
['ПЕРВ', 'перв', ['Первый канал HD', 'ПЕРВЫЙ КАНАЛ']],
|
||||
['Ёлки', 'ёлки', ['Ёлки 1914']],
|
||||
['Ёлки', 'ёлки', ['Ёлки ТВ']],
|
||||
['Йога', 'йога', ['Йога для всех']],
|
||||
['Ма', 'ма', ['Матч ТВ']],
|
||||
['ΕΛΛ', 'ελλ', ['Ελλάδα Σήμερα']],
|
||||
|
||||
@@ -537,7 +537,7 @@ identity is the query with only its case removed (`searchQueryIdentity`):
|
||||
variants are deduplicated by it and every attempted variant is cached under
|
||||
its own key (`title:<identity>|year:<y>|v4`, in the language that variant
|
||||
was searched in), never by the folded key and never only under the first
|
||||
variant — "Феик" and "Фейк" fold to one key but are different searches with
|
||||
variant — "Леика" and "Лейка" fold to one key but are different searches with
|
||||
different answers, so a verdict for one must not be read back for the other;
|
||||
a misspelled original title must not swallow the display title that TMDB
|
||||
actually knows; and two items that share an original title but not a display
|
||||
@@ -547,9 +547,9 @@ splits Cyrillic "й" into "и" + a combining breve and "ё" into "е" + a
|
||||
diaeresis, and Arabic hamza forms ("أ") into a bare alef + a combining hamza
|
||||
that the punctuation step then turns into a space inside the word. The key
|
||||
drops or splits on those marks, and TMDB's `/search` does not fold them the
|
||||
same way — a query of `феик` returns zero results while `Фейк` returns the
|
||||
same way — a query of `леика` returns zero results while `Лейка` returns the
|
||||
show. Under the old single-form design every Russian title with "й"/"ё"
|
||||
("Фейк (10 серий)", "Волшебный участок", "Молодой Шерлок") was searched
|
||||
("Лейка (10 серий)", "Тестовый Сериал", "Пробный Выпуск") was searched
|
||||
folded, missed, and cached as missing for the 7-day negative TTL. Compare
|
||||
results only through `normalized`; never send it over the wire.
|
||||
|
||||
|
||||
@@ -21,10 +21,10 @@ describe('stalker-series.adapters', () => {
|
||||
'The Gentlemen season2',
|
||||
'The Gentlemen (s02)',
|
||||
'The Gentlemen (season 2)',
|
||||
'Олдскул (2 сезон)',
|
||||
'Олдскул (сезон 2)',
|
||||
'Олдскул s02',
|
||||
'Олдскул season2',
|
||||
'Пример (2 сезон)',
|
||||
'Пример (сезон 2)',
|
||||
'Пример s02',
|
||||
'Пример season2',
|
||||
])(
|
||||
'uses the title season for lazy VOD %s without changing tracking IDs',
|
||||
(title) => {
|
||||
|
||||
@@ -9,20 +9,20 @@ import { TmdbSearchResult } from './tmdb.types';
|
||||
* Regression coverage for the search wire format. The comparison key folds
|
||||
* diacritics — Cyrillic "й" becomes "и" — and that key used to be sent as
|
||||
* the TMDB query, which matched nothing for any title carrying "й"/"ё" and
|
||||
* cached the miss for a week ("Фейк (10 серий)", "Волшебный участок").
|
||||
* cached the miss for a week ("Лейка (10 серий)", "Тестовый Сериал").
|
||||
*/
|
||||
describe('TmdbIdResolverService.resolveBySearch', () => {
|
||||
const fake2026: TmdbSearchResult = {
|
||||
id: 317869,
|
||||
name: 'Фейк',
|
||||
original_name: 'Фейк',
|
||||
const series2026: TmdbSearchResult = {
|
||||
id: 101101,
|
||||
name: 'Лейка',
|
||||
original_name: 'Лейка',
|
||||
first_air_date: '2026-07-16',
|
||||
vote_count: 3,
|
||||
};
|
||||
const fake2024: TmdbSearchResult = {
|
||||
id: 322696,
|
||||
name: 'Фейк',
|
||||
original_name: 'Фейк',
|
||||
const series2024: TmdbSearchResult = {
|
||||
id: 101102,
|
||||
name: 'Лейка',
|
||||
original_name: 'Лейка',
|
||||
first_air_date: '2024-12-05',
|
||||
vote_count: 0,
|
||||
};
|
||||
@@ -67,7 +67,7 @@ describe('TmdbIdResolverService.resolveBySearch', () => {
|
||||
beforeEach(() => {
|
||||
searchTv = jest.fn(async (query: string) =>
|
||||
// TMDB does not fold Cyrillic: only the provider spelling hits
|
||||
query === 'Фейк' ? [fake2026, fake2024] : []
|
||||
query === 'Лейка' ? [series2026, series2024] : []
|
||||
);
|
||||
searchMovie = jest.fn().mockResolvedValue([]);
|
||||
cacheGet = jest.fn().mockResolvedValue(null);
|
||||
@@ -78,18 +78,18 @@ describe('TmdbIdResolverService.resolveBySearch', () => {
|
||||
const service = createService();
|
||||
|
||||
const id = await service.resolveBySearch('tv', {
|
||||
title: 'Фейк (10 серий)',
|
||||
title: 'Лейка (10 серий)',
|
||||
year: 2026,
|
||||
});
|
||||
|
||||
expect(id).toBe(317869);
|
||||
expect(id).toBe(101101);
|
||||
expect(searchTv).toHaveBeenCalledTimes(1);
|
||||
expect(searchTv).toHaveBeenCalledWith('Фейк', null, 'ru-RU', 'key');
|
||||
expect(searchTv).toHaveBeenCalledWith('Лейка', null, 'ru-RU', 'key');
|
||||
expect(cacheSet).toHaveBeenCalledWith({
|
||||
mediaType: 'tv',
|
||||
lookupKey: 'title:фейк|year:2026|v4',
|
||||
lookupKey: 'title:лейка|year:2026|v4',
|
||||
language: 'ru-RU',
|
||||
tmdbId: 317869,
|
||||
tmdbId: 101101,
|
||||
payload: null,
|
||||
});
|
||||
});
|
||||
@@ -99,41 +99,41 @@ describe('TmdbIdResolverService.resolveBySearch', () => {
|
||||
const service = createService();
|
||||
|
||||
const id = await service.resolveBySearch('tv', {
|
||||
title: 'Молодой Шерлок',
|
||||
title: 'Пробный Выпуск',
|
||||
year: 2026,
|
||||
});
|
||||
|
||||
expect(id).toBeNull();
|
||||
expect(searchTv).toHaveBeenCalledWith(
|
||||
'Молодой Шерлок',
|
||||
'Пробный Выпуск',
|
||||
null,
|
||||
'ru-RU',
|
||||
'key'
|
||||
);
|
||||
expect(cacheSet).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
lookupKey: 'title:молодой шерлок|year:2026|v4',
|
||||
lookupKey: 'title:пробный выпуск|year:2026|v4',
|
||||
tmdbId: null,
|
||||
})
|
||||
);
|
||||
});
|
||||
|
||||
it('reads the cache before searching', async () => {
|
||||
cacheGet.mockResolvedValue({ tmdbId: 317869 });
|
||||
cacheGet.mockResolvedValue({ tmdbId: 101101 });
|
||||
const service = createService();
|
||||
const cacheService = (service as unknown as { cache: TmdbCacheService })
|
||||
.cache;
|
||||
jest.spyOn(cacheService, 'isFresh').mockReturnValue(true);
|
||||
|
||||
const id = await service.resolveBySearch('tv', {
|
||||
title: 'Фейк (10 серий)',
|
||||
title: 'Лейка (10 серий)',
|
||||
year: 2026,
|
||||
});
|
||||
|
||||
expect(id).toBe(317869);
|
||||
expect(id).toBe(101101);
|
||||
expect(cacheGet).toHaveBeenCalledWith(
|
||||
'tv',
|
||||
'title:фейк|year:2026|v4',
|
||||
'title:лейка|year:2026|v4',
|
||||
'ru-RU'
|
||||
);
|
||||
expect(searchTv).not.toHaveBeenCalled();
|
||||
@@ -143,31 +143,31 @@ describe('TmdbIdResolverService.resolveBySearch', () => {
|
||||
const service = createService();
|
||||
|
||||
const id = await service.resolveBySearch('tv', {
|
||||
title: 'Фейк',
|
||||
originalTitle: 'Феик',
|
||||
title: 'Лейка',
|
||||
originalTitle: 'Леика',
|
||||
year: 2026,
|
||||
});
|
||||
|
||||
expect(id).toBe(317869);
|
||||
expect(id).toBe(101101);
|
||||
expect(searchTv.mock.calls.map(([query]) => query)).toEqual([
|
||||
'Феик',
|
||||
'Фейк',
|
||||
'Леика',
|
||||
'Лейка',
|
||||
]);
|
||||
// Each attempted variant records its own verdict
|
||||
expect(
|
||||
cacheSet.mock.calls.map(([row]) => [row.lookupKey, row.tmdbId])
|
||||
).toEqual([
|
||||
['title:феик|year:2026|v4', null],
|
||||
['title:фейк|year:2026|v4', 317869],
|
||||
['title:леика|year:2026|v4', null],
|
||||
['title:лейка|year:2026|v4', 101101],
|
||||
]);
|
||||
});
|
||||
|
||||
it("does not let one variant's cached verdict answer for another", async () => {
|
||||
// Item A (original "Феик", display "Феик") cached a miss under
|
||||
// "феик". Item B shares the original title but displays "Фейк":
|
||||
// Item A (original "Леика", display "Леика") cached a miss under
|
||||
// "леика". Item B shares the original title but displays "Лейка":
|
||||
// the cached miss must not suppress B's own second variant.
|
||||
cacheGet.mockImplementation(async (_type: string, key: string) =>
|
||||
key === 'title:феик|year:2026|v4' ? { tmdbId: null } : null
|
||||
key === 'title:леика|year:2026|v4' ? { tmdbId: null } : null
|
||||
);
|
||||
const service = createService();
|
||||
const cacheService = (service as unknown as { cache: TmdbCacheService })
|
||||
@@ -177,17 +177,17 @@ describe('TmdbIdResolverService.resolveBySearch', () => {
|
||||
);
|
||||
|
||||
const id = await service.resolveBySearch('tv', {
|
||||
title: 'Фейк',
|
||||
originalTitle: 'Феик',
|
||||
title: 'Лейка',
|
||||
originalTitle: 'Леика',
|
||||
year: 2026,
|
||||
});
|
||||
|
||||
expect(id).toBe(317869);
|
||||
expect(id).toBe(101101);
|
||||
expect(searchTv).toHaveBeenCalledTimes(1);
|
||||
expect(searchTv).toHaveBeenCalledWith('Фейк', null, 'ru-RU', 'key');
|
||||
expect(searchTv).toHaveBeenCalledWith('Лейка', null, 'ru-RU', 'key');
|
||||
expect(cacheGet.mock.calls.map(([, key]) => key)).toEqual([
|
||||
'title:феик|year:2026|v4',
|
||||
'title:фейк|year:2026|v4',
|
||||
'title:леика|year:2026|v4',
|
||||
'title:лейка|year:2026|v4',
|
||||
]);
|
||||
});
|
||||
|
||||
|
||||
@@ -88,14 +88,14 @@ describe('lookup keys', () => {
|
||||
});
|
||||
|
||||
it('keys search rows by the wire spelling, not the folded key', () => {
|
||||
// "Феик" and "Фейк" fold to one comparison key but are different
|
||||
// "Леика" and "Лейка" fold to one comparison key but are different
|
||||
// searches with different answers; a verdict cached for one must
|
||||
// never be read back for the other.
|
||||
expect(buildSearchLookupKey('Фейк', 2026)).toBe(
|
||||
'title:фейк|year:2026|v4'
|
||||
expect(buildSearchLookupKey('Лейка', 2026)).toBe(
|
||||
'title:лейка|year:2026|v4'
|
||||
);
|
||||
expect(buildSearchLookupKey('Феик', 2026)).not.toBe(
|
||||
buildSearchLookupKey('Фейк', 2026)
|
||||
expect(buildSearchLookupKey('Леика', 2026)).not.toBe(
|
||||
buildSearchLookupKey('Лейка', 2026)
|
||||
);
|
||||
expect(buildSearchLookupKey('THE BOYS', 2019)).toBe(
|
||||
buildSearchLookupKey('The Boys', 2019)
|
||||
@@ -139,8 +139,8 @@ describe('buildSearchTitleVariants', () => {
|
||||
|
||||
it('sends the provider spelling to the search but compares on the folded key', () => {
|
||||
// The folded key rewrites "й" as "и"; TMDB finds nothing for it.
|
||||
expect(buildSearchTitleVariants('Фейк (10 серий)', null)).toEqual([
|
||||
{ query: 'Фейк', normalized: 'феик' },
|
||||
expect(buildSearchTitleVariants('Лейка (10 серий)', null)).toEqual([
|
||||
{ query: 'Лейка', normalized: 'леика' },
|
||||
]);
|
||||
// Arabic hamza forms fold into a space inside the word
|
||||
expect(buildSearchTitleVariants('إيمان', null)).toEqual([
|
||||
@@ -161,11 +161,11 @@ describe('buildSearchTitleVariants', () => {
|
||||
|
||||
it('keeps spellings that fold to one key but differ on the wire', () => {
|
||||
// A misspelled original title must not swallow the display title:
|
||||
// TMDB knows "Фейк" and not "Феик", and only the second variant
|
||||
// TMDB knows "Лейка" and not "Леика", and only the second variant
|
||||
// would find it.
|
||||
expect(buildSearchTitleVariants('Фейк', 'Феик')).toEqual([
|
||||
{ query: 'Феик', normalized: 'феик' },
|
||||
{ query: 'Фейк', normalized: 'феик' },
|
||||
expect(buildSearchTitleVariants('Лейка', 'Леика')).toEqual([
|
||||
{ query: 'Леика', normalized: 'леика' },
|
||||
{ query: 'Лейка', normalized: 'леика' },
|
||||
]);
|
||||
expect(buildSearchTitleVariants('Amelie', 'Amélie')).toEqual([
|
||||
{ query: 'Amélie', normalized: 'amelie' },
|
||||
|
||||
@@ -40,7 +40,7 @@ function stripLeadingLanguageToken(raw: string): string | null {
|
||||
/**
|
||||
* One search candidate: what to SEND to TMDB and what to COMPARE its
|
||||
* answers against. The two differ on purpose — see `cleanTitleForSearch`:
|
||||
* a folded query ("феик") finds nothing on TMDB while the folded key is
|
||||
* a folded query ("леика") finds nothing on TMDB while the folded key is
|
||||
* exactly what the confidence gate and the cache need.
|
||||
*/
|
||||
export interface SearchTitleVariant {
|
||||
@@ -53,7 +53,7 @@ export interface SearchTitleVariant {
|
||||
/**
|
||||
* The identity of one search on the wire: the query with only the case
|
||||
* removed, since TMDB matches case-insensitively and nothing else about
|
||||
* the spelling may be folded away — "Феик" and "Фейк" are different
|
||||
* the spelling may be folded away — "Леика" and "Лейка" are different
|
||||
* searches with different answers, however alike their comparison keys.
|
||||
* Both the variant deduplication and the cache row use this, so a cached
|
||||
* verdict can never be read back for a search that was never sent.
|
||||
@@ -116,8 +116,8 @@ export function buildSearchLookupKey(
|
||||
// v2: normalizeTitleKeys learned to strip appended language/quality
|
||||
// tags; the version suffix invalidates cached (incl. negative) match
|
||||
// resolutions keyed on the old polluted titles.
|
||||
// v3: the search query stopped being the folded key ("феик" for
|
||||
// "Фейк"), which TMDB answered with nothing; every negative row recorded
|
||||
// v3: the search query stopped being the folded key ("леика" for
|
||||
// "Лейка"), which TMDB answered with nothing; every negative row recorded
|
||||
// under v2 for a title with "й"/"ё" is that bug, not a missing title, and
|
||||
// must not block the retry for its 7-day TTL. Rows are keyed by the
|
||||
// query since then.
|
||||
|
||||
@@ -976,8 +976,8 @@ function widenTmdbMetadataMediaTypeCheck(sqliteDb: Database.Database): void {
|
||||
* - unversioned → v2: title normalization learned to strip appended
|
||||
* language/quality tags.
|
||||
* - v2 → v3: the search query stopped being the folded comparison key. Under
|
||||
* v2 every title with a Cyrillic "й"/"ё" was searched folded ("феик" for
|
||||
* "Фейк", "елки" for "Ёлки"), got no answer, and was cached as missing for
|
||||
* v2 every title with a Cyrillic "й"/"ё" was searched folded ("леика" for
|
||||
* "Лейка", "ежик" for "Ёжик"), got no answer, and was cached as missing for
|
||||
* 7 days.
|
||||
* - v3 → v4: year evidence became tiered. Under v3 a series admitted only by
|
||||
* the "premiered earlier" tolerance competed with an exact-year match on
|
||||
|
||||
@@ -30,13 +30,13 @@ it('drops only retired search rows across skipped, previous, pre-person, fresh a
|
||||
const V3_MARKER = 'migration:tmdb-search-lookup-v3-cache-cleanup:v1';
|
||||
const V4_MARKER = 'migration:tmdb-search-lookup-v4-cache-cleanup:v1';
|
||||
const ROWS = [
|
||||
['tv', 'title:феик|year:2026', 'ru-RU', null],
|
||||
['tv', 'title:феик|year:2026|v2', 'ru-RU', null],
|
||||
['tv', 'title:леика|year:2026', 'ru-RU', null],
|
||||
['tv', 'title:леика|year:2026|v2', 'ru-RU', null],
|
||||
['tv', 'title:the boys|year:2019|v2', 'en-US', 76479],
|
||||
['tv', 'title:фейк|year:2026|v3', 'ru-RU', 317869],
|
||||
['tv', 'title:лейка|year:2026|v3', 'ru-RU', 101101],
|
||||
['tv', 'title:nightfall|year:2026|v4', 'ru-RU', 424242],
|
||||
['tv', 'id:317869|v2', 'ru-RU', 317869],
|
||||
['tv', 'id:317869|season:1', 'ru-RU', 317869],
|
||||
['tv', 'id:101101|v2', 'ru-RU', 101101],
|
||||
['tv', 'id:101101|season:1', 'ru-RU', 101101],
|
||||
['person', 'person:287', 'en-US', 287],
|
||||
['movie', 'badProviderId:999', 'any', null],
|
||||
['movie', 'trending:week', 'en-US', null],
|
||||
@@ -63,7 +63,7 @@ it('drops only retired search rows across skipped, previous, pre-person, fresh a
|
||||
hooks.runMigrations(skipped);
|
||||
const skippedAfter = snapshot(skipped);
|
||||
// Next startup: a row written meanwhile under the current key survives
|
||||
skipped.prepare("INSERT INTO tmdb_metadata (media_type, lookup_key, language, tmdb_id) VALUES ('tv', 'title:гудовы|year:2026|v4', 'ru-RU', 318894)").run();
|
||||
skipped.prepare("INSERT INTO tmdb_metadata (media_type, lookup_key, language, tmdb_id) VALUES ('tv', 'title:гудовы|year:2026|v4', 'ru-RU', 101103)").run();
|
||||
hooks.runMigrations(skipped);
|
||||
const repeated = snapshot(skipped);
|
||||
// Previous release: the earlier cleanups already ran; only v3 rows go
|
||||
@@ -78,7 +78,7 @@ it('drops only retired search rows across skipped, previous, pre-person, fresh a
|
||||
const prePerson = new Database(':memory:');
|
||||
hooks.createTables(prePerson);
|
||||
prePerson.exec("DROP TABLE tmdb_metadata; CREATE TABLE tmdb_metadata (id INTEGER PRIMARY KEY AUTOINCREMENT, media_type TEXT NOT NULL CHECK (media_type IN ('movie', 'tv')), lookup_key TEXT NOT NULL, language TEXT NOT NULL, tmdb_id INTEGER, payload TEXT, fetched_at TEXT DEFAULT (datetime('now'))); CREATE UNIQUE INDEX tmdb_metadata_lookup_unique ON tmdb_metadata(media_type, lookup_key, language)");
|
||||
prePerson.prepare("INSERT INTO tmdb_metadata (media_type, lookup_key, language, tmdb_id) VALUES ('tv', 'title:феик|year:2026', 'ru-RU', NULL)").run();
|
||||
prePerson.prepare("INSERT INTO tmdb_metadata (media_type, lookup_key, language, tmdb_id) VALUES ('tv', 'title:леика|year:2026', 'ru-RU', NULL)").run();
|
||||
hooks.runMigrations(prePerson);
|
||||
const prePersonAfter = { ...snapshot(prePerson), check: prePerson.prepare("SELECT sql FROM sqlite_master WHERE name = 'tmdb_metadata'").get().sql.includes("'person'") };
|
||||
hooks.runMigrations(prePerson);
|
||||
@@ -109,13 +109,13 @@ it('drops only retired search rows across skipped, previous, pre-person, fresh a
|
||||
const V4_MARKER = 'migration:tmdb-search-lookup-v4-cache-cleanup:v1';
|
||||
const survivors = [
|
||||
'badProviderId:999',
|
||||
'id:317869|season:1',
|
||||
'id:317869|v2',
|
||||
'id:101101|season:1',
|
||||
'id:101101|v2',
|
||||
'person:287',
|
||||
'title:nightfall|year:2026|v4',
|
||||
'trending:week',
|
||||
];
|
||||
const detailsPayloads = ['{"id":317869}', '{"id":317869}'];
|
||||
const detailsPayloads = ['{"id":101101}', '{"id":101101}'];
|
||||
|
||||
expect(JSON.parse(result)).toEqual({
|
||||
skippedAfter: {
|
||||
@@ -132,8 +132,8 @@ it('drops only retired search rows across skipped, previous, pre-person, fresh a
|
||||
// The older generations' rows are their business, already done
|
||||
keys: [
|
||||
...survivors,
|
||||
'title:феик|year:2026',
|
||||
'title:феик|year:2026|v2',
|
||||
'title:леика|year:2026',
|
||||
'title:леика|year:2026|v2',
|
||||
'title:the boys|year:2019|v2',
|
||||
].sort(),
|
||||
payloads: detailsPayloads,
|
||||
@@ -142,8 +142,8 @@ it('drops only retired search rows across skipped, previous, pre-person, fresh a
|
||||
previousRepeated: {
|
||||
keys: [
|
||||
...survivors,
|
||||
'title:феик|year:2026',
|
||||
'title:феик|year:2026|v2',
|
||||
'title:леика|year:2026',
|
||||
'title:леика|year:2026|v2',
|
||||
'title:the boys|year:2019|v2',
|
||||
].sort(),
|
||||
payloads: detailsPayloads,
|
||||
|
||||
@@ -435,21 +435,21 @@ describe('cleanTitleForSearch', () => {
|
||||
it('keeps Cyrillic letters that folding would rewrite', () => {
|
||||
// NFD splits "й" into "и" + a breve and "ё" into "е" + a diaeresis;
|
||||
// the comparison key drops both marks, and TMDB answers the folded
|
||||
// spelling with nothing (issue: "Фейк (10 серий)" never matched).
|
||||
expect(normalizeTitle('Фейк (10 серий)')).toBe('феик');
|
||||
expect(cleanTitleForSearch('Фейк (10 серий)')).toBe('Фейк');
|
||||
expect(cleanTitleForSearch('Ёлки 2010')).toBe('Ёлки');
|
||||
expect(cleanTitleForSearch('Волшебный участок s02')).toBe(
|
||||
'Волшебный участок'
|
||||
// spelling with nothing (issue: "Лейка (10 серий)" never matched).
|
||||
expect(normalizeTitle('Лейка (10 серий)')).toBe('леика');
|
||||
expect(cleanTitleForSearch('Лейка (10 серий)')).toBe('Лейка');
|
||||
expect(cleanTitleForSearch('Ёжик 2010')).toBe('Ёжик');
|
||||
expect(cleanTitleForSearch('Тестовый Сериал s02')).toBe(
|
||||
'Тестовый Сериал'
|
||||
);
|
||||
});
|
||||
|
||||
it('keeps Arabic hamza forms that folding splits into two words', () => {
|
||||
// "أ" decomposes into a bare alef + U+0654, which is outside the
|
||||
// stripped mark range and so becomes a SPACE in the key. TMDB finds
|
||||
// "أطرق بابي" and nothing for "ا طرق بابي".
|
||||
expect(normalizeTitle('AR| أطرق بابي')).toBe('ا طرق بابي');
|
||||
expect(cleanTitleForSearch('AR| أطرق بابي')).toBe('أطرق بابي');
|
||||
// "أمثلة تجريبية" and nothing for "ا مثلة تجريبية".
|
||||
expect(normalizeTitle('AR| أمثلة تجريبية')).toBe('ا مثلة تجريبية');
|
||||
expect(cleanTitleForSearch('AR| أمثلة تجريبية')).toBe('أمثلة تجريبية');
|
||||
});
|
||||
|
||||
it('keeps Latin diacritics and casing', () => {
|
||||
@@ -490,9 +490,9 @@ describe('cleanTitleForSearch', () => {
|
||||
|
||||
it('normalizes to the same key as the text it was derived from', () => {
|
||||
for (const raw of [
|
||||
'Фейк (10 серий)',
|
||||
'Лейка (10 серий)',
|
||||
'EN - The Matrix (1999) 4K',
|
||||
'Ёлки 2010',
|
||||
'Ёжик 2010',
|
||||
'Amélie (2001)',
|
||||
'|FR|VO|Le dernier empereur',
|
||||
]) {
|
||||
|
||||
@@ -443,7 +443,7 @@ const SEASON_SUFFIX_PATTERN = new RegExp(
|
||||
* every comparison key and OFF for the text sent to a remote search — see
|
||||
* `cleanTitleForSearch`. Folding is lossy for scripts whose "diacritics" are
|
||||
* distinct letters: NFD turns Cyrillic "й" into "и" + a combining breve, and
|
||||
* dropping the breve rewrites "Фейк" as "феик", "ё" as "е". Two provider
|
||||
* dropping the breve rewrites "Лейка" as "леика", "ё" as "е". Two provider
|
||||
* copies of a title still meet on that key, which is all a comparison needs,
|
||||
* but TMDB's search does not fold Cyrillic the same way and answers a folded
|
||||
* query with nothing at all.
|
||||
@@ -520,7 +520,7 @@ export function normalizeTitleKeys(
|
||||
*
|
||||
* Comparison keys must fold so two spellings of one film meet; a search
|
||||
* query must not, because the search engine folds by its own rules and a
|
||||
* pre-folded Cyrillic query ("феик" for "Фейк") matches nothing there.
|
||||
* pre-folded Cyrillic query ("леика" for "Лейка") matches nothing there.
|
||||
* Compare the results with `normalizeTitle`, never with this.
|
||||
*/
|
||||
export function cleanTitleForSearch(raw: string | null | undefined): string {
|
||||
|
||||
Reference in new issue
Block a user