fix(matching): keep abbreviation titles out of bare-year keys (#1426)

A leading 2-5 character uppercase token before a dash, pipe or colon was
always read as a provider tag, so a film whose NAME is such a token lost it and
normalized down to its release year alone. "AKA - 2023" became the key "2023",
where it collided with BDE, BRO, OUT, WIL and IF — and they were offered to
each other as alternative sources in VOD multi-source.

"IT - 65 (2023)" (the Italian copy of the film "65") is structurally identical,
so only the token's meaning can separate them. The leading token is now tested
against a vocabulary — TRAILING_TAG_VOCABULARY plus a prefix-only list derived
from the real catalog — but only when the strip would leave no real word
behind. A compound is read by its head, so the open-ended "4K-<lang>" family
keeps working while "INU-OH" and "PC-4L" are recognized as film names. An
unknown token keeps its title: a refused strip costs one unmatched copy, a
wrong one corrupts that film's identity in VOD multi-source, the TMDB Similar
rail, DB_MATCH_TITLES and pin keys.

"No real word" is decided by running the rest of the pipeline on the stripped
form and looking at what comes out, never by re-implementing what later stages
remove. Quality tags, trailing and underscore tags, double-dash suffixes and
season markers each otherwise smuggle the strip through, and a stage added
later is covered for free.

Validated over the live catalog, movies and series: 83 keys fixed, 0 corrupted
across 1,616,111 titles. Deriving the vocabulary from movies alone missed AMZ,
D+ and P+ and broke the Paramount+/Disney+ copies of the numeric series 1923,
1883, 24 and 9-1-1.
This commit is contained in:
4gray authored and GitHub committed 2026-08-13 17:01:33 +02:00
1 parent 30d76a3a46
commit 82a2948ffd
5 files changed
+379 -26

No files matched your search

@@ -298,6 +298,119 @@ describe('provider tag stripping', () => {
}
);
describe("leading tag vs. the film's own name", () => {
// "IT - 65 (2023)" and "AKA - 2023" are the same shape: 2-5 uppercase
// characters, a separator, digits. Only the token's MEANING separates
// the Italian copy of the film "65" from the film "AKA" followed by
// its year, so the strip is vocabulary-gated whenever nothing but
// digits would survive it.
it('keeps a name that only looks like a tag before its year', () => {
// Measured on the live catalog: these all normalized to a bare
// year, so AKA/BDE/BRO/OUT/WIL/IF shared the single key "2023"
// and were offered to each other as alternative sources.
expect(normalizeTitle('AKA - 2023')).toBe('aka');
expect(normalizeTitle('AKA | 2023')).toBe('aka');
expect(normalizeTitle('IO - 2019')).toBe('io');
expect(normalizeTitle('ARQ - 2016')).toBe('arq');
expect(normalizeTitle('UFO - 2012')).toBe('ufo');
expect(normalizeTitle('LBJ - 2017')).toBe('lbj');
expect(normalizeTitle('EO - 2022')).toBe('eo');
expect(normalizeTitle('BDE - 2023')).toBe('bde');
expect(normalizeTitle('VFW - 2019 (4K HDR)')).toBe('vfw');
expect(normalizeTitleKeys('AKA - 2023').exact).toBe('aka 2023');
});
it('still strips real tags before a numeric film title', () => {
// The opposite shape, and the reason a shape-only guard was
// rejected: here the tag is real and the film's NAME is numeric.
expect(normalizeTitle('IT - 65 (2023)')).toBe('65');
expect(normalizeTitle('|FR|VO| 300 (2007)')).toBe('300');
expect(normalizeTitle('|FR|VO| 1922')).toBe('1922');
expect(normalizeTitle('OSN - 1917 - 2019')).toBe('1917');
expect(normalizeTitle('EN - 42 - 2013')).toBe('42');
expect(normalizeTitle('NRC - 7500 (2019)')).toBe('7500');
expect(normalizeTitle('EXYU| 3022')).toBe('3022');
});
it('strips streaming-provider tags before numeric series titles', () => {
// Found only in the series catalog: a movie-only vocabulary
// dropped these copies of 1923/1883/24/99 out of their groups.
expect(normalizeTitle('AMZ - 99 (2024)')).toBe('99');
expect(normalizeTitle('D+ - 24 (2001) (US)')).toBe('24');
expect(normalizeTitle('D+ - 9-1-1 (2018) (US)')).toBe('9 1 1');
expect(normalizeTitle('P+ - 1883 (2021)')).toBe('1883');
});
it('reads a compound tag by its head, so names survive', () => {
// "4K-<lang>" pairings are open-ended, so the head carries the
// meaning — and it is also what tells a compound tag apart from
// a hyphenated name.
expect(normalizeTitle('4K-FR - 1992 (2024)')).toBe('1992');
expect(normalizeTitle('AR-SUBS - 180 (2026)')).toBe('180');
expect(normalizeTitle('SO-IN - 65 (2023)')).toBe('65');
expect(normalizeTitle('INU-OH - 2022')).toBe('inu oh');
expect(normalizeTitle('PC-4L - 2020')).toBe('pc 4l');
});
it('asks the real pipeline what survives, not a list of its rules', () => {
// Every later stage removes something, so a guard that predicts
// them is a list to keep in sync. These are one case per stage,
// and all of them fall out of running the pipeline and looking:
expect(normalizeTitle('|TA| RRR - HEVC')).toBe('rrr'); // quality
expect(normalizeTitle('CAT - Multi ENG')).toBe('cat'); // trailing
expect(normalizeTitle('IF - 2024_sub')).toBe('if'); // underscore
expect(normalizeTitle('AKA --xyz')).toBe('aka'); // double dash
expect(normalizeTitle('CAT - 2022 S01')).toBe('cat 2022'); // season
// …while a known tag before a numeric title still strips on every
// one of those paths.
expect(normalizeTitle('P+ - 1923 S01')).toBe('1923');
expect(normalizeTitle('IT - 65 s01')).toBe('65');
expect(normalizeTitle('EX - 1917 (2019) 4K')).toBe('1917');
});
it('does not let a quality suffix smuggle the strip through', () => {
// The suffix has letters only until QUALITY_TAGS removes them, so
// testing the raw remainder would strip the name and normalize
// "RRR - HEVC" to the EMPTY key — while "RRR - 2022" keys as
// "rrr", hiding one copy of the film from the other.
expect(normalizeTitle('|TA| RRR - HEVC')).toBe('rrr');
expect(normalizeTitle('CAT - Multi')).toBe('cat');
expect(normalizeTitle('RRR - 2022 - 4K')).toBe('rrr');
expect(normalizeTitle('VFW - 2019 UHD')).toBe('vfw');
// A TRAILING language tag is dropped a few lines later too, so it
// is no more a word than a quality tag is. Real catalog titles:
// "sub" made the remainder look meaningful and the key came out
// as the bare year this whole guard exists to prevent.
expect(normalizeTitle('IF - 2024_sub')).toBe('if');
expect(normalizeTitle('O2 - 2024_sub')).toBe('o2');
expect(normalizeTitle('UFO - 2022_sub')).toBe('ufo');
expect(normalizeTitle('CAT - Multi ENG')).toBe('cat');
// …but a tag word next to a real one is just part of the title.
expect(normalizeTitle('EN - Sub Zero')).toBe('sub zero');
// …and a real tag before a numeric title still strips, because it
// now goes through the vocabulary instead of the raw-letter test.
expect(normalizeTitle('EX - 1917 (2019) 4K')).toBe('1917');
expect(normalizeTitle('NF - 1899 4K (2022)')).toBe('1899');
expect(normalizeTitle('EN - 180 - 2026 4K')).toBe('180');
});
it('leaves the wrapped-pipe form ungated', () => {
// A film name is never wrapped in pipes on both sides: all 174
// letterless cases in the catalog were genuine tags.
expect(normalizeTitle('|EN| 65')).toBe('65');
expect(normalizeTitle('|DE| 2067')).toBe('2067');
});
it('no longer normalizes a title down to a degenerate key', () => {
// Both used to collapse into piles of unrelated junk — "" with 72
// other members, and "2" with everything named after a numeral.
expect(normalizeTitle('DSP: (2022)')).toBe('dsp');
expect(normalizeTitle('HIT: 2 (2022)')).toBe('hit 2');
expect(normalizeTitle('EGO - (Erkeğe Güven Olmaz)')).toBe('ego');
});
});
it('keeps localized subtitles (indistinguishable from real ones)', () => {
expect(normalizeTitle('Breaking Bad: A Química do Mal')).toBe(
'breaking bad a quimica do mal'