mirror of
https://github.com/4gray/iptvnator.git
synced 2026-10-15 13:46:16 -08:00
fix(matching): keep abbreviation titles out of bare-year keys (#1426)
A leading 2-5 character uppercase token before a dash, pipe or colon was always read as a provider tag, so a film whose NAME is such a token lost it and normalized down to its release year alone. "AKA - 2023" became the key "2023", where it collided with BDE, BRO, OUT, WIL and IF — and they were offered to each other as alternative sources in VOD multi-source. "IT - 65 (2023)" (the Italian copy of the film "65") is structurally identical, so only the token's meaning can separate them. The leading token is now tested against a vocabulary — TRAILING_TAG_VOCABULARY plus a prefix-only list derived from the real catalog — but only when the strip would leave no real word behind. A compound is read by its head, so the open-ended "4K-<lang>" family keeps working while "INU-OH" and "PC-4L" are recognized as film names. An unknown token keeps its title: a refused strip costs one unmatched copy, a wrong one corrupts that film's identity in VOD multi-source, the TMDB Similar rail, DB_MATCH_TITLES and pin keys. "No real word" is decided by running the rest of the pipeline on the stripped form and looking at what comes out, never by re-implementing what later stages remove. Quality tags, trailing and underscore tags, double-dash suffixes and season markers each otherwise smuggle the strip through, and a stage added later is covered for free. Validated over the live catalog, movies and series: 83 keys fixed, 0 corrupted across 1,616,111 titles. Deriving the vocabulary from movies alone missed AMZ, D+ and P+ and broke the Paramount+/Disney+ copies of the numeric series 1923, 1883, 24 and 9-1-1.
This commit is contained in:
1 parent
30d76a3a46
commit
82a2948ffd
5 files changed
+379
-26
No files matched your search
@@ -298,6 +298,119 @@ describe('provider tag stripping', () => {
|
||||
}
|
||||
);
|
||||
|
||||
describe("leading tag vs. the film's own name", () => {
|
||||
// "IT - 65 (2023)" and "AKA - 2023" are the same shape: 2-5 uppercase
|
||||
// characters, a separator, digits. Only the token's MEANING separates
|
||||
// the Italian copy of the film "65" from the film "AKA" followed by
|
||||
// its year, so the strip is vocabulary-gated whenever nothing but
|
||||
// digits would survive it.
|
||||
|
||||
it('keeps a name that only looks like a tag before its year', () => {
|
||||
// Measured on the live catalog: these all normalized to a bare
|
||||
// year, so AKA/BDE/BRO/OUT/WIL/IF shared the single key "2023"
|
||||
// and were offered to each other as alternative sources.
|
||||
expect(normalizeTitle('AKA - 2023')).toBe('aka');
|
||||
expect(normalizeTitle('AKA | 2023')).toBe('aka');
|
||||
expect(normalizeTitle('IO - 2019')).toBe('io');
|
||||
expect(normalizeTitle('ARQ - 2016')).toBe('arq');
|
||||
expect(normalizeTitle('UFO - 2012')).toBe('ufo');
|
||||
expect(normalizeTitle('LBJ - 2017')).toBe('lbj');
|
||||
expect(normalizeTitle('EO - 2022')).toBe('eo');
|
||||
expect(normalizeTitle('BDE - 2023')).toBe('bde');
|
||||
expect(normalizeTitle('VFW - 2019 (4K HDR)')).toBe('vfw');
|
||||
expect(normalizeTitleKeys('AKA - 2023').exact).toBe('aka 2023');
|
||||
});
|
||||
|
||||
it('still strips real tags before a numeric film title', () => {
|
||||
// The opposite shape, and the reason a shape-only guard was
|
||||
// rejected: here the tag is real and the film's NAME is numeric.
|
||||
expect(normalizeTitle('IT - 65 (2023)')).toBe('65');
|
||||
expect(normalizeTitle('|FR|VO| 300 (2007)')).toBe('300');
|
||||
expect(normalizeTitle('|FR|VO| 1922')).toBe('1922');
|
||||
expect(normalizeTitle('OSN - 1917 - 2019')).toBe('1917');
|
||||
expect(normalizeTitle('EN - 42 - 2013')).toBe('42');
|
||||
expect(normalizeTitle('NRC - 7500 (2019)')).toBe('7500');
|
||||
expect(normalizeTitle('EXYU| 3022')).toBe('3022');
|
||||
});
|
||||
|
||||
it('strips streaming-provider tags before numeric series titles', () => {
|
||||
// Found only in the series catalog: a movie-only vocabulary
|
||||
// dropped these copies of 1923/1883/24/99 out of their groups.
|
||||
expect(normalizeTitle('AMZ - 99 (2024)')).toBe('99');
|
||||
expect(normalizeTitle('D+ - 24 (2001) (US)')).toBe('24');
|
||||
expect(normalizeTitle('D+ - 9-1-1 (2018) (US)')).toBe('9 1 1');
|
||||
expect(normalizeTitle('P+ - 1883 (2021)')).toBe('1883');
|
||||
});
|
||||
|
||||
it('reads a compound tag by its head, so names survive', () => {
|
||||
// "4K-<lang>" pairings are open-ended, so the head carries the
|
||||
// meaning — and it is also what tells a compound tag apart from
|
||||
// a hyphenated name.
|
||||
expect(normalizeTitle('4K-FR - 1992 (2024)')).toBe('1992');
|
||||
expect(normalizeTitle('AR-SUBS - 180 (2026)')).toBe('180');
|
||||
expect(normalizeTitle('SO-IN - 65 (2023)')).toBe('65');
|
||||
expect(normalizeTitle('INU-OH - 2022')).toBe('inu oh');
|
||||
expect(normalizeTitle('PC-4L - 2020')).toBe('pc 4l');
|
||||
});
|
||||
|
||||
it('asks the real pipeline what survives, not a list of its rules', () => {
|
||||
// Every later stage removes something, so a guard that predicts
|
||||
// them is a list to keep in sync. These are one case per stage,
|
||||
// and all of them fall out of running the pipeline and looking:
|
||||
expect(normalizeTitle('|TA| RRR - HEVC')).toBe('rrr'); // quality
|
||||
expect(normalizeTitle('CAT - Multi ENG')).toBe('cat'); // trailing
|
||||
expect(normalizeTitle('IF - 2024_sub')).toBe('if'); // underscore
|
||||
expect(normalizeTitle('AKA --xyz')).toBe('aka'); // double dash
|
||||
expect(normalizeTitle('CAT - 2022 S01')).toBe('cat 2022'); // season
|
||||
// …while a known tag before a numeric title still strips on every
|
||||
// one of those paths.
|
||||
expect(normalizeTitle('P+ - 1923 S01')).toBe('1923');
|
||||
expect(normalizeTitle('IT - 65 s01')).toBe('65');
|
||||
expect(normalizeTitle('EX - 1917 (2019) 4K')).toBe('1917');
|
||||
});
|
||||
|
||||
it('does not let a quality suffix smuggle the strip through', () => {
|
||||
// The suffix has letters only until QUALITY_TAGS removes them, so
|
||||
// testing the raw remainder would strip the name and normalize
|
||||
// "RRR - HEVC" to the EMPTY key — while "RRR - 2022" keys as
|
||||
// "rrr", hiding one copy of the film from the other.
|
||||
expect(normalizeTitle('|TA| RRR - HEVC')).toBe('rrr');
|
||||
expect(normalizeTitle('CAT - Multi')).toBe('cat');
|
||||
expect(normalizeTitle('RRR - 2022 - 4K')).toBe('rrr');
|
||||
expect(normalizeTitle('VFW - 2019 UHD')).toBe('vfw');
|
||||
// A TRAILING language tag is dropped a few lines later too, so it
|
||||
// is no more a word than a quality tag is. Real catalog titles:
|
||||
// "sub" made the remainder look meaningful and the key came out
|
||||
// as the bare year this whole guard exists to prevent.
|
||||
expect(normalizeTitle('IF - 2024_sub')).toBe('if');
|
||||
expect(normalizeTitle('O2 - 2024_sub')).toBe('o2');
|
||||
expect(normalizeTitle('UFO - 2022_sub')).toBe('ufo');
|
||||
expect(normalizeTitle('CAT - Multi ENG')).toBe('cat');
|
||||
// …but a tag word next to a real one is just part of the title.
|
||||
expect(normalizeTitle('EN - Sub Zero')).toBe('sub zero');
|
||||
// …and a real tag before a numeric title still strips, because it
|
||||
// now goes through the vocabulary instead of the raw-letter test.
|
||||
expect(normalizeTitle('EX - 1917 (2019) 4K')).toBe('1917');
|
||||
expect(normalizeTitle('NF - 1899 4K (2022)')).toBe('1899');
|
||||
expect(normalizeTitle('EN - 180 - 2026 4K')).toBe('180');
|
||||
});
|
||||
|
||||
it('leaves the wrapped-pipe form ungated', () => {
|
||||
// A film name is never wrapped in pipes on both sides: all 174
|
||||
// letterless cases in the catalog were genuine tags.
|
||||
expect(normalizeTitle('|EN| 65')).toBe('65');
|
||||
expect(normalizeTitle('|DE| 2067')).toBe('2067');
|
||||
});
|
||||
|
||||
it('no longer normalizes a title down to a degenerate key', () => {
|
||||
// Both used to collapse into piles of unrelated junk — "" with 72
|
||||
// other members, and "2" with everything named after a numeral.
|
||||
expect(normalizeTitle('DSP: (2022)')).toBe('dsp');
|
||||
expect(normalizeTitle('HIT: 2 (2022)')).toBe('hit 2');
|
||||
expect(normalizeTitle('EGO - (Erkeğe Güven Olmaz)')).toBe('ego');
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps localized subtitles (indistinguishable from real ones)', () => {
|
||||
expect(normalizeTitle('Breaking Bad: A Química do Mal')).toBe(
|
||||
'breaking bad a quimica do mal'
|
||||
|
||||
Reference in new issue
Block a user