Files
iptvnator/docs/architecture/stalker-mock-server.md
T
4grayandClaude Fable 5 297e9fbef8 fix(stalker): send cmd in the reference MAG wire format (#1334)
* fix(stalker): send cmd in the reference MAG wire format

A real MAG sends cmd unencoded and the portal decodes its query exactly
once, so a cmd that already contains percent sequences (%3A tokens,
pre-encoded path segments) must pass through untouched. The previous
encodeURIComponent transport (2c032cd3c, 0.22) double-encoded such cmds
(%3A -> %253A): strict portals and reseller panels that compare cmd
literally, and stock create_link handlers matching the decoded value,
saw a different string than a real STB sends.

The new shared encodeStalkerCmdValue() reproduces the reference wire
bytes: % passes through verbatim, characters the WHATWG URL serializer
keeps raw in a query stay raw (so the bytes survive the axios/new URL
transport unchanged), and everything else is percent-encoded. That
preserves the 0.22 injection protection - &, # (and ; for PHP setups
with a ; argument separator) inside cmd cannot append or truncate query
parameters; they decode back to the original byte server-side.

Both transports now share the format: the Electron query builder is
extracted to buildStalkerRequestUrl() and the web-backend /stalker
proxy appends cmd to the portal URL itself instead of letting axios
turn slashes into %2F (the opposite divergence).

Also unifies the two divergent response-side cmd normalizers: the
cross-portal collection resolver now uses the Stalker store's
normalizeStalkerPlaybackCommand/resolveStalkerPlaybackUrl, so playing
from Favorites/global collections resolves relative (/media/...) and
query-only (?token=...) create_link replies against the portal base
instead of handing the player a bare relative path.

The mock portal's create_link response gains mock-only cmd_received/
query_keys_received diagnostics; a new Electron e2e pins the contract
end-to-end (single decode, injection blocked). Unit corpus tests cover
the encoder, the Electron builder, the web-backend proxy, and the
resolver.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(pwa): sanitize portal URL before appending stalker cmd

A registered portal URL carrying a fragment would swallow the appended
cmd (everything after # is never transmitted), and a trailing bare '?'
produced '??cmd='. Drop the hash and pick the separator from the
sanitized href before appending. Flagged by Greptile/Codex on #1334.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(pwa): make stalker cmd append visibly query-only for CodeQL

Rebuild the /stalker request URL through the URL object and concatenate
the encoded cmd strictly behind a literal '?', so static analysis can
see the tainted value never reaches host or path (js/request-forgery
alert on the previous separator ternary). Behavior unchanged; the
fragment/bare-'?' regression tests still pin the wire format.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 07:38:11 +02:00

14 KiB
Raw Blame History

Stalker Mock Server Architecture

This document describes the design decisions, data flow, and extension points of the stalker-mock-server development tool.

Purpose

The mock server enables:

  1. Local development without access to a real Stalker portal
  2. Playwright E2E testing with predictable, deterministic data
  3. Scenario-based testing via predefined MAC addresses that map to specific data shapes
  4. Screenshot-safe marketing capture with committed fictional posters shared with the Xtream mock

Key Design Decisions

Seeded Determinism (Not Per-Request Random)

Per-request random data would break navigation: if category IDs change between calls, content fetched under a category ID won't match the category list. Instead:

  • Data is generated once per MAC address on first request, then cached in memory.
  • @faker-js/faker is seeded with the scenario's seed value before generation: predefined scenario MACs use fixed seeds from scenarios.ts; unknown MACs derive the seed from the MAC via macToSeed().
  • Same MAC → identical data on every server restart.
  • Restart the server to reshuffle all data.

MAC Address as Identity

Stalker portals use MAC address as the primary credential. The mock server follows the same model:

  • Each unique MAC gets its own isolated dataset.
  • Predefined MACs map to specific ScenarioConfig shapes (see src/app/scenarios.ts).
  • Unknown MACs use the sum of their byte values as a seed, producing unique but deterministic data.

In-Memory Only

No files or databases are written. All state (generated content + favorites + portal sessions) lives in process memory and resets on server restart. This is intentional — tests should not share state across runs.

Two Endpoints With Different Strictness

The app decides how to talk to a portal from the shape of its URL: a URL containing /stalker_portal is imported as a full portal (handshake, Authorization: Bearer, watchdog), anything else as a simple portal with no authentication at all. The mock therefore serves the same action set at two paths:

Path Router Behaviour
/portal.php createPortalRouter(false) Tolerant: ignores the token and the MAC format, like most reseller panels
/stalker_portal/server/load.php createPortalRouter(true) Strict: enforces both, like the real middleware
/server/load.php createPortalRouter(true) Strict: the second URL shape isFullStalkerPortal recognizes

The /stalker proxy route applies the same rule through isFullPortalUrlShape() — every URL the client would authenticate against is enforced, so tests cannot silently fall into the tolerant branch.

Keeping the tolerant path is what lets the pre-existing e2e suite (which imports portal.php) stay meaningful — it covers the simple-portal branch — while the strict path finally covers the authenticated branch that had no coverage at all.

The strict behaviours mirror the plaintext Stalker 4.9.35 middleware (server/lib/stb.class.php), the last openly readable ancestor of the encoded 5.x core:

  • Plain-text auth failures. Authorization failed. / Unauthorized request. are returned with HTTP 200 and a text/html body, because the real server exits before the JSON envelope is built. A client checking only status codes sees "success" and renders nothing. The /stalker proxy route still wraps the body in the { payload } envelope, matching what apps/web-backend does.

  • A handshake is not a session. The token only authorizes requests once get_profile has adopted it for that MAC. Adoption is deliberately stricter than the stock server: 4.9.35 issues handshake tokens statelessly and pins whatever Bearer get_profile presents, so a forged token would become a session on a real portal — the mock only adopts tokens it actually issued, so a client with a broken token pipeline fails loudly in tests.

  • Idempotent handshake. Presenting the MAC's current token returns that same token, which is what allows real clients to persist tokens across restarts.

  • Device-id pinning. device_id/device_id2 are stored on first non-empty value; any later change — including reverting to empty — is a permanent device conflict carrying the "Your STB is damaged." block message. This is the only identity check the stock server actually enforces.

  • signature, metrics, prehash are ignored, exactly as upstream ignores them; they exist for portals with a custom access_filter.php.

  • MAC format validation. Non-Infomir MACs (00:1A:79:XX:XX:XX) get a bare { status: 1 } from get_profile.

  • do_auth is a boolean login step. Non-empty credentials answer {js:true} and are recorded; the login-required scenario's get_profile keeps answering status: 2 until that record exists, because the app sends auth_second_step=1 on its very first profile request and a parameter check alone would be trivially bypassed.

Session state lives in src/app/auth-store.ts and is cleared by /reset. POST /invalidate-session?macAddress=<mac> drops a single MAC's tokens so tests can assert the client re-handshakes and retries instead of surfacing an error; pinned device identity survives invalidation, as it does on a real portal.

Data Generation Pipeline

faker.seed(config.seed)   // scenario seed; unknown MACs: macToSeed(mac)
│
├── generateCategories('itv', N)  → itvCategories[]
│     └── generateChannels()      → channels Map<categoryId, channel[]>
│           └── generateEpg()     → epg Map<channelId, program[]>
│
├── generateCategories('radio', N) → radioCategories[]
│     └── generateRadioStations()  → radio Map<categoryId, station[]>
│
├── generateCategories('vod', N)  → vodCategories[]
│     └── generateVodItems()      → vod Map<categoryId, item[]>
│           ├── normal VOD items
│           ├── is_series=1 items (fraction, Ministra flow)
│           └── embedded series[] items (fraction)
│
├── marketingFixture?             → shared curated VOD catalog
│     ├── @iptvnator/shared/marketing-fixtures
│     ├── vod Map<categoryId, item[]>
│     └── vodOrder[] preserves screenshot catalog order
│
└── generateCategories('series', N) → seriesCategories[]
      └── generateSeriesItems()    → series Map<categoryId, item[]>
            └── generateSeasons()  → seasons Map<seriesItemId, season[]>

Response Shapes

All responses follow the Stalker portal.php envelope:

{ "js": <action-specific payload> }

get_categories

{
  "js": [
    { "id": "2001", "title": "Action", "alias": "action" },
    ...
  ]
}

get_ordered_list (content)

{
  "js": {
    "data": [
      {
        "id": "20001",
        "name": "...",
        "cmd": "ffrt4://vod/20001/index.m3u8",
        "screenshot_uri": "https://picsum.photos/seed/vod-20001/300/200",
        "cover": "https://picsum.photos/seed/vod-cover-20001/300/450",
        "description": "...",
        "actors": "...",
        "director": "...",
        "year": "2019",
        "rating_imdb": "7.3",
        "category_id": "2001",
        "is_series": 0,
        "has_files": 1
      }
    ],
    "total_items": 40,
    "max_page_items": 14,
    "cur_page": 1,
    "total_pages": 3
  }
}

For type=radio, items use the same paginated response envelope as live TV channels, but each item is generated as a radio station with a radio: true marker and an ffrt4://radio/... command.

get_ordered_list (seasons — when movie_id is present)

{
  "js": [
    {
      "id": "30001-s1",
      "name": "Season 1",
      "cmd": "ffrt4://series/30001/season/1",
      "series": ["1", "2", "3", ...],
      "screenshot_uri": "https://picsum.photos/seed/30001-s1/300/200",
      "director": "...",
      "actors": "...",
      "year": "2021",
      "rating_imdb": "8.1"
    }
  ]
}
{
  "js": {
    "cmd": "https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8",
    "streamer_id": "1",
    "load": "",
    "error": "",
    "cmd_received": "ffrt4://ch/live/1001/index.m3u8",
    "query_keys_received": ["JsHttpRequest", "action", "cmd", "type"]
  }
}

The stream URL is selected from a pool of 4 real public HLS test streams. The choice is deterministic based on the cmd field's character sum, so the same item always returns the same stream.

cmd_received and query_keys_received are mock-only diagnostics (a real portal does not send them): they echo the request's cmd after Express' single query decode — the same view a PHP portal gets from $_GET — plus the sorted set of query keys. E2E uses them to pin the client's cmd wire contract: no double-encoding, and no query-parameter injection through cmd.

get_short_epg

{
  "js": {
    "data": [
      {
        "id": "1",
        "name": "Channel Name: Program Title",
        "start": "2026-02-21T10:00:00.000Z",
        "stop": "2026-02-21T12:00:00.000Z",
        "start_timestamp": 1740128400,
        "stop_timestamp": 1740135600,
        "descr": "...",
        "category": "News"
      }
    ]
  }
}

get_short_epg returns the current program and upcoming items from the generated schedule, limited by the requested size.

get_epg_info

{
  "js": {
    "data": {
      "10000": [
        {
          "id": "1",
          "name": "Channel Name: Program Title",
          "start": "2026-02-21T10:00:00.000Z",
          "stop": "2026-02-21T12:00:00.000Z",
          "start_timestamp": 1740128400,
          "stop_timestamp": 1740135600,
          "descr": "...",
          "category": "News"
        }
      ]
    }
  }
}

get_epg_info returns bulk EPG keyed by channel id and filters the generated 7-day schedule from the current UTC day start through now + period.

EPG programs are generated as 2-hour slots across 7 days for each channel, starting at the current UTC day boundary.

Scenarios

Scenarios are defined in src/app/scenarios.ts. Each scenario is a ScenarioConfig:

interface ScenarioConfig {
  name: string;
  description: string;
  seed: number;
  categoryCount: { itv: number; radio: number; vod: number; series: number };
  itemsPerCategory: number;
  seasonsPerSeries: number;
  episodesPerSeason: number;
  isSeriesFraction: number;      // 0–1: fraction of VOD with is_series=1
  embeddedSeriesFraction: number; // 0–1: fraction of VOD with embedded series[]
  supportsGetAllChannels?: boolean; // default true; false mimics legacy portals
                                    // without the ITV get_all_channels action
  marketingFixture?: true;          // replace generated VOD with shared posters
}

The legacy-pagination scenario (00:1A:79:00:00:06) sets supportsGetAllChannels: false: get_all_channels then answers with an error payload so clients fall back to the paginated get_ordered_list crawl. For supporting scenarios, get_all_channels (get-all-channels.handler.ts, type=itv only) returns the complete ITV channel list in one { js: { data, total_items } } response, excluding channels from censored (adult) genres.

The marketing-demo scenario (00:1A:79:00:00:07) replaces faker-generated VOD with the 35-movie provider-neutral showcase catalog from @iptvnator/shared/marketing-fixtures. Its newest 20 movies are returned first for the wildcard VOD listing. Fixtures keep deployment-neutral poster paths, then the JSON middleware resolves them against the request origin (including forwarded host/protocol) as <portal-origin>/assets/marketing/poster/<slug>.png. main.ts serves the committed PNG directory directly, so the Xtream server does not need to run.

Adding a New Scenario

  1. Add an entry to the SCENARIOS map in src/app/scenarios.ts.
  2. Use any unique MAC address as the key (lowercase, colon-separated).
  3. Document it in README.md and this file.

Favorites

Favorites are stored in a Map<mac, Set<itemId>> in src/app/data-store.ts. They persist for the lifetime of the server process and are shared across all requests for the same MAC.

Call POST /reset to clear all favorites (and regenerated data) between test runs.

Playwright Integration

apps/web-e2e/playwright.config.ts registers the mock server as a second webServer entry:

webServer: [
  {
    command: 'pnpm nx run web:serve',
    url: 'http://localhost:4200',
    reuseExistingServer: !process.env['CI'],
  },
  {
    command: 'pnpm nx run stalker-mock-server:serve',
    url: 'http://localhost:3210/health',
    reuseExistingServer: !process.env['CI'],
  },
]

Playwright waits for both servers to be healthy before starting tests. If either is already running (e.g. in local dev), it reuses the existing instance.

Test Isolation

Each stalker e2e test calls POST http://localhost:3210/reset in beforeEach to clear in-memory state. This ensures tests don't bleed favorites or other mutable state into each other.

resetAll() clears both the generated-content cache and in-memory favorites (data-store.ts). Because generation is seed-deterministic, the next request regenerates identical content, so the observable data does not change across resets.

import { test, expect } from '@playwright/test';

const MOCK_URL = 'http://localhost:3210/portal.php';
const MOCK_MAC = '00:1A:79:00:00:01'; // default scenario

test.beforeEach(async ({ request }) => {
  await request.post('http://localhost:3210/reset');
});

test('browse VOD categories', async ({ page }) => {
  // Add portal via UI or programmatically via IndexedDB
  // Navigate to portal
  // Assert category list matches expected count (8 for default scenario)
});

Extension Points

  • New content types: Add a new generator function in data-generator.ts and a new handler in handlers/.
  • New scenarios: Add to SCENARIOS in scenarios.ts.
  • Session behaviour: auth-store.ts owns tokens and device pinning. Add TTLs or a "token replaced by another device" mode there rather than in the handlers.
  • Error simulation: Add a special MAC or query param to trigger error responses for testing error handling in the Stalker store. Note that portal-level auth errors are not HTTP errors — see Two Endpoints With Different Strictness.
  • Slow responses: Add a MOCK_DELAY_MS env var and apply it in middleware for testing loading states.