From 1e027b8e400ee936f67290872a9886c2e753912e Mon Sep 17 00:00:00 2001 From: 4gray Date: Thu, 30 Jul 2026 11:18:21 +0200 Subject: [PATCH] test(skills): validate repository skill contracts --- AGENTS.md | 46 +-- CLAUDE.md | 8 +- package.json | 1 + tools/skills/project.json | 28 ++ tools/skills/validate-repository-skills.mjs | 287 +++++++++++++++++ .../validate-repository-skills.test.mjs | 301 ++++++++++++++++++ 6 files changed, 641 insertions(+), 30 deletions(-) create mode 100644 tools/skills/project.json create mode 100644 tools/skills/validate-repository-skills.mjs create mode 100644 tools/skills/validate-repository-skills.test.mjs diff --git a/AGENTS.md b/AGENTS.md index 1825f96f9..d27ab57a6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -18,7 +18,13 @@ This file provides guidance to coding agents working in this repository. - See `docs/architecture/nx-workspace-boundaries.md` for the current Nx tag and alias policy. - ESLint enforces `max-lines` on TypeScript files: production code targets under 300 with a hard maximum of 400, while tests (`**/*.spec.ts`, `**/*.e2e.ts`, `apps/*-e2e/**`) are held to 1200 — a long spec signals coverage, not the design debt the production limit catches. Blank lines and comments are not counted, so a docblock never forces a split. Limits live in `tools/eslint/max-lines-config.mjs`, imported by both `eslint.config.mjs` and the generator so the rule and the baseline cannot drift. Files that predate the rule are baselined in `tools/eslint/max-lines-baseline.mjs`; after splitting a file, regenerate it with `node tools/eslint/generate-max-lines-baseline.mjs` (it runs ESLint's own rule rather than counting lines itself). Never add new files to the baseline — the list must only shrink. A new file that genuinely cannot be split (for example a function serialized into another process) instead carries its own file-wide `/* eslint-disable max-lines -- */`; the generator skips those files, so a justified exemption never lands in the baseline. Remove such a directive once ESLint reports it as unused. - Project `lint` targets that shell out to eslint must quote the glob, e.g. `eslint "apps//**/*.ts"`. An unquoted `**` is expanded by the POSIX shell on Linux and macOS (which has no `globstar`, so it matches only a shallow subset of files) while Windows passes the literal pattern to ESLint, which expands it recursively — the two hosts then lint different file sets. The target still reports success either way, so a broken glob hides missing coverage instead of failing. After changing such a target, compare the linted file count against `find -name '*.ts' | wc -l`. -- Repository-specific skills are committed under `.codex/skills/`. Claude Code only discovers skills under `.claude/skills/`, so `release-notes` and `release-cut` are mirrored there and the two copies must be kept in sync; every other entry in `.claude/skills/` is personal and stays gitignored. If an external agent does not support skills, treat those files as concise ownership docs. +- Repository-specific skills live under `.codex/skills/`. +- Frontmatter descriptions are trigger-only and begin with `Use when`; keep + each skill at or below 500 words. +- Run `pnpm run skills:validate` after editing a committed skill or a literal + path it documents. +- Keep `.codex` and `.claude` copies of `release-notes` and `release-cut` + byte-identical. ## Documentation After Changes @@ -410,35 +416,17 @@ Key files: ## Repo Skills -- `iptvnator-ui-design` - Repository-specific UI design guidance for IPTVnator. - Use when working on channel rows, EPG views, settings surfaces, shared selection styles, or light/dark theme consistency. - File: `.codex/skills/iptvnator-ui-design/SKILL.md` +- `.codex/skills/iptvnator-nx-architecture/SKILL.md` +- `.codex/skills/iptvnator-sqlite-db-worker/SKILL.md` +- `.codex/skills/iptvnator-theme-style/SKILL.md` +- `.codex/skills/iptvnator-ui-design/SKILL.md` +- `.codex/skills/release-cut/SKILL.md` +- `.codex/skills/release-notes/SKILL.md` +- `.codex/skills/stalker-portal/SKILL.md` +- `.codex/skills/xtream-electron/SKILL.md` -- `iptvnator-theme-style` - Theme architecture, design token reference, shared SCSS library, portal header pattern, Electron drag region, and common styling mistakes. - Use when adding/changing CSS tokens, styling portal headers or sidebars, using shared SCSS mixins (`portal-layout`, `content-grid`, `portal-sidebar`), or auditing cross-portal visual consistency. - File: `.codex/skills/iptvnator-theme-style/SKILL.md` - -- `iptvnator-nx-architecture` - Repository-specific Nx monorepo structure, library placement rules, path alias guidance, and migration guardrails for portal/workspace/app code. - Use when deciding where code belongs, extracting code into libs, choosing imports, or refactoring Xtream/Stalker/Workspace boundaries. - File: `.codex/skills/iptvnator-nx-architecture/SKILL.md` - -- `iptvnator-sqlite-db-worker` - Repository-specific guidance for the Electron non-EPG SQLite worker, including worker boundaries, request-scoped DB progress events, and validation steps for slow DB operations. - Use when moving heavy database work off the main thread, adding worker-backed SQLite operations, or wiring loading/progress UI for Xtream and playlist DB flows. - File: `.codex/skills/iptvnator-sqlite-db-worker/SKILL.md` - -- `stalker-portal` - Repository-specific guidance for Stalker/Ministra catalogs, all three VOD/series modes, cross-surface `is_series` behavior, playback metadata, collections, EPG, and remote control. - Use when changing Stalker routes, stores, detail views, playback, favorites/recent activity, EPG, or remote control. - File: `.codex/skills/stalker-portal/SKILL.md` - -- `xtream-electron` - Repository-specific guidance for IPTVnator's Electron-first Xtream implementation, including feature/data-access boundaries, worker-backed DB flows, and Xtream loading/progress UX expectations. - Use when working on Xtream routes, store/data-source logic, or Electron-backed Xtream import/search/delete behavior. - File: `.codex/skills/xtream-electron/SKILL.md` +Descriptions and trigger conditions are canonical in each skill's frontmatter; +do not duplicate them here. diff --git a/CLAUDE.md b/CLAUDE.md index 72ccc3aa7..4678a75ae 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -74,7 +74,13 @@ pnpm nx show projects - Do not add new imports from legacy bare aliases such as `services`, `shared-interfaces`, `components`, `m3u-state`, or `database`. - Every Nx project should keep `scope:*`, `domain:*`, and `type:*` tags in `project.json`. - See `docs/architecture/nx-workspace-boundaries.md` for the current Nx tag and alias policy. -- Repository-specific skills are committed under `.codex/skills/`. Claude Code only discovers skills under `.claude/skills/`, so `release-notes` and `release-cut` are mirrored there and the two copies must be kept in sync; every other entry in `.claude/skills/` is personal and stays gitignored. If an agent does not load skills directly, treat those files as concise ownership docs. +- Repository-specific skills live under `.codex/skills/`. +- Frontmatter descriptions are trigger-only and begin with `Use when`; keep + each skill at or below 500 words. +- Run `pnpm run skills:validate` after editing a committed skill or a literal + path it documents. +- Keep `.codex` and `.claude` copies of `release-notes` and `release-cut` + byte-identical. ### Building and Serving diff --git a/package.json b/package.json index 23215675c..3c6b8f80e 100644 --- a/package.json +++ b/package.json @@ -58,6 +58,7 @@ "serve:website": "nx serve website", "build:website": "nx build website", "i18n:check": "node tools/i18n/check-drift.mjs", + "skills:validate": "node tools/skills/validate-repository-skills.mjs", "release:artwork:dry-run": "tsx --tsconfig tsconfig.base.json tools/release/generate-marketing-artwork.ts --dry-run", "release:artwork:manifest": "tsx --tsconfig tsconfig.base.json tools/release/generate-marketing-artwork.ts --manifest", "release:artwork:generate": "tsx --tsconfig tsconfig.base.json tools/release/generate-marketing-artwork.ts --generate", diff --git a/tools/skills/project.json b/tools/skills/project.json new file mode 100644 index 000000000..8b6468d6a --- /dev/null +++ b/tools/skills/project.json @@ -0,0 +1,28 @@ +{ + "name": "repository-skills", + "$schema": "../../node_modules/nx/schemas/project-schema.json", + "projectType": "library", + "sourceRoot": "tools/skills", + "tags": ["scope:tools", "domain:skills", "type:tool"], + "targets": { + "test": { + "executor": "nx:run-commands", + "cache": true, + "inputs": [ + "{workspaceRoot}/tools/skills/validate-repository-skills.mjs", + "{workspaceRoot}/tools/skills/validate-repository-skills.test.mjs" + ], + "options": { + "command": "node --test tools/skills/validate-repository-skills.test.mjs", + "cwd": "{workspaceRoot}" + } + }, + "lint": { + "executor": "nx:run-commands", + "options": { + "command": "node --check tools/skills/validate-repository-skills.mjs", + "cwd": "{workspaceRoot}" + } + } + } +} diff --git a/tools/skills/validate-repository-skills.mjs b/tools/skills/validate-repository-skills.mjs new file mode 100644 index 000000000..8f0b26b5f --- /dev/null +++ b/tools/skills/validate-repository-skills.mjs @@ -0,0 +1,287 @@ +import { access, readdir, readFile } from 'node:fs/promises'; +import { isAbsolute, relative, resolve, sep } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const SKILLS_ROOT = '.codex/skills'; +const MIRRORED_SKILLS = ['release-cut', 'release-notes']; +const PATH_PREFIXES = [ + 'apps/', + 'libs/', + 'docs/', + 'tools/', + '.changes/', + '.github/', +]; +const ROOT_PATHS = new Set([ + 'package.json', + 'pnpm-lock.yaml', + 'nx.json', + 'tsconfig.base.json', + 'eslint.config.mjs', + 'CHANGELOG.md', + 'AGENTS.md', + 'CLAUDE.md', +]); +const NON_LITERAL_PATH_CHARACTERS = /[*?[\]{}<>]/u; + +function compareNames(left, right) { + if (left < right) return -1; + if (left > right) return 1; + return 0; +} + +function unquoteScalar(value) { + if (value.length < 2) return value; + + const firstCharacter = value[0]; + const lastCharacter = value.at(-1); + const hasMatchingQuotes = + (firstCharacter === '"' || firstCharacter === "'") && + lastCharacter === firstCharacter; + + return hasMatchingQuotes ? value.slice(1, -1) : value; +} + +function parseFrontmatter(markdown) { + const lines = markdown.split(/\r?\n/u); + if (lines[0] !== '---') { + return { fields: {}, continuedFields: new Set() }; + } + + const closingDelimiter = lines.indexOf('---', 1); + if (closingDelimiter === -1) { + return { fields: {}, continuedFields: new Set() }; + } + + const fields = {}; + const continuedFields = new Set(); + let activeField; + + for (const line of lines.slice(1, closingDelimiter)) { + if (line.trim() === '') continue; + + if (/^[ \t]/u.test(line)) { + if (activeField !== undefined) { + continuedFields.add(activeField); + } + continue; + } + + const separator = line.indexOf(':'); + if (separator <= 0) { + activeField = undefined; + continue; + } + + const key = line.slice(0, separator).trim(); + const value = line.slice(separator + 1).trim(); + fields[key] = unquoteScalar(value); + activeField = key; + } + + return { fields, continuedFields }; +} + +function countWords(markdown) { + const trimmedMarkdown = markdown.trim(); + return trimmedMarkdown === '' + ? 0 + : trimmedMarkdown.split(/\s+/u).length; +} + +function findLiteralRepositoryPaths(markdown) { + const paths = new Set(); + + for (const match of markdown.matchAll(/`([^`\r\n]+)`/gu)) { + const token = match[1]; + const isRepositoryPath = + ROOT_PATHS.has(token) || + PATH_PREFIXES.some((prefix) => token.startsWith(prefix)); + + if (isRepositoryPath && !NON_LITERAL_PATH_CHARACTERS.test(token)) { + paths.add(token); + } + } + + return [...paths].sort(compareNames); +} + +async function pathExists(path) { + try { + await access(path); + return true; + } catch (error) { + if (error?.code === 'ENOENT' || error?.code === 'ENOTDIR') { + return false; + } + throw error; + } +} + +function isPathWithinRoot(rootDir, absolutePath) { + const relativePath = relative(resolve(rootDir), absolutePath); + return ( + relativePath === '' || + (!isAbsolute(relativePath) && + relativePath !== '..' && + !relativePath.startsWith(`..${sep}`)) + ); +} + +async function readFileIfPresent(path) { + try { + return await readFile(path); + } catch (error) { + if (error?.code === 'ENOENT') return undefined; + throw error; + } +} + +async function validateSkill({ rootDir, directoryName }) { + const relativePath = `${SKILLS_ROOT}/${directoryName}/SKILL.md`; + const markdown = await readFile(resolve(rootDir, relativePath), 'utf8'); + const { fields: frontmatter, continuedFields } = + parseFrontmatter(markdown); + const diagnostics = []; + + if (frontmatter.name !== directoryName) { + diagnostics.push( + `${relativePath}: frontmatter name "${frontmatter.name ?? ''}" must match directory "${directoryName}"` + ); + } + + if ( + typeof frontmatter.description !== 'string' || + frontmatter.description === '' || + continuedFields.has('description') + ) { + diagnostics.push( + `${relativePath}: description must be a one-line frontmatter value` + ); + } else { + if (!frontmatter.description.startsWith('Use when')) { + diagnostics.push( + `${relativePath}: description must start with "Use when"` + ); + } + if (frontmatter.description.length > 500) { + diagnostics.push( + `${relativePath}: description must be at most 500 characters (received ${frontmatter.description.length})` + ); + } + } + + const wordCount = countWords(markdown); + if (wordCount > 500) { + diagnostics.push( + `${relativePath}: skill must be at most 500 words (received ${wordCount})` + ); + } + + for (const referencedPath of findLiteralRepositoryPaths(markdown)) { + const absolutePath = resolve(rootDir, referencedPath); + if (!isPathWithinRoot(rootDir, absolutePath)) { + diagnostics.push( + `${relativePath}: referenced path escapes repository root: ${referencedPath}` + ); + continue; + } + if (!(await pathExists(absolutePath))) { + diagnostics.push( + `${relativePath}: referenced path does not exist: ${referencedPath}` + ); + } + } + + return diagnostics; +} + +async function validateReleaseMirrors(rootDir) { + const diagnostics = []; + + for (const skillName of MIRRORED_SKILLS) { + const codexPath = `.codex/skills/${skillName}/SKILL.md`; + const claudePath = `.claude/skills/${skillName}/SKILL.md`; + const [codexContents, claudeContents] = await Promise.all([ + readFileIfPresent(resolve(rootDir, codexPath)), + readFileIfPresent(resolve(rootDir, claudePath)), + ]); + + if (codexContents === undefined) { + diagnostics.push( + `${codexPath}: required release skill mirror is missing` + ); + } + if (claudeContents === undefined) { + diagnostics.push( + `${claudePath}: required release skill mirror is missing` + ); + } + if ( + codexContents !== undefined && + claudeContents !== undefined && + !codexContents.equals(claudeContents) + ) { + diagnostics.push(`${codexPath}: differs from ${claudePath}`); + } + } + + return diagnostics; +} + +export async function validateRepositorySkills({ rootDir }) { + const directoryEntries = (await readdir(resolve(rootDir, SKILLS_ROOT), { + withFileTypes: true, + })) + .filter((entry) => entry.isDirectory()) + .sort((left, right) => compareNames(left.name, right.name)); + const skillDirectories = []; + const diagnostics = []; + + for (const entry of directoryEntries) { + if ( + await pathExists( + resolve(rootDir, SKILLS_ROOT, entry.name, 'SKILL.md') + ) + ) { + skillDirectories.push(entry.name); + } + } + + for (const directoryName of skillDirectories) { + diagnostics.push( + ...(await validateSkill({ rootDir, directoryName })) + ); + } + + diagnostics.push(...(await validateReleaseMirrors(rootDir))); + + return { + checkedSkills: skillDirectories.length, + diagnostics, + }; +} + +async function runCli() { + const { checkedSkills, diagnostics } = await validateRepositorySkills({ + rootDir: process.cwd(), + }); + + if (diagnostics.length > 0) { + for (const diagnostic of diagnostics) { + console.error(diagnostic); + } + process.exitCode = 1; + return; + } + + console.log(`Validated ${checkedSkills} repository skills.`); +} + +const isCli = + process.argv[1] !== undefined && + resolve(process.argv[1]) === fileURLToPath(import.meta.url); + +if (isCli) { + await runCli(); +} diff --git a/tools/skills/validate-repository-skills.test.mjs b/tools/skills/validate-repository-skills.test.mjs new file mode 100644 index 000000000..ab5eb1ee4 --- /dev/null +++ b/tools/skills/validate-repository-skills.test.mjs @@ -0,0 +1,301 @@ +import assert from 'node:assert/strict'; +import { + mkdir, + mkdtemp, + rm, + symlink, + writeFile, +} from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { basename, dirname, join } from 'node:path'; +import test from 'node:test'; + +import { validateRepositorySkills } from './validate-repository-skills.mjs'; + +const validReleaseSkills = { + 'release-cut': `--- +name: release-cut +description: Use when preparing a repository release. +--- + +# Release Cut +`, + 'release-notes': `--- +name: release-notes +description: Use when deciding whether a change needs release notes. +--- + +# Release Notes +`, +}; + +async function writeFixtureFile(rootDir, relativePath, contents) { + const absolutePath = join(rootDir, relativePath); + await mkdir(dirname(absolutePath), { recursive: true }); + await writeFile(absolutePath, contents); +} + +async function createRepository( + t, + { + directoryName = 'example-skill', + name = directoryName, + description = + '"Use when deciding whether type: internal applies to a change."', + body = [ + '# Example Skill', + '', + 'Inspect `package.json` and `tools/existing.mjs`.', + 'Examples such as `apps/*-e2e` and `libs//feature` are not literal paths.', + '', + ].join('\n'), + releaseMirrorTransform = (contents) => contents, + } = {} +) { + const rootDir = await mkdtemp(join(tmpdir(), 'repository-skills-')); + t.after(() => rm(rootDir, { recursive: true, force: true })); + + await writeFixtureFile(rootDir, 'package.json', '{}\n'); + await writeFixtureFile(rootDir, 'tools/existing.mjs', 'export {};\n'); + await writeFixtureFile( + rootDir, + '.codex/skills/not-a-skill/README.md', + 'No SKILL.md lives here.\n' + ); + await writeFixtureFile( + rootDir, + `.codex/skills/${directoryName}/SKILL.md`, + `--- +name: ${name} +description: ${description} +--- + +${body}` + ); + await writeFixtureFile( + rootDir, + '.codex/skills/single-quoted-skill/SKILL.md', + `--- +name: single-quoted-skill +description: 'Use when deciding whether type: internal applies to a release.' +--- + +# Single-Quoted Skill +` + ); + + for (const [skillName, contents] of Object.entries(validReleaseSkills)) { + await writeFixtureFile( + rootDir, + `.codex/skills/${skillName}/SKILL.md`, + contents + ); + await writeFixtureFile( + rootDir, + `.claude/skills/${skillName}/SKILL.md`, + releaseMirrorTransform(contents, skillName) + ); + } + + return rootDir; +} + +test('accepts single- and double-quoted colon-space descriptions and skips path examples', async (t) => { + const rootDir = await createRepository(t); + + const result = await validateRepositorySkills({ rootDir }); + + assert.deepEqual(result, { + checkedSkills: 4, + diagnostics: [], + }); +}); + +test('reports a frontmatter name that differs from its directory', async (t) => { + const rootDir = await createRepository(t, { name: 'different-name' }); + + const { diagnostics } = await validateRepositorySkills({ rootDir }); + + assert.deepEqual(diagnostics, [ + '.codex/skills/example-skill/SKILL.md: frontmatter name "different-name" must match directory "example-skill"', + ]); +}); + +test('reports a description that does not begin with Use when', async (t) => { + const rootDir = await createRepository(t, { + description: 'Repository-specific implementation guidance.', + }); + + const { diagnostics } = await validateRepositorySkills({ rootDir }); + + assert.deepEqual(diagnostics, [ + '.codex/skills/example-skill/SKILL.md: description must start with "Use when"', + ]); +}); + +test('reports a description longer than 500 characters', async (t) => { + const description = `Use when ${'x'.repeat(493)}`; + assert.equal(description.length, 502); + const rootDir = await createRepository(t, { description }); + + const { diagnostics } = await validateRepositorySkills({ rootDir }); + + assert.deepEqual(diagnostics, [ + '.codex/skills/example-skill/SKILL.md: description must be at most 500 characters (received 502)', + ]); +}); + +test('rejects an indented plain-scalar description continuation', async (t) => { + const rootDir = await createRepository(t, { + description: `Use when editing a skill.\n ${'x'.repeat(501)}`, + }); + + const { diagnostics } = await validateRepositorySkills({ rootDir }); + + assert.deepEqual(diagnostics, [ + '.codex/skills/example-skill/SKILL.md: description must be a one-line frontmatter value', + ]); +}); + +test('rejects a blank-separated plain-scalar description continuation', async (t) => { + const rootDir = await createRepository(t, { + description: `Use when editing a skill.\n\n ${'x'.repeat(501)}`, + }); + + const { diagnostics } = await validateRepositorySkills({ rootDir }); + + assert.deepEqual(diagnostics, [ + '.codex/skills/example-skill/SKILL.md: description must be a one-line frontmatter value', + ]); +}); + +test('allows a blank frontmatter separator without treating it as a continuation', async (t) => { + const rootDir = await createRepository(t, { + description: 'Use when editing a skill.\n ', + }); + + const { diagnostics } = await validateRepositorySkills({ rootDir }); + + assert.deepEqual(diagnostics, []); +}); + +test('reports a complete skill longer than 500 words', async (t) => { + const rootDir = await createRepository(t, { + body: `# Example Skill\n\n${Array.from({ length: 501 }, () => 'word').join(' ')}`, + }); + + const { diagnostics } = await validateRepositorySkills({ rootDir }); + + assert.equal(diagnostics.length, 1); + assert.match( + diagnostics[0], + /^\.codex\/skills\/example-skill\/SKILL\.md: skill must be at most 500 words \(received \d+\)$/ + ); +}); + +test('reports a backticked literal repository path that does not exist', async (t) => { + const rootDir = await createRepository(t, { + body: '# Example Skill\n\nInspect `docs/missing-file.md`.', + }); + + const { diagnostics } = await validateRepositorySkills({ rootDir }); + + assert.deepEqual(diagnostics, [ + '.codex/skills/example-skill/SKILL.md: referenced path does not exist: docs/missing-file.md', + ]); +}); + +test('rejects a literal path that escapes the repository even when it exists', async (t) => { + const rootDir = await createRepository(t); + const outsideName = `${basename(rootDir)}-outside.md`; + const outsidePath = join(dirname(rootDir), outsideName); + const referencedPath = `docs/../../${outsideName}`; + t.after(() => rm(outsidePath, { force: true })); + await writeFile(outsidePath, 'outside\n'); + await writeFixtureFile( + rootDir, + '.codex/skills/example-skill/SKILL.md', + `--- +name: example-skill +description: Use when validating a literal path. +--- + +Inspect \`${referencedPath}\`. +` + ); + + const { diagnostics } = await validateRepositorySkills({ rootDir }); + + assert.deepEqual(diagnostics, [ + `.codex/skills/example-skill/SKILL.md: referenced path escapes repository root: ${referencedPath}`, + ]); +}); + +test( + 'propagates non-missing path access failures', + { skip: process.platform === 'win32' }, + async (t) => { + const rootDir = await createRepository(t, { + body: '# Example Skill\n\nInspect `docs/loop/file.md`.', + }); + await mkdir(join(rootDir, 'docs')); + await symlink('loop', join(rootDir, 'docs/loop')); + + await assert.rejects( + validateRepositorySkills({ rootDir }), + (error) => { + assert.equal(error.code, 'ELOOP'); + return true; + } + ); + } +); + +test('reports byte-different release skill mirrors', async (t) => { + const rootDir = await createRepository(t, { + releaseMirrorTransform: (contents, skillName) => + skillName === 'release-notes' ? `${contents}\n` : contents, + }); + + const { diagnostics } = await validateRepositorySkills({ rootDir }); + + assert.deepEqual(diagnostics, [ + '.codex/skills/release-notes/SKILL.md: differs from .claude/skills/release-notes/SKILL.md', + ]); +}); + +test('reports a missing release skill mirror without stopping validation', async (t) => { + const rootDir = await createRepository(t, { + description: 'Implementation guidance.', + }); + await rm( + join(rootDir, '.claude/skills/release-cut/SKILL.md') + ); + + const { diagnostics } = await validateRepositorySkills({ rootDir }); + + assert.deepEqual(diagnostics, [ + '.codex/skills/example-skill/SKILL.md: description must start with "Use when"', + '.claude/skills/release-cut/SKILL.md: required release skill mirror is missing', + ]); +}); + +test('returns every diagnostic in deterministic order', async (t) => { + const rootDir = await createRepository(t, { + name: 'wrong-name', + description: 'Implementation guidance.', + body: 'Inspect `docs/z-missing.md` and `docs/a-missing.md`.', + releaseMirrorTransform: (contents, skillName) => + skillName === 'release-notes' ? `${contents}\n` : contents, + }); + + const { diagnostics } = await validateRepositorySkills({ rootDir }); + + assert.deepEqual(diagnostics, [ + '.codex/skills/example-skill/SKILL.md: frontmatter name "wrong-name" must match directory "example-skill"', + '.codex/skills/example-skill/SKILL.md: description must start with "Use when"', + '.codex/skills/example-skill/SKILL.md: referenced path does not exist: docs/a-missing.md', + '.codex/skills/example-skill/SKILL.md: referenced path does not exist: docs/z-missing.md', + '.codex/skills/release-notes/SKILL.md: differs from .claude/skills/release-notes/SKILL.md', + ]); +});