test(spec): discover the migrate-sentence markdown corpus instead of naming one file#14557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
os-sam merged 1 commit into from
Sep 2, 2026
Merged

test(spec): discover the migrate-sentence markdown corpus instead of naming one file #14557

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
228 changes: 183 additions & 45 deletions packages/spec/src/shared/retired-key-migrate-sentence.test.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -70,21 +70,36 @@ const SPEC_SRC_ROOT = path.resolve(HERE, '..');
/** #7030: `packages/lint/src`, the one other corpus carrying this sentence. */
const LINT_SRC_ROOT = path.resolve(HERE, '../../../lint/src');
/**
* [#10848] The population widens by EXACTLY ONE governed file (maintainer
* ruling 2026-08-22, deliberately not all of `.claude/**`): the retirement
* [#10848] The population widened by EXACTLY ONE governed file (maintainer
* ruling 2026-08-22, deliberately not all of `.claude/`): the retirement
* playbook every new tombstone's guidance string is authored from. It sat
* outside both corpora and prescribed the withdrawn sentence, so the skill
* taught authors to red this very pin — and a red pin over a skill-taught
* sentence invites weakening the PIN rather than the skill. It cannot ride
* the corpus walk: it is markdown (the walk yields `.ts` only), its `--from`
* operand is a placeholder like `<N-1>` (never `\d+`), its sentences end at a
* code-span close (never at a string-literal quote), and `reconstruct()`
* would drop every markdown line that opens with `*`/`**`. So it is judged
* would drop every markdown line that opens with an asterisk. So it is judged
* below as its own corpus: raw text, whitespace-normalised, with
* placeholder-aware anchors — the withdrawn-claim direction reuses
* `WITHDRAWN_CLAIM` verbatim.
*
* [#13859] ONE hard-coded path is a population of one, and the sentence does
* not stay inside it: the withdrawn automatic-rewrite claim reached the
* PUBLISHED skill catalog — the artifacts a customer agent actually loads, and
* that land in codebases this repo cannot see — with nothing scanning them.
* The corpus below is therefore DISCOVERED rather than named: this playbook
* plus every `.md` file under `skills/`. Discovery is the input, so it is
* itself asserted non-vacuous — a walk that silently reached zero published
* files would leave this pin exactly as narrow as it was, while reading green.
*/
const RETIREMENT_SKILL_MD = path.resolve(HERE, '../../../../.claude/skills/spec-property-retirement/SKILL.md');
/**
* [#13859] The published catalog root. Discovered, never listed: a checked-in
* file list is a second population to keep in sync, and the one that rots is
* always the list.
*/
const PUBLISHED_SKILLS_ROOT = path.resolve(HERE, '../../../../skills');

/** One scanned corpus: a root directory, plus its own out-of-scope exemptions. */
interface Corpus {
Expand DownExpand Up@@ -388,62 +403,185 @@ describe('`os migrate meta` sentences are the house sentence, across corpora (#6
});

/**
* [#10848] Markdown-corpus anchors for the retirement skill (see the
* `RETIREMENT_SKILL_MD` docblock). Same two legal shapes as
* `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on exactly three axes: the
* `--from` operand may be a placeholder (`<N-1>`, `<N>`) as well as a literal
* major; the judged text is the whole file with runs of whitespace collapsed
* (markdown wraps sentences mid-clause); and "last sentence of its literal"
* becomes "last sentence of its double-backtick code span" (`.``), so prose
* cannot bury the command mid-span either. The marker requires the leading
* `Run` on purpose: the skill legitimately NAMES the command mid-prose
* (`migrate meta --from <old>` in §3's `retiredFromLoadPath` bullet) without
* prescribing a sentence — only taught sentence templates are judged.
* [#10848] Markdown-corpus anchors (see the `RETIREMENT_SKILL_MD` docblock).
* Same two legal shapes as `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on
* exactly three axes: the `--from` operand may be a placeholder (`<N-1>`,
* `<N>`) as well as a literal major; the judged text is the whole file with
* runs of whitespace collapsed (markdown wraps sentences mid-clause); and
* "last sentence of its literal" becomes "last sentence of its container".
*
* The marker requires the leading `Run` on purpose: a skill legitimately NAMES
* the command mid-prose (`migrate meta --from <old>` in the playbook's §3
* `retiredFromLoadPath` bullet; a dozen bare `os migrate meta …` invocations in
* the published upgrade skill) without prescribing a sentence — only sentences
* are judged. That property is what makes the widened population safe, so it
* stays.
*
* [#13859] "Container" is TWO things once the corpus is more than the playbook,
* and one anchor for both would be a false-positive machine. A taught TEMPLATE
* is written as a double-backtick code span, so its sentence must close that
* span (`.` then the span's two backticks) — prose cannot bury the command
* mid-span, which is the property the source corpus gets from its closing
* quote. A sentence QUOTED in prose or inside a fenced transcript has no span
* to close: the published upgrade skill quotes the house sentence verbatim
* inside a rendered parse error, whose next line is `expected: never`. Under a
* span-only anchor every such occurrence is an unconditional RED — not a
* property, since nothing but a code span could ever satisfy it, but a trap
* that fires the moment the corpus stops being one file. So a non-span
* occurrence is judged for its WORDING, ending at its own period. What that
* gives up, deliberately and only in prose: burying (`… apply them by hand.
* Also do X.`) passes there, while it stays RED inside a template and in every
* `.ts` prescription above — the two places a reader copies text from.
*/
const SKILL_FROM_OPERAND = /(?:\d+|<[^>`]+>)/.source;
const SKILL_MARKER = new RegExp(
`Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\``,
'g',
);
const SKILL_HOUSE_AT_MARKER = new RegExp(
`^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.\`\``,
);
const SKILL_MIXED_AT_MARKER = new RegExp(
`^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+\\.\`\``,
);
/** The two legal wordings, from the marker up to the sentence's final period. */
const SKILL_HOUSE_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.`;
const SKILL_MIXED_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+?\\.`;
/** Container-final anchors: a code span closes; a quoted sentence just ends. */
const SKILL_TEMPLATE_END = '``';
const SKILL_PROSE_END = '(?:\\s|$)';
const SKILL_HOUSE_TEMPLATE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_MIXED_TEMPLATE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_HOUSE_PROSE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_PROSE_END}`);
const SKILL_MIXED_PROSE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_PROSE_END}`);

/** Label prefix for the internal playbook, mirroring the `spec:`/`lint:` convention. */
const PLAYBOOK_LABEL = 'internal:spec-property-retirement/SKILL.md';

interface MarkdownFile {
/** Corpus-prefixed path, e.g. `skills:objectstack-upgrade/SKILL.md`. */
file: string;
/** Whole file, runs of whitespace collapsed. */
flat: string;
}

describe('the spec-property-retirement skill agrees with this pin (#10848 — the one-file population widening)', () => {
const raw = fs.readFileSync(RETIREMENT_SKILL_MD, 'utf8');
const flat = raw.replace(/\s+/g, ' ');

it('every prescription sentence the skill teaches is house-form or MIXED two-clause', () => {
const bad = [...flat.matchAll(SKILL_MARKER)]
.filter((m) => {
const rest = flat.slice(m.index ?? 0);
return !(SKILL_HOUSE_AT_MARKER.test(rest) || SKILL_MIXED_AT_MARKER.test(rest));
})
.map((m) => flat.slice(m.index ?? 0, (m.index ?? 0) + 120));
expect(bad, bad.join('\n')).toEqual([]);
/**
* [#13859] Deterministic markdown discovery. `readdirSync(…).sort()` so the
* corpus order — and therefore every failure message — is identical on every
* machine and every run. `lstatSync` so a symlink is SKIPPED rather than
* followed: a link pointing outside the repo would make the corpus depend on
* the checkout around it, and following one back inside would judge a file
* twice. Nothing else is skipped, and nothing is skipped silently — the
* discovery anti-vacuity assertion below is what notices when it is.
*/
function* walkMarkdown(dir: string): Generator<string> {
for (const name of fs.readdirSync(dir).sort()) {
const p = path.join(dir, name);
const st = fs.lstatSync(p);
if (st.isSymbolicLink()) continue;
if (st.isDirectory()) yield* walkMarkdown(p);
else if (st.isFile() && p.endsWith('.md')) yield p;
}
}

/** The internal retirement playbook, then the published catalog in sorted order. */
function markdownCorpus(): MarkdownFile[] {
const files: Array<{ file: string; abs: string }> = [{ file: PLAYBOOK_LABEL, abs: RETIREMENT_SKILL_MD }];
for (const abs of walkMarkdown(PUBLISHED_SKILLS_ROOT)) {
files.push({ file: `skills:${path.relative(PUBLISHED_SKILLS_ROOT, abs)}`, abs });
}
return files.map(({ file, abs }) => ({ file, flat: fs.readFileSync(abs, 'utf8').replace(/\s+/g, ' ') }));
}

interface MarkdownSite {
file: string;
/** True when the marker opens a double-backtick code span — a taught template. */
template: boolean;
excerpt: string;
ok: boolean;
}

/** Every `os migrate meta` prescription sentence in one markdown file, with its verdict. */
function judgeMarkdownSentences({ file, flat }: MarkdownFile): MarkdownSite[] {
return [...flat.matchAll(SKILL_MARKER)].map((m) => {
const at = m.index ?? 0;
const rest = flat.slice(at);
const template = flat.slice(0, at).endsWith('``');
const ok = template
? SKILL_HOUSE_TEMPLATE.test(rest) || SKILL_MIXED_TEMPLATE.test(rest)
: SKILL_HOUSE_PROSE.test(rest) || SKILL_MIXED_PROSE.test(rest);
return { file, template, excerpt: rest.slice(0, 120), ok };
});
}

describe('the retirement playbook and the published skill catalog agree with this pin (#10848, corpus widened #13859)', () => {
const corpus = markdownCorpus();

it('[#13859] anti-vacuity for the DISCOVERY: the playbook and at least one published skill', () => {
// The corpus is this suite's input, so a walk that reached zero published
// files would restore the exact one-file blindness #13859 is about — and
// every assertion below would stay green while it did. Assert the shape of
// the population itself, not a count that ages out with the catalog.
expect(corpus.length).toBeGreaterThan(0);
const labels = corpus.map((c) => c.file);
expect(labels).toContain(PLAYBOOK_LABEL);
const published = labels.filter((l) => l.startsWith('skills:'));
expect(published.length).toBeGreaterThanOrEqual(1);
// Deterministic ordering: the published tail is sorted, so a failure names
// its sites in a stable order rather than in readdir order.
expect(published).toEqual([...published].sort());
});

it('every prescription sentence in the markdown corpus is house-form or MIXED two-clause', () => {
const bad = corpus.flatMap(judgeMarkdownSentences).filter((s) => !s.ok);
expect(
bad,
bad.map((s) => `${s.file} [${s.template ? 'template' : 'prose'}] — "${s.excerpt}"`).join('\n'),
).toEqual([]);
});

it('anti-vacuity: the skill teaches BOTH shapes, so the scan judges at least two sites', () => {
// Convention 5 carries the house template and its one allowed variant
// (the partial-conversion two-clause shape). Zero or one marker means the
// skill stopped teaching a shape — or this scan went blind on the file.
const rests = [...flat.matchAll(SKILL_MARKER)].map((m) => flat.slice(m.index ?? 0));
it('anti-vacuity: the PLAYBOOK teaches BOTH shapes, so the scan judges at least two sites', () => {
// Pinned to the one file that OWNS both templates — never a whole-corpus
// claim. Convention 5 carries the house template and its one allowed
// variant (the partial-conversion two-clause shape); zero or one marker
// means the playbook stopped teaching a shape, or this scan went blind on
// it. A published skill that names the command once is not a regression,
// so widening this floor to the corpus would assert nothing and fail on
// the catalog's own editorial choices.
const playbook = corpus.find((c) => c.file === PLAYBOOK_LABEL);
expect(playbook, PLAYBOOK_LABEL).toBeDefined();
const rests = [...playbook!.flat.matchAll(SKILL_MARKER)].map((m) => playbook!.flat.slice(m.index ?? 0));
expect(rests.length).toBeGreaterThanOrEqual(2);
expect(rests.some((r) => SKILL_HOUSE_AT_MARKER.test(r))).toBe(true);
expect(rests.some((r) => SKILL_MIXED_AT_MARKER.test(r))).toBe(true);
expect(rests.some((r) => SKILL_HOUSE_TEMPLATE.test(r))).toBe(true);
expect(rests.some((r) => SKILL_MIXED_TEMPLATE.test(r))).toBe(true);
});

it('[#9529] the withdrawn automatic-rewrite claim is absent from the skill, in every spelling', () => {
it('[#9529] the withdrawn automatic-rewrite claim is absent from the corpus, in every spelling', () => {
// Judged over the raw text rather than reconstruct(): a markdown line
// opening with `*`/`**` would be dropped as a comment line, hiding a
// claim. WITHDRAWN_CLAIM is English-only, so the skill's Chinese prose
// cannot fabricate a match; a hit is a real regression of the ruling.
const claims = [...flat.matchAll(WITHDRAWN_CLAIM)].map(
(m) => flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60),
// opening with an asterisk would be dropped as a comment line, hiding a
// claim. WITHDRAWN_CLAIM is English-only BY DESIGN — the playbook's
// Chinese prose (and any that follows it into the catalog) cannot
// fabricate a match, so a hit is a real regression of the ruling. This is
// the direction #13859 widened for: the claim is red wherever it appears,
// and "wherever" now includes the published catalog.
const claims = corpus.flatMap(({ file, flat }) =>
[...flat.matchAll(WITHDRAWN_CLAIM)].map(
(m) => `${file} — "${flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60)}"`,
),
);
expect(claims, claims.join('\n')).toEqual([]);
});

it('[#13859] the markdown judge is not vacuous — template and prose anchors each hold', () => {
const judge = (flat: string): MarkdownSite[] => judgeMarkdownSentences({ file: 'synthetic.md', flat });
const house = 'Run `os migrate meta --from <N-1>` to list the mechanical edits for existing sources; apply them by hand.';
const mixed = 'Run `os migrate meta --from 16` to list the mechanical edits for the `1y` case; the rest are reported.';
// A taught template must close its code span, in both legal shapes.
expect(judge(`5. \`\`${house}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
expect(judge(`\`\`${mixed}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
// …and a template that buries the command mid-span stays RED.
expect(judge(`\`\`${house} Also do X.\`\``).map((s) => s.ok)).toEqual([false]);
// A sentence quoted in prose or a transcript is judged on wording alone.
expect(judge(`error text … ${house} expected: never`).map((s) => [s.template, s.ok])).toEqual([[false, true]]);
expect(judge('Run `os migrate meta --from 16` to rewrite it automatically.').map((s) => s.ok)).toEqual([false]);
expect(judge('Run `os migrate meta --from 16` to remove it.').map((s) => s.ok)).toEqual([false]);
// Naming the command mid-prose without the leading `Run` is not a sentence.
expect(judge('Stored flows convert with `os migrate meta --from 16`.')).toEqual([]);
// The withdrawn claim trips wherever it appears, fence or prose.
expect([...'``os migrate meta --from 16` rewrites your source files.``'.matchAll(WITHDRAWN_CLAIM)]).not.toEqual([]);
});
});
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
test(spec): discover the migrate-sentence markdown corpus instead of naming one file#14557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

os-sam
merged 1 commit into

from

Sep 2, 2026
 Merged


test(spec): discover the migrate-sentence markdown corpus instead of naming one file
#14557
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


228 changes: 183 additions & 45 deletions
packages/spec/src/shared/retired-key-migrate-sentence.test.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -70,21 +70,36 @@ const SPEC_SRC_ROOT = path.resolve(HERE, '..');
/** #7030: `packages/lint/src`, the one other corpus carrying this sentence. */
const LINT_SRC_ROOT = path.resolve(HERE, '../../../lint/src');
/**
 * [#10848] The population widens by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/**`): the retirement
 * [#10848] The population widened by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/`): the retirement
 * playbook every new tombstone's guidance string is authored from. It sat
 * outside both corpora and prescribed the withdrawn sentence, so the skill
 * taught authors to red this very pin — and a red pin over a skill-taught
 * sentence invites weakening the PIN rather than the skill. It cannot ride
 * the corpus walk: it is markdown (the walk yields `.ts` only), its `--from`
 * operand is a placeholder like `<N-1>` (never `\d+`), its sentences end at a
 * code-span close (never at a string-literal quote), and `reconstruct()`
 * would drop every markdown line that opens with `*`/`**`. So it is judged
 * would drop every markdown line that opens with an asterisk. So it is judged
 * below as its own corpus: raw text, whitespace-normalised, with
 * placeholder-aware anchors — the withdrawn-claim direction reuses
 * `WITHDRAWN_CLAIM` verbatim.
 *
 * [#13859] ONE hard-coded path is a population of one, and the sentence does
 * not stay inside it: the withdrawn automatic-rewrite claim reached the
 * PUBLISHED skill catalog — the artifacts a customer agent actually loads, and
 * that land in codebases this repo cannot see — with nothing scanning them.
 * The corpus below is therefore DISCOVERED rather than named: this playbook
 * plus every `.md` file under `skills/`. Discovery is the input, so it is
 * itself asserted non-vacuous — a walk that silently reached zero published
 * files would leave this pin exactly as narrow as it was, while reading green.
 */
const RETIREMENT_SKILL_MD = path.resolve(HERE, '../../../../.claude/skills/spec-property-retirement/SKILL.md');
/**
 * [#13859] The published catalog root. Discovered, never listed: a checked-in
 * file list is a second population to keep in sync, and the one that rots is
 * always the list.
 */
const PUBLISHED_SKILLS_ROOT = path.resolve(HERE, '../../../../skills');

/** One scanned corpus: a root directory, plus its own out-of-scope exemptions. */
interface Corpus {
Expand DownExpand Up@@ -388,62 +403,185 @@ describe('`os migrate meta` sentences are the house sentence, across corpora (#6
});

/**
 * [#10848] Markdown-corpus anchors for the retirement skill (see the
 * `RETIREMENT_SKILL_MD` docblock). Same two legal shapes as
 * `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on exactly three axes: the
 * `--from` operand may be a placeholder (`<N-1>`, `<N>`) as well as a literal
 * major; the judged text is the whole file with runs of whitespace collapsed
 * (markdown wraps sentences mid-clause); and "last sentence of its literal"
 * becomes "last sentence of its double-backtick code span" (`.``), so prose
 * cannot bury the command mid-span either. The marker requires the leading
 * `Run` on purpose: the skill legitimately NAMES the command mid-prose
 * (`migrate meta --from <old>` in §3's `retiredFromLoadPath` bullet) without
 * prescribing a sentence — only taught sentence templates are judged.
 * [#10848] Markdown-corpus anchors (see the `RETIREMENT_SKILL_MD` docblock).
 * Same two legal shapes as `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on
 * exactly three axes: the `--from` operand may be a placeholder (`<N-1>`,
 * `<N>`) as well as a literal major; the judged text is the whole file with
 * runs of whitespace collapsed (markdown wraps sentences mid-clause); and
 * "last sentence of its literal" becomes "last sentence of its container".
 *
 * The marker requires the leading `Run` on purpose: a skill legitimately NAMES
 * the command mid-prose (`migrate meta --from <old>` in the playbook's §3
 * `retiredFromLoadPath` bullet; a dozen bare `os migrate meta …` invocations in
 * the published upgrade skill) without prescribing a sentence — only sentences
 * are judged. That property is what makes the widened population safe, so it
 * stays.
 *
 * [#13859] "Container" is TWO things once the corpus is more than the playbook,
 * and one anchor for both would be a false-positive machine. A taught TEMPLATE
 * is written as a double-backtick code span, so its sentence must close that
 * span (`.` then the span's two backticks) — prose cannot bury the command
 * mid-span, which is the property the source corpus gets from its closing
 * quote. A sentence QUOTED in prose or inside a fenced transcript has no span
 * to close: the published upgrade skill quotes the house sentence verbatim
 * inside a rendered parse error, whose next line is `expected: never`. Under a
 * span-only anchor every such occurrence is an unconditional RED — not a
 * property, since nothing but a code span could ever satisfy it, but a trap
 * that fires the moment the corpus stops being one file. So a non-span
 * occurrence is judged for its WORDING, ending at its own period. What that
 * gives up, deliberately and only in prose: burying (`… apply them by hand.
 * Also do X.`) passes there, while it stays RED inside a template and in every
 * `.ts` prescription above — the two places a reader copies text from.
 */
const SKILL_FROM_OPERAND = /(?:\d+|<[^>`]+>)/.source;
const SKILL_MARKER = new RegExp(
 `Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\``,
 'g',
);
const SKILL_HOUSE_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.\`\``,
);
const SKILL_MIXED_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+\\.\`\``,
);
/** The two legal wordings, from the marker up to the sentence's final period. */
const SKILL_HOUSE_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.`;
const SKILL_MIXED_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+?\\.`;
/** Container-final anchors: a code span closes; a quoted sentence just ends. */
const SKILL_TEMPLATE_END = '``';
const SKILL_PROSE_END = '(?:\\s|$)';
const SKILL_HOUSE_TEMPLATE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_MIXED_TEMPLATE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_HOUSE_PROSE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_PROSE_END}`);
const SKILL_MIXED_PROSE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_PROSE_END}`);

/** Label prefix for the internal playbook, mirroring the `spec:`/`lint:` convention. */
const PLAYBOOK_LABEL = 'internal:spec-property-retirement/SKILL.md';

interface MarkdownFile {
 /** Corpus-prefixed path, e.g. `skills:objectstack-upgrade/SKILL.md`. */
 file: string;
 /** Whole file, runs of whitespace collapsed. */
 flat: string;
}

describe('the spec-property-retirement skill agrees with this pin (#10848 — the one-file population widening)', () => {
 const raw = fs.readFileSync(RETIREMENT_SKILL_MD, 'utf8');
 const flat = raw.replace(/\s+/g, ' ');

 it('every prescription sentence the skill teaches is house-form or MIXED two-clause', () => {
 const bad = [...flat.matchAll(SKILL_MARKER)]
 .filter((m) => {
 const rest = flat.slice(m.index ?? 0);
 return !(SKILL_HOUSE_AT_MARKER.test(rest) || SKILL_MIXED_AT_MARKER.test(rest));
 })
 .map((m) => flat.slice(m.index ?? 0, (m.index ?? 0) + 120));
 expect(bad, bad.join('\n')).toEqual([]);
/**
 * [#13859] Deterministic markdown discovery. `readdirSync(…).sort()` so the
 * corpus order — and therefore every failure message — is identical on every
 * machine and every run. `lstatSync` so a symlink is SKIPPED rather than
 * followed: a link pointing outside the repo would make the corpus depend on
 * the checkout around it, and following one back inside would judge a file
 * twice. Nothing else is skipped, and nothing is skipped silently — the
 * discovery anti-vacuity assertion below is what notices when it is.
 */
function* walkMarkdown(dir: string): Generator<string> {
 for (const name of fs.readdirSync(dir).sort()) {
 const p = path.join(dir, name);
 const st = fs.lstatSync(p);
 if (st.isSymbolicLink()) continue;
 if (st.isDirectory()) yield* walkMarkdown(p);
 else if (st.isFile() && p.endsWith('.md')) yield p;
 }
}

/** The internal retirement playbook, then the published catalog in sorted order. */
function markdownCorpus(): MarkdownFile[] {
 const files: Array<{ file: string; abs: string }> = [{ file: PLAYBOOK_LABEL, abs: RETIREMENT_SKILL_MD }];
 for (const abs of walkMarkdown(PUBLISHED_SKILLS_ROOT)) {
 files.push({ file: `skills:${path.relative(PUBLISHED_SKILLS_ROOT, abs)}`, abs });
 }
 return files.map(({ file, abs }) => ({ file, flat: fs.readFileSync(abs, 'utf8').replace(/\s+/g, ' ') }));
}

interface MarkdownSite {
 file: string;
 /** True when the marker opens a double-backtick code span — a taught template. */
 template: boolean;
 excerpt: string;
 ok: boolean;
}

/** Every `os migrate meta` prescription sentence in one markdown file, with its verdict. */
function judgeMarkdownSentences({ file, flat }: MarkdownFile): MarkdownSite[] {
 return [...flat.matchAll(SKILL_MARKER)].map((m) => {
 const at = m.index ?? 0;
 const rest = flat.slice(at);
 const template = flat.slice(0, at).endsWith('``');
 const ok = template
 ? SKILL_HOUSE_TEMPLATE.test(rest) || SKILL_MIXED_TEMPLATE.test(rest)
 : SKILL_HOUSE_PROSE.test(rest) || SKILL_MIXED_PROSE.test(rest);
 return { file, template, excerpt: rest.slice(0, 120), ok };
 });
}

describe('the retirement playbook and the published skill catalog agree with this pin (#10848, corpus widened #13859)', () => {
 const corpus = markdownCorpus();

 it('[#13859] anti-vacuity for the DISCOVERY: the playbook and at least one published skill', () => {
 // The corpus is this suite's input, so a walk that reached zero published
 // files would restore the exact one-file blindness #13859 is about — and
 // every assertion below would stay green while it did. Assert the shape of
 // the population itself, not a count that ages out with the catalog.
 expect(corpus.length).toBeGreaterThan(0);
 const labels = corpus.map((c) => c.file);
 expect(labels).toContain(PLAYBOOK_LABEL);
 const published = labels.filter((l) => l.startsWith('skills:'));
 expect(published.length).toBeGreaterThanOrEqual(1);
 // Deterministic ordering: the published tail is sorted, so a failure names
 // its sites in a stable order rather than in readdir order.
 expect(published).toEqual([...published].sort());
 });

 it('every prescription sentence in the markdown corpus is house-form or MIXED two-clause', () => {
 const bad = corpus.flatMap(judgeMarkdownSentences).filter((s) => !s.ok);
 expect(
 bad,
 bad.map((s) => `${s.file} [${s.template ? 'template' : 'prose'}] — "${s.excerpt}"`).join('\n'),
 ).toEqual([]);
 });

 it('anti-vacuity: the skill teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Convention 5 carries the house template and its one allowed variant
 // (the partial-conversion two-clause shape). Zero or one marker means the
 // skill stopped teaching a shape — or this scan went blind on the file.
 const rests = [...flat.matchAll(SKILL_MARKER)].map((m) => flat.slice(m.index ?? 0));
 it('anti-vacuity: the PLAYBOOK teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Pinned to the one file that OWNS both templates — never a whole-corpus
 // claim. Convention 5 carries the house template and its one allowed
 // variant (the partial-conversion two-clause shape); zero or one marker
 // means the playbook stopped teaching a shape, or this scan went blind on
 // it. A published skill that names the command once is not a regression,
 // so widening this floor to the corpus would assert nothing and fail on
 // the catalog's own editorial choices.
 const playbook = corpus.find((c) => c.file === PLAYBOOK_LABEL);
 expect(playbook, PLAYBOOK_LABEL).toBeDefined();
 const rests = [...playbook!.flat.matchAll(SKILL_MARKER)].map((m) => playbook!.flat.slice(m.index ?? 0));
 expect(rests.length).toBeGreaterThanOrEqual(2);
 expect(rests.some((r) => SKILL_HOUSE_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_HOUSE_TEMPLATE.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_TEMPLATE.test(r))).toBe(true);
 });

 it('[#9529] the withdrawn automatic-rewrite claim is absent from the skill, in every spelling', () => {
 it('[#9529] the withdrawn automatic-rewrite claim is absent from the corpus, in every spelling', () => {
 // Judged over the raw text rather than reconstruct(): a markdown line
 // opening with `*`/`**` would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only, so the skill's Chinese prose
 // cannot fabricate a match; a hit is a real regression of the ruling.
 const claims = [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60),
 // opening with an asterisk would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only BY DESIGN — the playbook's
 // Chinese prose (and any that follows it into the catalog) cannot
 // fabricate a match, so a hit is a real regression of the ruling. This is
 // the direction #13859 widened for: the claim is red wherever it appears,
 // and "wherever" now includes the published catalog.
 const claims = corpus.flatMap(({ file, flat }) =>
 [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => `${file} — "${flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60)}"`,
 ),
 );
 expect(claims, claims.join('\n')).toEqual([]);
 });

 it('[#13859] the markdown judge is not vacuous — template and prose anchors each hold', () => {
 const judge = (flat: string): MarkdownSite[] => judgeMarkdownSentences({ file: 'synthetic.md', flat });
 const house = 'Run `os migrate meta --from <N-1>` to list the mechanical edits for existing sources; apply them by hand.';
 const mixed = 'Run `os migrate meta --from 16` to list the mechanical edits for the `1y` case; the rest are reported.';
 // A taught template must close its code span, in both legal shapes.
 expect(judge(`5. \`\`${house}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 expect(judge(`\`\`${mixed}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 // …and a template that buries the command mid-span stays RED.
 expect(judge(`\`\`${house} Also do X.\`\``).map((s) => s.ok)).toEqual([false]);
 // A sentence quoted in prose or a transcript is judged on wording alone.
 expect(judge(`error text … ${house} expected: never`).map((s) => [s.template, s.ok])).toEqual([[false, true]]);
 expect(judge('Run `os migrate meta --from 16` to rewrite it automatically.').map((s) => s.ok)).toEqual([false]);
 expect(judge('Run `os migrate meta --from 16` to remove it.').map((s) => s.ok)).toEqual([false]);
 // Naming the command mid-prose without the leading `Run` is not a sentence.
 expect(judge('Stored flows convert with `os migrate meta --from 16`.')).toEqual([]);
 // The withdrawn claim trips wherever it appears, fence or prose.
 expect([...'``os migrate meta --from 16` rewrites your source files.``'.matchAll(WITHDRAWN_CLAIM)]).not.toEqual([]);
 });
});
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
test(spec): discover the migrate-sentence markdown corpus instead of naming one file#14557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

os-sam
merged 1 commit into

from

Sep 2, 2026
 Merged


test(spec): discover the migrate-sentence markdown corpus instead of naming one file
#14557
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


228 changes: 183 additions & 45 deletions
packages/spec/src/shared/retired-key-migrate-sentence.test.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -70,21 +70,36 @@ const SPEC_SRC_ROOT = path.resolve(HERE, '..');
/** #7030: `packages/lint/src`, the one other corpus carrying this sentence. */
const LINT_SRC_ROOT = path.resolve(HERE, '../../../lint/src');
/**
 * [#10848] The population widens by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/**`): the retirement
 * [#10848] The population widened by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/`): the retirement
 * playbook every new tombstone's guidance string is authored from. It sat
 * outside both corpora and prescribed the withdrawn sentence, so the skill
 * taught authors to red this very pin — and a red pin over a skill-taught
 * sentence invites weakening the PIN rather than the skill. It cannot ride
 * the corpus walk: it is markdown (the walk yields `.ts` only), its `--from`
 * operand is a placeholder like `<N-1>` (never `\d+`), its sentences end at a
 * code-span close (never at a string-literal quote), and `reconstruct()`
 * would drop every markdown line that opens with `*`/`**`. So it is judged
 * would drop every markdown line that opens with an asterisk. So it is judged
 * below as its own corpus: raw text, whitespace-normalised, with
 * placeholder-aware anchors — the withdrawn-claim direction reuses
 * `WITHDRAWN_CLAIM` verbatim.
 *
 * [#13859] ONE hard-coded path is a population of one, and the sentence does
 * not stay inside it: the withdrawn automatic-rewrite claim reached the
 * PUBLISHED skill catalog — the artifacts a customer agent actually loads, and
 * that land in codebases this repo cannot see — with nothing scanning them.
 * The corpus below is therefore DISCOVERED rather than named: this playbook
 * plus every `.md` file under `skills/`. Discovery is the input, so it is
 * itself asserted non-vacuous — a walk that silently reached zero published
 * files would leave this pin exactly as narrow as it was, while reading green.
 */
const RETIREMENT_SKILL_MD = path.resolve(HERE, '../../../../.claude/skills/spec-property-retirement/SKILL.md');
/**
 * [#13859] The published catalog root. Discovered, never listed: a checked-in
 * file list is a second population to keep in sync, and the one that rots is
 * always the list.
 */
const PUBLISHED_SKILLS_ROOT = path.resolve(HERE, '../../../../skills');

/** One scanned corpus: a root directory, plus its own out-of-scope exemptions. */
interface Corpus {
Expand DownExpand Up@@ -388,62 +403,185 @@ describe('`os migrate meta` sentences are the house sentence, across corpora (#6
});

/**
 * [#10848] Markdown-corpus anchors for the retirement skill (see the
 * `RETIREMENT_SKILL_MD` docblock). Same two legal shapes as
 * `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on exactly three axes: the
 * `--from` operand may be a placeholder (`<N-1>`, `<N>`) as well as a literal
 * major; the judged text is the whole file with runs of whitespace collapsed
 * (markdown wraps sentences mid-clause); and "last sentence of its literal"
 * becomes "last sentence of its double-backtick code span" (`.``), so prose
 * cannot bury the command mid-span either. The marker requires the leading
 * `Run` on purpose: the skill legitimately NAMES the command mid-prose
 * (`migrate meta --from <old>` in §3's `retiredFromLoadPath` bullet) without
 * prescribing a sentence — only taught sentence templates are judged.
 * [#10848] Markdown-corpus anchors (see the `RETIREMENT_SKILL_MD` docblock).
 * Same two legal shapes as `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on
 * exactly three axes: the `--from` operand may be a placeholder (`<N-1>`,
 * `<N>`) as well as a literal major; the judged text is the whole file with
 * runs of whitespace collapsed (markdown wraps sentences mid-clause); and
 * "last sentence of its literal" becomes "last sentence of its container".
 *
 * The marker requires the leading `Run` on purpose: a skill legitimately NAMES
 * the command mid-prose (`migrate meta --from <old>` in the playbook's §3
 * `retiredFromLoadPath` bullet; a dozen bare `os migrate meta …` invocations in
 * the published upgrade skill) without prescribing a sentence — only sentences
 * are judged. That property is what makes the widened population safe, so it
 * stays.
 *
 * [#13859] "Container" is TWO things once the corpus is more than the playbook,
 * and one anchor for both would be a false-positive machine. A taught TEMPLATE
 * is written as a double-backtick code span, so its sentence must close that
 * span (`.` then the span's two backticks) — prose cannot bury the command
 * mid-span, which is the property the source corpus gets from its closing
 * quote. A sentence QUOTED in prose or inside a fenced transcript has no span
 * to close: the published upgrade skill quotes the house sentence verbatim
 * inside a rendered parse error, whose next line is `expected: never`. Under a
 * span-only anchor every such occurrence is an unconditional RED — not a
 * property, since nothing but a code span could ever satisfy it, but a trap
 * that fires the moment the corpus stops being one file. So a non-span
 * occurrence is judged for its WORDING, ending at its own period. What that
 * gives up, deliberately and only in prose: burying (`… apply them by hand.
 * Also do X.`) passes there, while it stays RED inside a template and in every
 * `.ts` prescription above — the two places a reader copies text from.
 */
const SKILL_FROM_OPERAND = /(?:\d+|<[^>`]+>)/.source;
const SKILL_MARKER = new RegExp(
 `Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\``,
 'g',
);
const SKILL_HOUSE_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.\`\``,
);
const SKILL_MIXED_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+\\.\`\``,
);
/** The two legal wordings, from the marker up to the sentence's final period. */
const SKILL_HOUSE_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.`;
const SKILL_MIXED_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+?\\.`;
/** Container-final anchors: a code span closes; a quoted sentence just ends. */
const SKILL_TEMPLATE_END = '``';
const SKILL_PROSE_END = '(?:\\s|$)';
const SKILL_HOUSE_TEMPLATE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_MIXED_TEMPLATE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_HOUSE_PROSE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_PROSE_END}`);
const SKILL_MIXED_PROSE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_PROSE_END}`);

/** Label prefix for the internal playbook, mirroring the `spec:`/`lint:` convention. */
const PLAYBOOK_LABEL = 'internal:spec-property-retirement/SKILL.md';

interface MarkdownFile {
 /** Corpus-prefixed path, e.g. `skills:objectstack-upgrade/SKILL.md`. */
 file: string;
 /** Whole file, runs of whitespace collapsed. */
 flat: string;
}

describe('the spec-property-retirement skill agrees with this pin (#10848 — the one-file population widening)', () => {
 const raw = fs.readFileSync(RETIREMENT_SKILL_MD, 'utf8');
 const flat = raw.replace(/\s+/g, ' ');

 it('every prescription sentence the skill teaches is house-form or MIXED two-clause', () => {
 const bad = [...flat.matchAll(SKILL_MARKER)]
 .filter((m) => {
 const rest = flat.slice(m.index ?? 0);
 return !(SKILL_HOUSE_AT_MARKER.test(rest) || SKILL_MIXED_AT_MARKER.test(rest));
 })
 .map((m) => flat.slice(m.index ?? 0, (m.index ?? 0) + 120));
 expect(bad, bad.join('\n')).toEqual([]);
/**
 * [#13859] Deterministic markdown discovery. `readdirSync(…).sort()` so the
 * corpus order — and therefore every failure message — is identical on every
 * machine and every run. `lstatSync` so a symlink is SKIPPED rather than
 * followed: a link pointing outside the repo would make the corpus depend on
 * the checkout around it, and following one back inside would judge a file
 * twice. Nothing else is skipped, and nothing is skipped silently — the
 * discovery anti-vacuity assertion below is what notices when it is.
 */
function* walkMarkdown(dir: string): Generator<string> {
 for (const name of fs.readdirSync(dir).sort()) {
 const p = path.join(dir, name);
 const st = fs.lstatSync(p);
 if (st.isSymbolicLink()) continue;
 if (st.isDirectory()) yield* walkMarkdown(p);
 else if (st.isFile() && p.endsWith('.md')) yield p;
 }
}

/** The internal retirement playbook, then the published catalog in sorted order. */
function markdownCorpus(): MarkdownFile[] {
 const files: Array<{ file: string; abs: string }> = [{ file: PLAYBOOK_LABEL, abs: RETIREMENT_SKILL_MD }];
 for (const abs of walkMarkdown(PUBLISHED_SKILLS_ROOT)) {
 files.push({ file: `skills:${path.relative(PUBLISHED_SKILLS_ROOT, abs)}`, abs });
 }
 return files.map(({ file, abs }) => ({ file, flat: fs.readFileSync(abs, 'utf8').replace(/\s+/g, ' ') }));
}

interface MarkdownSite {
 file: string;
 /** True when the marker opens a double-backtick code span — a taught template. */
 template: boolean;
 excerpt: string;
 ok: boolean;
}

/** Every `os migrate meta` prescription sentence in one markdown file, with its verdict. */
function judgeMarkdownSentences({ file, flat }: MarkdownFile): MarkdownSite[] {
 return [...flat.matchAll(SKILL_MARKER)].map((m) => {
 const at = m.index ?? 0;
 const rest = flat.slice(at);
 const template = flat.slice(0, at).endsWith('``');
 const ok = template
 ? SKILL_HOUSE_TEMPLATE.test(rest) || SKILL_MIXED_TEMPLATE.test(rest)
 : SKILL_HOUSE_PROSE.test(rest) || SKILL_MIXED_PROSE.test(rest);
 return { file, template, excerpt: rest.slice(0, 120), ok };
 });
}

describe('the retirement playbook and the published skill catalog agree with this pin (#10848, corpus widened #13859)', () => {
 const corpus = markdownCorpus();

 it('[#13859] anti-vacuity for the DISCOVERY: the playbook and at least one published skill', () => {
 // The corpus is this suite's input, so a walk that reached zero published
 // files would restore the exact one-file blindness #13859 is about — and
 // every assertion below would stay green while it did. Assert the shape of
 // the population itself, not a count that ages out with the catalog.
 expect(corpus.length).toBeGreaterThan(0);
 const labels = corpus.map((c) => c.file);
 expect(labels).toContain(PLAYBOOK_LABEL);
 const published = labels.filter((l) => l.startsWith('skills:'));
 expect(published.length).toBeGreaterThanOrEqual(1);
 // Deterministic ordering: the published tail is sorted, so a failure names
 // its sites in a stable order rather than in readdir order.
 expect(published).toEqual([...published].sort());
 });

 it('every prescription sentence in the markdown corpus is house-form or MIXED two-clause', () => {
 const bad = corpus.flatMap(judgeMarkdownSentences).filter((s) => !s.ok);
 expect(
 bad,
 bad.map((s) => `${s.file} [${s.template ? 'template' : 'prose'}] — "${s.excerpt}"`).join('\n'),
 ).toEqual([]);
 });

 it('anti-vacuity: the skill teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Convention 5 carries the house template and its one allowed variant
 // (the partial-conversion two-clause shape). Zero or one marker means the
 // skill stopped teaching a shape — or this scan went blind on the file.
 const rests = [...flat.matchAll(SKILL_MARKER)].map((m) => flat.slice(m.index ?? 0));
 it('anti-vacuity: the PLAYBOOK teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Pinned to the one file that OWNS both templates — never a whole-corpus
 // claim. Convention 5 carries the house template and its one allowed
 // variant (the partial-conversion two-clause shape); zero or one marker
 // means the playbook stopped teaching a shape, or this scan went blind on
 // it. A published skill that names the command once is not a regression,
 // so widening this floor to the corpus would assert nothing and fail on
 // the catalog's own editorial choices.
 const playbook = corpus.find((c) => c.file === PLAYBOOK_LABEL);
 expect(playbook, PLAYBOOK_LABEL).toBeDefined();
 const rests = [...playbook!.flat.matchAll(SKILL_MARKER)].map((m) => playbook!.flat.slice(m.index ?? 0));
 expect(rests.length).toBeGreaterThanOrEqual(2);
 expect(rests.some((r) => SKILL_HOUSE_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_HOUSE_TEMPLATE.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_TEMPLATE.test(r))).toBe(true);
 });

 it('[#9529] the withdrawn automatic-rewrite claim is absent from the skill, in every spelling', () => {
 it('[#9529] the withdrawn automatic-rewrite claim is absent from the corpus, in every spelling', () => {
 // Judged over the raw text rather than reconstruct(): a markdown line
 // opening with `*`/`**` would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only, so the skill's Chinese prose
 // cannot fabricate a match; a hit is a real regression of the ruling.
 const claims = [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60),
 // opening with an asterisk would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only BY DESIGN — the playbook's
 // Chinese prose (and any that follows it into the catalog) cannot
 // fabricate a match, so a hit is a real regression of the ruling. This is
 // the direction #13859 widened for: the claim is red wherever it appears,
 // and "wherever" now includes the published catalog.
 const claims = corpus.flatMap(({ file, flat }) =>
 [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => `${file} — "${flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60)}"`,
 ),
 );
 expect(claims, claims.join('\n')).toEqual([]);
 });

 it('[#13859] the markdown judge is not vacuous — template and prose anchors each hold', () => {
 const judge = (flat: string): MarkdownSite[] => judgeMarkdownSentences({ file: 'synthetic.md', flat });
 const house = 'Run `os migrate meta --from <N-1>` to list the mechanical edits for existing sources; apply them by hand.';
 const mixed = 'Run `os migrate meta --from 16` to list the mechanical edits for the `1y` case; the rest are reported.';
 // A taught template must close its code span, in both legal shapes.
 expect(judge(`5. \`\`${house}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 expect(judge(`\`\`${mixed}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 // …and a template that buries the command mid-span stays RED.
 expect(judge(`\`\`${house} Also do X.\`\``).map((s) => s.ok)).toEqual([false]);
 // A sentence quoted in prose or a transcript is judged on wording alone.
 expect(judge(`error text … ${house} expected: never`).map((s) => [s.template, s.ok])).toEqual([[false, true]]);
 expect(judge('Run `os migrate meta --from 16` to rewrite it automatically.').map((s) => s.ok)).toEqual([false]);
 expect(judge('Run `os migrate meta --from 16` to remove it.').map((s) => s.ok)).toEqual([false]);
 // Naming the command mid-prose without the leading `Run` is not a sentence.
 expect(judge('Stored flows convert with `os migrate meta --from 16`.')).toEqual([]);
 // The withdrawn claim trips wherever it appears, fence or prose.
 expect([...'``os migrate meta --from 16` rewrites your source files.``'.matchAll(WITHDRAWN_CLAIM)]).not.toEqual([]);
 });
});
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
test(spec): discover the migrate-sentence markdown corpus instead of naming one file#14557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

os-sam
merged 1 commit into

from

Sep 2, 2026
 Merged


test(spec): discover the migrate-sentence markdown corpus instead of naming one file
#14557
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


228 changes: 183 additions & 45 deletions
packages/spec/src/shared/retired-key-migrate-sentence.test.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -70,21 +70,36 @@ const SPEC_SRC_ROOT = path.resolve(HERE, '..');
/** #7030: `packages/lint/src`, the one other corpus carrying this sentence. */
const LINT_SRC_ROOT = path.resolve(HERE, '../../../lint/src');
/**
 * [#10848] The population widens by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/**`): the retirement
 * [#10848] The population widened by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/`): the retirement
 * playbook every new tombstone's guidance string is authored from. It sat
 * outside both corpora and prescribed the withdrawn sentence, so the skill
 * taught authors to red this very pin — and a red pin over a skill-taught
 * sentence invites weakening the PIN rather than the skill. It cannot ride
 * the corpus walk: it is markdown (the walk yields `.ts` only), its `--from`
 * operand is a placeholder like `<N-1>` (never `\d+`), its sentences end at a
 * code-span close (never at a string-literal quote), and `reconstruct()`
 * would drop every markdown line that opens with `*`/`**`. So it is judged
 * would drop every markdown line that opens with an asterisk. So it is judged
 * below as its own corpus: raw text, whitespace-normalised, with
 * placeholder-aware anchors — the withdrawn-claim direction reuses
 * `WITHDRAWN_CLAIM` verbatim.
 *
 * [#13859] ONE hard-coded path is a population of one, and the sentence does
 * not stay inside it: the withdrawn automatic-rewrite claim reached the
 * PUBLISHED skill catalog — the artifacts a customer agent actually loads, and
 * that land in codebases this repo cannot see — with nothing scanning them.
 * The corpus below is therefore DISCOVERED rather than named: this playbook
 * plus every `.md` file under `skills/`. Discovery is the input, so it is
 * itself asserted non-vacuous — a walk that silently reached zero published
 * files would leave this pin exactly as narrow as it was, while reading green.
 */
const RETIREMENT_SKILL_MD = path.resolve(HERE, '../../../../.claude/skills/spec-property-retirement/SKILL.md');
/**
 * [#13859] The published catalog root. Discovered, never listed: a checked-in
 * file list is a second population to keep in sync, and the one that rots is
 * always the list.
 */
const PUBLISHED_SKILLS_ROOT = path.resolve(HERE, '../../../../skills');

/** One scanned corpus: a root directory, plus its own out-of-scope exemptions. */
interface Corpus {
Expand DownExpand Up@@ -388,62 +403,185 @@ describe('`os migrate meta` sentences are the house sentence, across corpora (#6
});

/**
 * [#10848] Markdown-corpus anchors for the retirement skill (see the
 * `RETIREMENT_SKILL_MD` docblock). Same two legal shapes as
 * `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on exactly three axes: the
 * `--from` operand may be a placeholder (`<N-1>`, `<N>`) as well as a literal
 * major; the judged text is the whole file with runs of whitespace collapsed
 * (markdown wraps sentences mid-clause); and "last sentence of its literal"
 * becomes "last sentence of its double-backtick code span" (`.``), so prose
 * cannot bury the command mid-span either. The marker requires the leading
 * `Run` on purpose: the skill legitimately NAMES the command mid-prose
 * (`migrate meta --from <old>` in §3's `retiredFromLoadPath` bullet) without
 * prescribing a sentence — only taught sentence templates are judged.
 * [#10848] Markdown-corpus anchors (see the `RETIREMENT_SKILL_MD` docblock).
 * Same two legal shapes as `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on
 * exactly three axes: the `--from` operand may be a placeholder (`<N-1>`,
 * `<N>`) as well as a literal major; the judged text is the whole file with
 * runs of whitespace collapsed (markdown wraps sentences mid-clause); and
 * "last sentence of its literal" becomes "last sentence of its container".
 *
 * The marker requires the leading `Run` on purpose: a skill legitimately NAMES
 * the command mid-prose (`migrate meta --from <old>` in the playbook's §3
 * `retiredFromLoadPath` bullet; a dozen bare `os migrate meta …` invocations in
 * the published upgrade skill) without prescribing a sentence — only sentences
 * are judged. That property is what makes the widened population safe, so it
 * stays.
 *
 * [#13859] "Container" is TWO things once the corpus is more than the playbook,
 * and one anchor for both would be a false-positive machine. A taught TEMPLATE
 * is written as a double-backtick code span, so its sentence must close that
 * span (`.` then the span's two backticks) — prose cannot bury the command
 * mid-span, which is the property the source corpus gets from its closing
 * quote. A sentence QUOTED in prose or inside a fenced transcript has no span
 * to close: the published upgrade skill quotes the house sentence verbatim
 * inside a rendered parse error, whose next line is `expected: never`. Under a
 * span-only anchor every such occurrence is an unconditional RED — not a
 * property, since nothing but a code span could ever satisfy it, but a trap
 * that fires the moment the corpus stops being one file. So a non-span
 * occurrence is judged for its WORDING, ending at its own period. What that
 * gives up, deliberately and only in prose: burying (`… apply them by hand.
 * Also do X.`) passes there, while it stays RED inside a template and in every
 * `.ts` prescription above — the two places a reader copies text from.
 */
const SKILL_FROM_OPERAND = /(?:\d+|<[^>`]+>)/.source;
const SKILL_MARKER = new RegExp(
 `Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\``,
 'g',
);
const SKILL_HOUSE_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.\`\``,
);
const SKILL_MIXED_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+\\.\`\``,
);
/** The two legal wordings, from the marker up to the sentence's final period. */
const SKILL_HOUSE_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.`;
const SKILL_MIXED_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+?\\.`;
/** Container-final anchors: a code span closes; a quoted sentence just ends. */
const SKILL_TEMPLATE_END = '``';
const SKILL_PROSE_END = '(?:\\s|$)';
const SKILL_HOUSE_TEMPLATE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_MIXED_TEMPLATE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_HOUSE_PROSE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_PROSE_END}`);
const SKILL_MIXED_PROSE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_PROSE_END}`);

/** Label prefix for the internal playbook, mirroring the `spec:`/`lint:` convention. */
const PLAYBOOK_LABEL = 'internal:spec-property-retirement/SKILL.md';

interface MarkdownFile {
 /** Corpus-prefixed path, e.g. `skills:objectstack-upgrade/SKILL.md`. */
 file: string;
 /** Whole file, runs of whitespace collapsed. */
 flat: string;
}

describe('the spec-property-retirement skill agrees with this pin (#10848 — the one-file population widening)', () => {
 const raw = fs.readFileSync(RETIREMENT_SKILL_MD, 'utf8');
 const flat = raw.replace(/\s+/g, ' ');

 it('every prescription sentence the skill teaches is house-form or MIXED two-clause', () => {
 const bad = [...flat.matchAll(SKILL_MARKER)]
 .filter((m) => {
 const rest = flat.slice(m.index ?? 0);
 return !(SKILL_HOUSE_AT_MARKER.test(rest) || SKILL_MIXED_AT_MARKER.test(rest));
 })
 .map((m) => flat.slice(m.index ?? 0, (m.index ?? 0) + 120));
 expect(bad, bad.join('\n')).toEqual([]);
/**
 * [#13859] Deterministic markdown discovery. `readdirSync(…).sort()` so the
 * corpus order — and therefore every failure message — is identical on every
 * machine and every run. `lstatSync` so a symlink is SKIPPED rather than
 * followed: a link pointing outside the repo would make the corpus depend on
 * the checkout around it, and following one back inside would judge a file
 * twice. Nothing else is skipped, and nothing is skipped silently — the
 * discovery anti-vacuity assertion below is what notices when it is.
 */
function* walkMarkdown(dir: string): Generator<string> {
 for (const name of fs.readdirSync(dir).sort()) {
 const p = path.join(dir, name);
 const st = fs.lstatSync(p);
 if (st.isSymbolicLink()) continue;
 if (st.isDirectory()) yield* walkMarkdown(p);
 else if (st.isFile() && p.endsWith('.md')) yield p;
 }
}

/** The internal retirement playbook, then the published catalog in sorted order. */
function markdownCorpus(): MarkdownFile[] {
 const files: Array<{ file: string; abs: string }> = [{ file: PLAYBOOK_LABEL, abs: RETIREMENT_SKILL_MD }];
 for (const abs of walkMarkdown(PUBLISHED_SKILLS_ROOT)) {
 files.push({ file: `skills:${path.relative(PUBLISHED_SKILLS_ROOT, abs)}`, abs });
 }
 return files.map(({ file, abs }) => ({ file, flat: fs.readFileSync(abs, 'utf8').replace(/\s+/g, ' ') }));
}

interface MarkdownSite {
 file: string;
 /** True when the marker opens a double-backtick code span — a taught template. */
 template: boolean;
 excerpt: string;
 ok: boolean;
}

/** Every `os migrate meta` prescription sentence in one markdown file, with its verdict. */
function judgeMarkdownSentences({ file, flat }: MarkdownFile): MarkdownSite[] {
 return [...flat.matchAll(SKILL_MARKER)].map((m) => {
 const at = m.index ?? 0;
 const rest = flat.slice(at);
 const template = flat.slice(0, at).endsWith('``');
 const ok = template
 ? SKILL_HOUSE_TEMPLATE.test(rest) || SKILL_MIXED_TEMPLATE.test(rest)
 : SKILL_HOUSE_PROSE.test(rest) || SKILL_MIXED_PROSE.test(rest);
 return { file, template, excerpt: rest.slice(0, 120), ok };
 });
}

describe('the retirement playbook and the published skill catalog agree with this pin (#10848, corpus widened #13859)', () => {
 const corpus = markdownCorpus();

 it('[#13859] anti-vacuity for the DISCOVERY: the playbook and at least one published skill', () => {
 // The corpus is this suite's input, so a walk that reached zero published
 // files would restore the exact one-file blindness #13859 is about — and
 // every assertion below would stay green while it did. Assert the shape of
 // the population itself, not a count that ages out with the catalog.
 expect(corpus.length).toBeGreaterThan(0);
 const labels = corpus.map((c) => c.file);
 expect(labels).toContain(PLAYBOOK_LABEL);
 const published = labels.filter((l) => l.startsWith('skills:'));
 expect(published.length).toBeGreaterThanOrEqual(1);
 // Deterministic ordering: the published tail is sorted, so a failure names
 // its sites in a stable order rather than in readdir order.
 expect(published).toEqual([...published].sort());
 });

 it('every prescription sentence in the markdown corpus is house-form or MIXED two-clause', () => {
 const bad = corpus.flatMap(judgeMarkdownSentences).filter((s) => !s.ok);
 expect(
 bad,
 bad.map((s) => `${s.file} [${s.template ? 'template' : 'prose'}] — "${s.excerpt}"`).join('\n'),
 ).toEqual([]);
 });

 it('anti-vacuity: the skill teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Convention 5 carries the house template and its one allowed variant
 // (the partial-conversion two-clause shape). Zero or one marker means the
 // skill stopped teaching a shape — or this scan went blind on the file.
 const rests = [...flat.matchAll(SKILL_MARKER)].map((m) => flat.slice(m.index ?? 0));
 it('anti-vacuity: the PLAYBOOK teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Pinned to the one file that OWNS both templates — never a whole-corpus
 // claim. Convention 5 carries the house template and its one allowed
 // variant (the partial-conversion two-clause shape); zero or one marker
 // means the playbook stopped teaching a shape, or this scan went blind on
 // it. A published skill that names the command once is not a regression,
 // so widening this floor to the corpus would assert nothing and fail on
 // the catalog's own editorial choices.
 const playbook = corpus.find((c) => c.file === PLAYBOOK_LABEL);
 expect(playbook, PLAYBOOK_LABEL).toBeDefined();
 const rests = [...playbook!.flat.matchAll(SKILL_MARKER)].map((m) => playbook!.flat.slice(m.index ?? 0));
 expect(rests.length).toBeGreaterThanOrEqual(2);
 expect(rests.some((r) => SKILL_HOUSE_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_HOUSE_TEMPLATE.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_TEMPLATE.test(r))).toBe(true);
 });

 it('[#9529] the withdrawn automatic-rewrite claim is absent from the skill, in every spelling', () => {
 it('[#9529] the withdrawn automatic-rewrite claim is absent from the corpus, in every spelling', () => {
 // Judged over the raw text rather than reconstruct(): a markdown line
 // opening with `*`/`**` would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only, so the skill's Chinese prose
 // cannot fabricate a match; a hit is a real regression of the ruling.
 const claims = [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60),
 // opening with an asterisk would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only BY DESIGN — the playbook's
 // Chinese prose (and any that follows it into the catalog) cannot
 // fabricate a match, so a hit is a real regression of the ruling. This is
 // the direction #13859 widened for: the claim is red wherever it appears,
 // and "wherever" now includes the published catalog.
 const claims = corpus.flatMap(({ file, flat }) =>
 [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => `${file} — "${flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60)}"`,
 ),
 );
 expect(claims, claims.join('\n')).toEqual([]);
 });

 it('[#13859] the markdown judge is not vacuous — template and prose anchors each hold', () => {
 const judge = (flat: string): MarkdownSite[] => judgeMarkdownSentences({ file: 'synthetic.md', flat });
 const house = 'Run `os migrate meta --from <N-1>` to list the mechanical edits for existing sources; apply them by hand.';
 const mixed = 'Run `os migrate meta --from 16` to list the mechanical edits for the `1y` case; the rest are reported.';
 // A taught template must close its code span, in both legal shapes.
 expect(judge(`5. \`\`${house}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 expect(judge(`\`\`${mixed}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 // …and a template that buries the command mid-span stays RED.
 expect(judge(`\`\`${house} Also do X.\`\``).map((s) => s.ok)).toEqual([false]);
 // A sentence quoted in prose or a transcript is judged on wording alone.
 expect(judge(`error text … ${house} expected: never`).map((s) => [s.template, s.ok])).toEqual([[false, true]]);
 expect(judge('Run `os migrate meta --from 16` to rewrite it automatically.').map((s) => s.ok)).toEqual([false]);
 expect(judge('Run `os migrate meta --from 16` to remove it.').map((s) => s.ok)).toEqual([false]);
 // Naming the command mid-prose without the leading `Run` is not a sentence.
 expect(judge('Stored flows convert with `os migrate meta --from 16`.')).toEqual([]);
 // The withdrawn claim trips wherever it appears, fence or prose.
 expect([...'``os migrate meta --from 16` rewrites your source files.``'.matchAll(WITHDRAWN_CLAIM)]).not.toEqual([]);
 });
});
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
test(spec): discover the migrate-sentence markdown corpus instead of naming one file#14557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

os-sam
merged 1 commit into

from

Sep 2, 2026
 Merged


test(spec): discover the migrate-sentence markdown corpus instead of naming one file
#14557
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


228 changes: 183 additions & 45 deletions
packages/spec/src/shared/retired-key-migrate-sentence.test.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -70,21 +70,36 @@ const SPEC_SRC_ROOT = path.resolve(HERE, '..');
/** #7030: `packages/lint/src`, the one other corpus carrying this sentence. */
const LINT_SRC_ROOT = path.resolve(HERE, '../../../lint/src');
/**
 * [#10848] The population widens by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/**`): the retirement
 * [#10848] The population widened by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/`): the retirement
 * playbook every new tombstone's guidance string is authored from. It sat
 * outside both corpora and prescribed the withdrawn sentence, so the skill
 * taught authors to red this very pin — and a red pin over a skill-taught
 * sentence invites weakening the PIN rather than the skill. It cannot ride
 * the corpus walk: it is markdown (the walk yields `.ts` only), its `--from`
 * operand is a placeholder like `<N-1>` (never `\d+`), its sentences end at a
 * code-span close (never at a string-literal quote), and `reconstruct()`
 * would drop every markdown line that opens with `*`/`**`. So it is judged
 * would drop every markdown line that opens with an asterisk. So it is judged
 * below as its own corpus: raw text, whitespace-normalised, with
 * placeholder-aware anchors — the withdrawn-claim direction reuses
 * `WITHDRAWN_CLAIM` verbatim.
 *
 * [#13859] ONE hard-coded path is a population of one, and the sentence does
 * not stay inside it: the withdrawn automatic-rewrite claim reached the
 * PUBLISHED skill catalog — the artifacts a customer agent actually loads, and
 * that land in codebases this repo cannot see — with nothing scanning them.
 * The corpus below is therefore DISCOVERED rather than named: this playbook
 * plus every `.md` file under `skills/`. Discovery is the input, so it is
 * itself asserted non-vacuous — a walk that silently reached zero published
 * files would leave this pin exactly as narrow as it was, while reading green.
 */
const RETIREMENT_SKILL_MD = path.resolve(HERE, '../../../../.claude/skills/spec-property-retirement/SKILL.md');
/**
 * [#13859] The published catalog root. Discovered, never listed: a checked-in
 * file list is a second population to keep in sync, and the one that rots is
 * always the list.
 */
const PUBLISHED_SKILLS_ROOT = path.resolve(HERE, '../../../../skills');

/** One scanned corpus: a root directory, plus its own out-of-scope exemptions. */
interface Corpus {
Expand DownExpand Up@@ -388,62 +403,185 @@ describe('`os migrate meta` sentences are the house sentence, across corpora (#6
});

/**
 * [#10848] Markdown-corpus anchors for the retirement skill (see the
 * `RETIREMENT_SKILL_MD` docblock). Same two legal shapes as
 * `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on exactly three axes: the
 * `--from` operand may be a placeholder (`<N-1>`, `<N>`) as well as a literal
 * major; the judged text is the whole file with runs of whitespace collapsed
 * (markdown wraps sentences mid-clause); and "last sentence of its literal"
 * becomes "last sentence of its double-backtick code span" (`.``), so prose
 * cannot bury the command mid-span either. The marker requires the leading
 * `Run` on purpose: the skill legitimately NAMES the command mid-prose
 * (`migrate meta --from <old>` in §3's `retiredFromLoadPath` bullet) without
 * prescribing a sentence — only taught sentence templates are judged.
 * [#10848] Markdown-corpus anchors (see the `RETIREMENT_SKILL_MD` docblock).
 * Same two legal shapes as `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on
 * exactly three axes: the `--from` operand may be a placeholder (`<N-1>`,
 * `<N>`) as well as a literal major; the judged text is the whole file with
 * runs of whitespace collapsed (markdown wraps sentences mid-clause); and
 * "last sentence of its literal" becomes "last sentence of its container".
 *
 * The marker requires the leading `Run` on purpose: a skill legitimately NAMES
 * the command mid-prose (`migrate meta --from <old>` in the playbook's §3
 * `retiredFromLoadPath` bullet; a dozen bare `os migrate meta …` invocations in
 * the published upgrade skill) without prescribing a sentence — only sentences
 * are judged. That property is what makes the widened population safe, so it
 * stays.
 *
 * [#13859] "Container" is TWO things once the corpus is more than the playbook,
 * and one anchor for both would be a false-positive machine. A taught TEMPLATE
 * is written as a double-backtick code span, so its sentence must close that
 * span (`.` then the span's two backticks) — prose cannot bury the command
 * mid-span, which is the property the source corpus gets from its closing
 * quote. A sentence QUOTED in prose or inside a fenced transcript has no span
 * to close: the published upgrade skill quotes the house sentence verbatim
 * inside a rendered parse error, whose next line is `expected: never`. Under a
 * span-only anchor every such occurrence is an unconditional RED — not a
 * property, since nothing but a code span could ever satisfy it, but a trap
 * that fires the moment the corpus stops being one file. So a non-span
 * occurrence is judged for its WORDING, ending at its own period. What that
 * gives up, deliberately and only in prose: burying (`… apply them by hand.
 * Also do X.`) passes there, while it stays RED inside a template and in every
 * `.ts` prescription above — the two places a reader copies text from.
 */
const SKILL_FROM_OPERAND = /(?:\d+|<[^>`]+>)/.source;
const SKILL_MARKER = new RegExp(
 `Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\``,
 'g',
);
const SKILL_HOUSE_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.\`\``,
);
const SKILL_MIXED_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+\\.\`\``,
);
/** The two legal wordings, from the marker up to the sentence's final period. */
const SKILL_HOUSE_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.`;
const SKILL_MIXED_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+?\\.`;
/** Container-final anchors: a code span closes; a quoted sentence just ends. */
const SKILL_TEMPLATE_END = '``';
const SKILL_PROSE_END = '(?:\\s|$)';
const SKILL_HOUSE_TEMPLATE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_MIXED_TEMPLATE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_HOUSE_PROSE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_PROSE_END}`);
const SKILL_MIXED_PROSE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_PROSE_END}`);

/** Label prefix for the internal playbook, mirroring the `spec:`/`lint:` convention. */
const PLAYBOOK_LABEL = 'internal:spec-property-retirement/SKILL.md';

interface MarkdownFile {
 /** Corpus-prefixed path, e.g. `skills:objectstack-upgrade/SKILL.md`. */
 file: string;
 /** Whole file, runs of whitespace collapsed. */
 flat: string;
}

describe('the spec-property-retirement skill agrees with this pin (#10848 — the one-file population widening)', () => {
 const raw = fs.readFileSync(RETIREMENT_SKILL_MD, 'utf8');
 const flat = raw.replace(/\s+/g, ' ');

 it('every prescription sentence the skill teaches is house-form or MIXED two-clause', () => {
 const bad = [...flat.matchAll(SKILL_MARKER)]
 .filter((m) => {
 const rest = flat.slice(m.index ?? 0);
 return !(SKILL_HOUSE_AT_MARKER.test(rest) || SKILL_MIXED_AT_MARKER.test(rest));
 })
 .map((m) => flat.slice(m.index ?? 0, (m.index ?? 0) + 120));
 expect(bad, bad.join('\n')).toEqual([]);
/**
 * [#13859] Deterministic markdown discovery. `readdirSync(…).sort()` so the
 * corpus order — and therefore every failure message — is identical on every
 * machine and every run. `lstatSync` so a symlink is SKIPPED rather than
 * followed: a link pointing outside the repo would make the corpus depend on
 * the checkout around it, and following one back inside would judge a file
 * twice. Nothing else is skipped, and nothing is skipped silently — the
 * discovery anti-vacuity assertion below is what notices when it is.
 */
function* walkMarkdown(dir: string): Generator<string> {
 for (const name of fs.readdirSync(dir).sort()) {
 const p = path.join(dir, name);
 const st = fs.lstatSync(p);
 if (st.isSymbolicLink()) continue;
 if (st.isDirectory()) yield* walkMarkdown(p);
 else if (st.isFile() && p.endsWith('.md')) yield p;
 }
}

/** The internal retirement playbook, then the published catalog in sorted order. */
function markdownCorpus(): MarkdownFile[] {
 const files: Array<{ file: string; abs: string }> = [{ file: PLAYBOOK_LABEL, abs: RETIREMENT_SKILL_MD }];
 for (const abs of walkMarkdown(PUBLISHED_SKILLS_ROOT)) {
 files.push({ file: `skills:${path.relative(PUBLISHED_SKILLS_ROOT, abs)}`, abs });
 }
 return files.map(({ file, abs }) => ({ file, flat: fs.readFileSync(abs, 'utf8').replace(/\s+/g, ' ') }));
}

interface MarkdownSite {
 file: string;
 /** True when the marker opens a double-backtick code span — a taught template. */
 template: boolean;
 excerpt: string;
 ok: boolean;
}

/** Every `os migrate meta` prescription sentence in one markdown file, with its verdict. */
function judgeMarkdownSentences({ file, flat }: MarkdownFile): MarkdownSite[] {
 return [...flat.matchAll(SKILL_MARKER)].map((m) => {
 const at = m.index ?? 0;
 const rest = flat.slice(at);
 const template = flat.slice(0, at).endsWith('``');
 const ok = template
 ? SKILL_HOUSE_TEMPLATE.test(rest) || SKILL_MIXED_TEMPLATE.test(rest)
 : SKILL_HOUSE_PROSE.test(rest) || SKILL_MIXED_PROSE.test(rest);
 return { file, template, excerpt: rest.slice(0, 120), ok };
 });
}

describe('the retirement playbook and the published skill catalog agree with this pin (#10848, corpus widened #13859)', () => {
 const corpus = markdownCorpus();

 it('[#13859] anti-vacuity for the DISCOVERY: the playbook and at least one published skill', () => {
 // The corpus is this suite's input, so a walk that reached zero published
 // files would restore the exact one-file blindness #13859 is about — and
 // every assertion below would stay green while it did. Assert the shape of
 // the population itself, not a count that ages out with the catalog.
 expect(corpus.length).toBeGreaterThan(0);
 const labels = corpus.map((c) => c.file);
 expect(labels).toContain(PLAYBOOK_LABEL);
 const published = labels.filter((l) => l.startsWith('skills:'));
 expect(published.length).toBeGreaterThanOrEqual(1);
 // Deterministic ordering: the published tail is sorted, so a failure names
 // its sites in a stable order rather than in readdir order.
 expect(published).toEqual([...published].sort());
 });

 it('every prescription sentence in the markdown corpus is house-form or MIXED two-clause', () => {
 const bad = corpus.flatMap(judgeMarkdownSentences).filter((s) => !s.ok);
 expect(
 bad,
 bad.map((s) => `${s.file} [${s.template ? 'template' : 'prose'}] — "${s.excerpt}"`).join('\n'),
 ).toEqual([]);
 });

 it('anti-vacuity: the skill teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Convention 5 carries the house template and its one allowed variant
 // (the partial-conversion two-clause shape). Zero or one marker means the
 // skill stopped teaching a shape — or this scan went blind on the file.
 const rests = [...flat.matchAll(SKILL_MARKER)].map((m) => flat.slice(m.index ?? 0));
 it('anti-vacuity: the PLAYBOOK teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Pinned to the one file that OWNS both templates — never a whole-corpus
 // claim. Convention 5 carries the house template and its one allowed
 // variant (the partial-conversion two-clause shape); zero or one marker
 // means the playbook stopped teaching a shape, or this scan went blind on
 // it. A published skill that names the command once is not a regression,
 // so widening this floor to the corpus would assert nothing and fail on
 // the catalog's own editorial choices.
 const playbook = corpus.find((c) => c.file === PLAYBOOK_LABEL);
 expect(playbook, PLAYBOOK_LABEL).toBeDefined();
 const rests = [...playbook!.flat.matchAll(SKILL_MARKER)].map((m) => playbook!.flat.slice(m.index ?? 0));
 expect(rests.length).toBeGreaterThanOrEqual(2);
 expect(rests.some((r) => SKILL_HOUSE_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_HOUSE_TEMPLATE.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_TEMPLATE.test(r))).toBe(true);
 });

 it('[#9529] the withdrawn automatic-rewrite claim is absent from the skill, in every spelling', () => {
 it('[#9529] the withdrawn automatic-rewrite claim is absent from the corpus, in every spelling', () => {
 // Judged over the raw text rather than reconstruct(): a markdown line
 // opening with `*`/`**` would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only, so the skill's Chinese prose
 // cannot fabricate a match; a hit is a real regression of the ruling.
 const claims = [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60),
 // opening with an asterisk would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only BY DESIGN — the playbook's
 // Chinese prose (and any that follows it into the catalog) cannot
 // fabricate a match, so a hit is a real regression of the ruling. This is
 // the direction #13859 widened for: the claim is red wherever it appears,
 // and "wherever" now includes the published catalog.
 const claims = corpus.flatMap(({ file, flat }) =>
 [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => `${file} — "${flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60)}"`,
 ),
 );
 expect(claims, claims.join('\n')).toEqual([]);
 });

 it('[#13859] the markdown judge is not vacuous — template and prose anchors each hold', () => {
 const judge = (flat: string): MarkdownSite[] => judgeMarkdownSentences({ file: 'synthetic.md', flat });
 const house = 'Run `os migrate meta --from <N-1>` to list the mechanical edits for existing sources; apply them by hand.';
 const mixed = 'Run `os migrate meta --from 16` to list the mechanical edits for the `1y` case; the rest are reported.';
 // A taught template must close its code span, in both legal shapes.
 expect(judge(`5. \`\`${house}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 expect(judge(`\`\`${mixed}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 // …and a template that buries the command mid-span stays RED.
 expect(judge(`\`\`${house} Also do X.\`\``).map((s) => s.ok)).toEqual([false]);
 // A sentence quoted in prose or a transcript is judged on wording alone.
 expect(judge(`error text … ${house} expected: never`).map((s) => [s.template, s.ok])).toEqual([[false, true]]);
 expect(judge('Run `os migrate meta --from 16` to rewrite it automatically.').map((s) => s.ok)).toEqual([false]);
 expect(judge('Run `os migrate meta --from 16` to remove it.').map((s) => s.ok)).toEqual([false]);
 // Naming the command mid-prose without the leading `Run` is not a sentence.
 expect(judge('Stored flows convert with `os migrate meta --from 16`.')).toEqual([]);
 // The withdrawn claim trips wherever it appears, fence or prose.
 expect([...'``os migrate meta --from 16` rewrites your source files.``'.matchAll(WITHDRAWN_CLAIM)]).not.toEqual([]);
 });
});
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
test(spec): discover the migrate-sentence markdown corpus instead of naming one file#14557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

os-sam
merged 1 commit into

from

Sep 2, 2026
 Merged


test(spec): discover the migrate-sentence markdown corpus instead of naming one file
#14557
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


228 changes: 183 additions & 45 deletions
packages/spec/src/shared/retired-key-migrate-sentence.test.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -70,21 +70,36 @@ const SPEC_SRC_ROOT = path.resolve(HERE, '..');
/** #7030: `packages/lint/src`, the one other corpus carrying this sentence. */
const LINT_SRC_ROOT = path.resolve(HERE, '../../../lint/src');
/**
 * [#10848] The population widens by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/**`): the retirement
 * [#10848] The population widened by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/`): the retirement
 * playbook every new tombstone's guidance string is authored from. It sat
 * outside both corpora and prescribed the withdrawn sentence, so the skill
 * taught authors to red this very pin — and a red pin over a skill-taught
 * sentence invites weakening the PIN rather than the skill. It cannot ride
 * the corpus walk: it is markdown (the walk yields `.ts` only), its `--from`
 * operand is a placeholder like `<N-1>` (never `\d+`), its sentences end at a
 * code-span close (never at a string-literal quote), and `reconstruct()`
 * would drop every markdown line that opens with `*`/`**`. So it is judged
 * would drop every markdown line that opens with an asterisk. So it is judged
 * below as its own corpus: raw text, whitespace-normalised, with
 * placeholder-aware anchors — the withdrawn-claim direction reuses
 * `WITHDRAWN_CLAIM` verbatim.
 *
 * [#13859] ONE hard-coded path is a population of one, and the sentence does
 * not stay inside it: the withdrawn automatic-rewrite claim reached the
 * PUBLISHED skill catalog — the artifacts a customer agent actually loads, and
 * that land in codebases this repo cannot see — with nothing scanning them.
 * The corpus below is therefore DISCOVERED rather than named: this playbook
 * plus every `.md` file under `skills/`. Discovery is the input, so it is
 * itself asserted non-vacuous — a walk that silently reached zero published
 * files would leave this pin exactly as narrow as it was, while reading green.
 */
const RETIREMENT_SKILL_MD = path.resolve(HERE, '../../../../.claude/skills/spec-property-retirement/SKILL.md');
/**
 * [#13859] The published catalog root. Discovered, never listed: a checked-in
 * file list is a second population to keep in sync, and the one that rots is
 * always the list.
 */
const PUBLISHED_SKILLS_ROOT = path.resolve(HERE, '../../../../skills');

/** One scanned corpus: a root directory, plus its own out-of-scope exemptions. */
interface Corpus {
Expand DownExpand Up@@ -388,62 +403,185 @@ describe('`os migrate meta` sentences are the house sentence, across corpora (#6
});

/**
 * [#10848] Markdown-corpus anchors for the retirement skill (see the
 * `RETIREMENT_SKILL_MD` docblock). Same two legal shapes as
 * `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on exactly three axes: the
 * `--from` operand may be a placeholder (`<N-1>`, `<N>`) as well as a literal
 * major; the judged text is the whole file with runs of whitespace collapsed
 * (markdown wraps sentences mid-clause); and "last sentence of its literal"
 * becomes "last sentence of its double-backtick code span" (`.``), so prose
 * cannot bury the command mid-span either. The marker requires the leading
 * `Run` on purpose: the skill legitimately NAMES the command mid-prose
 * (`migrate meta --from <old>` in §3's `retiredFromLoadPath` bullet) without
 * prescribing a sentence — only taught sentence templates are judged.
 * [#10848] Markdown-corpus anchors (see the `RETIREMENT_SKILL_MD` docblock).
 * Same two legal shapes as `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on
 * exactly three axes: the `--from` operand may be a placeholder (`<N-1>`,
 * `<N>`) as well as a literal major; the judged text is the whole file with
 * runs of whitespace collapsed (markdown wraps sentences mid-clause); and
 * "last sentence of its literal" becomes "last sentence of its container".
 *
 * The marker requires the leading `Run` on purpose: a skill legitimately NAMES
 * the command mid-prose (`migrate meta --from <old>` in the playbook's §3
 * `retiredFromLoadPath` bullet; a dozen bare `os migrate meta …` invocations in
 * the published upgrade skill) without prescribing a sentence — only sentences
 * are judged. That property is what makes the widened population safe, so it
 * stays.
 *
 * [#13859] "Container" is TWO things once the corpus is more than the playbook,
 * and one anchor for both would be a false-positive machine. A taught TEMPLATE
 * is written as a double-backtick code span, so its sentence must close that
 * span (`.` then the span's two backticks) — prose cannot bury the command
 * mid-span, which is the property the source corpus gets from its closing
 * quote. A sentence QUOTED in prose or inside a fenced transcript has no span
 * to close: the published upgrade skill quotes the house sentence verbatim
 * inside a rendered parse error, whose next line is `expected: never`. Under a
 * span-only anchor every such occurrence is an unconditional RED — not a
 * property, since nothing but a code span could ever satisfy it, but a trap
 * that fires the moment the corpus stops being one file. So a non-span
 * occurrence is judged for its WORDING, ending at its own period. What that
 * gives up, deliberately and only in prose: burying (`… apply them by hand.
 * Also do X.`) passes there, while it stays RED inside a template and in every
 * `.ts` prescription above — the two places a reader copies text from.
 */
const SKILL_FROM_OPERAND = /(?:\d+|<[^>`]+>)/.source;
const SKILL_MARKER = new RegExp(
 `Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\``,
 'g',
);
const SKILL_HOUSE_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.\`\``,
);
const SKILL_MIXED_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+\\.\`\``,
);
/** The two legal wordings, from the marker up to the sentence's final period. */
const SKILL_HOUSE_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.`;
const SKILL_MIXED_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+?\\.`;
/** Container-final anchors: a code span closes; a quoted sentence just ends. */
const SKILL_TEMPLATE_END = '``';
const SKILL_PROSE_END = '(?:\\s|$)';
const SKILL_HOUSE_TEMPLATE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_MIXED_TEMPLATE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_HOUSE_PROSE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_PROSE_END}`);
const SKILL_MIXED_PROSE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_PROSE_END}`);

/** Label prefix for the internal playbook, mirroring the `spec:`/`lint:` convention. */
const PLAYBOOK_LABEL = 'internal:spec-property-retirement/SKILL.md';

interface MarkdownFile {
 /** Corpus-prefixed path, e.g. `skills:objectstack-upgrade/SKILL.md`. */
 file: string;
 /** Whole file, runs of whitespace collapsed. */
 flat: string;
}

describe('the spec-property-retirement skill agrees with this pin (#10848 — the one-file population widening)', () => {
 const raw = fs.readFileSync(RETIREMENT_SKILL_MD, 'utf8');
 const flat = raw.replace(/\s+/g, ' ');

 it('every prescription sentence the skill teaches is house-form or MIXED two-clause', () => {
 const bad = [...flat.matchAll(SKILL_MARKER)]
 .filter((m) => {
 const rest = flat.slice(m.index ?? 0);
 return !(SKILL_HOUSE_AT_MARKER.test(rest) || SKILL_MIXED_AT_MARKER.test(rest));
 })
 .map((m) => flat.slice(m.index ?? 0, (m.index ?? 0) + 120));
 expect(bad, bad.join('\n')).toEqual([]);
/**
 * [#13859] Deterministic markdown discovery. `readdirSync(…).sort()` so the
 * corpus order — and therefore every failure message — is identical on every
 * machine and every run. `lstatSync` so a symlink is SKIPPED rather than
 * followed: a link pointing outside the repo would make the corpus depend on
 * the checkout around it, and following one back inside would judge a file
 * twice. Nothing else is skipped, and nothing is skipped silently — the
 * discovery anti-vacuity assertion below is what notices when it is.
 */
function* walkMarkdown(dir: string): Generator<string> {
 for (const name of fs.readdirSync(dir).sort()) {
 const p = path.join(dir, name);
 const st = fs.lstatSync(p);
 if (st.isSymbolicLink()) continue;
 if (st.isDirectory()) yield* walkMarkdown(p);
 else if (st.isFile() && p.endsWith('.md')) yield p;
 }
}

/** The internal retirement playbook, then the published catalog in sorted order. */
function markdownCorpus(): MarkdownFile[] {
 const files: Array<{ file: string; abs: string }> = [{ file: PLAYBOOK_LABEL, abs: RETIREMENT_SKILL_MD }];
 for (const abs of walkMarkdown(PUBLISHED_SKILLS_ROOT)) {
 files.push({ file: `skills:${path.relative(PUBLISHED_SKILLS_ROOT, abs)}`, abs });
 }
 return files.map(({ file, abs }) => ({ file, flat: fs.readFileSync(abs, 'utf8').replace(/\s+/g, ' ') }));
}

interface MarkdownSite {
 file: string;
 /** True when the marker opens a double-backtick code span — a taught template. */
 template: boolean;
 excerpt: string;
 ok: boolean;
}

/** Every `os migrate meta` prescription sentence in one markdown file, with its verdict. */
function judgeMarkdownSentences({ file, flat }: MarkdownFile): MarkdownSite[] {
 return [...flat.matchAll(SKILL_MARKER)].map((m) => {
 const at = m.index ?? 0;
 const rest = flat.slice(at);
 const template = flat.slice(0, at).endsWith('``');
 const ok = template
 ? SKILL_HOUSE_TEMPLATE.test(rest) || SKILL_MIXED_TEMPLATE.test(rest)
 : SKILL_HOUSE_PROSE.test(rest) || SKILL_MIXED_PROSE.test(rest);
 return { file, template, excerpt: rest.slice(0, 120), ok };
 });
}

describe('the retirement playbook and the published skill catalog agree with this pin (#10848, corpus widened #13859)', () => {
 const corpus = markdownCorpus();

 it('[#13859] anti-vacuity for the DISCOVERY: the playbook and at least one published skill', () => {
 // The corpus is this suite's input, so a walk that reached zero published
 // files would restore the exact one-file blindness #13859 is about — and
 // every assertion below would stay green while it did. Assert the shape of
 // the population itself, not a count that ages out with the catalog.
 expect(corpus.length).toBeGreaterThan(0);
 const labels = corpus.map((c) => c.file);
 expect(labels).toContain(PLAYBOOK_LABEL);
 const published = labels.filter((l) => l.startsWith('skills:'));
 expect(published.length).toBeGreaterThanOrEqual(1);
 // Deterministic ordering: the published tail is sorted, so a failure names
 // its sites in a stable order rather than in readdir order.
 expect(published).toEqual([...published].sort());
 });

 it('every prescription sentence in the markdown corpus is house-form or MIXED two-clause', () => {
 const bad = corpus.flatMap(judgeMarkdownSentences).filter((s) => !s.ok);
 expect(
 bad,
 bad.map((s) => `${s.file} [${s.template ? 'template' : 'prose'}] — "${s.excerpt}"`).join('\n'),
 ).toEqual([]);
 });

 it('anti-vacuity: the skill teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Convention 5 carries the house template and its one allowed variant
 // (the partial-conversion two-clause shape). Zero or one marker means the
 // skill stopped teaching a shape — or this scan went blind on the file.
 const rests = [...flat.matchAll(SKILL_MARKER)].map((m) => flat.slice(m.index ?? 0));
 it('anti-vacuity: the PLAYBOOK teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Pinned to the one file that OWNS both templates — never a whole-corpus
 // claim. Convention 5 carries the house template and its one allowed
 // variant (the partial-conversion two-clause shape); zero or one marker
 // means the playbook stopped teaching a shape, or this scan went blind on
 // it. A published skill that names the command once is not a regression,
 // so widening this floor to the corpus would assert nothing and fail on
 // the catalog's own editorial choices.
 const playbook = corpus.find((c) => c.file === PLAYBOOK_LABEL);
 expect(playbook, PLAYBOOK_LABEL).toBeDefined();
 const rests = [...playbook!.flat.matchAll(SKILL_MARKER)].map((m) => playbook!.flat.slice(m.index ?? 0));
 expect(rests.length).toBeGreaterThanOrEqual(2);
 expect(rests.some((r) => SKILL_HOUSE_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_HOUSE_TEMPLATE.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_TEMPLATE.test(r))).toBe(true);
 });

 it('[#9529] the withdrawn automatic-rewrite claim is absent from the skill, in every spelling', () => {
 it('[#9529] the withdrawn automatic-rewrite claim is absent from the corpus, in every spelling', () => {
 // Judged over the raw text rather than reconstruct(): a markdown line
 // opening with `*`/`**` would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only, so the skill's Chinese prose
 // cannot fabricate a match; a hit is a real regression of the ruling.
 const claims = [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60),
 // opening with an asterisk would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only BY DESIGN — the playbook's
 // Chinese prose (and any that follows it into the catalog) cannot
 // fabricate a match, so a hit is a real regression of the ruling. This is
 // the direction #13859 widened for: the claim is red wherever it appears,
 // and "wherever" now includes the published catalog.
 const claims = corpus.flatMap(({ file, flat }) =>
 [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => `${file} — "${flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60)}"`,
 ),
 );
 expect(claims, claims.join('\n')).toEqual([]);
 });

 it('[#13859] the markdown judge is not vacuous — template and prose anchors each hold', () => {
 const judge = (flat: string): MarkdownSite[] => judgeMarkdownSentences({ file: 'synthetic.md', flat });
 const house = 'Run `os migrate meta --from <N-1>` to list the mechanical edits for existing sources; apply them by hand.';
 const mixed = 'Run `os migrate meta --from 16` to list the mechanical edits for the `1y` case; the rest are reported.';
 // A taught template must close its code span, in both legal shapes.
 expect(judge(`5. \`\`${house}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 expect(judge(`\`\`${mixed}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 // …and a template that buries the command mid-span stays RED.
 expect(judge(`\`\`${house} Also do X.\`\``).map((s) => s.ok)).toEqual([false]);
 // A sentence quoted in prose or a transcript is judged on wording alone.
 expect(judge(`error text … ${house} expected: never`).map((s) => [s.template, s.ok])).toEqual([[false, true]]);
 expect(judge('Run `os migrate meta --from 16` to rewrite it automatically.').map((s) => s.ok)).toEqual([false]);
 expect(judge('Run `os migrate meta --from 16` to remove it.').map((s) => s.ok)).toEqual([false]);
 // Naming the command mid-prose without the leading `Run` is not a sentence.
 expect(judge('Stored flows convert with `os migrate meta --from 16`.')).toEqual([]);
 // The withdrawn claim trips wherever it appears, fence or prose.
 expect([...'``os migrate meta --from 16` rewrites your source files.``'.matchAll(WITHDRAWN_CLAIM)]).not.toEqual([]);
 });
});
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
test(spec): discover the migrate-sentence markdown corpus instead of naming one file#14557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

os-sam
merged 1 commit into

from

Sep 2, 2026
 Merged


test(spec): discover the migrate-sentence markdown corpus instead of naming one file
#14557
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


228 changes: 183 additions & 45 deletions
packages/spec/src/shared/retired-key-migrate-sentence.test.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -70,21 +70,36 @@ const SPEC_SRC_ROOT = path.resolve(HERE, '..');
/** #7030: `packages/lint/src`, the one other corpus carrying this sentence. */
const LINT_SRC_ROOT = path.resolve(HERE, '../../../lint/src');
/**
 * [#10848] The population widens by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/**`): the retirement
 * [#10848] The population widened by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/`): the retirement
 * playbook every new tombstone's guidance string is authored from. It sat
 * outside both corpora and prescribed the withdrawn sentence, so the skill
 * taught authors to red this very pin — and a red pin over a skill-taught
 * sentence invites weakening the PIN rather than the skill. It cannot ride
 * the corpus walk: it is markdown (the walk yields `.ts` only), its `--from`
 * operand is a placeholder like `<N-1>` (never `\d+`), its sentences end at a
 * code-span close (never at a string-literal quote), and `reconstruct()`
 * would drop every markdown line that opens with `*`/`**`. So it is judged
 * would drop every markdown line that opens with an asterisk. So it is judged
 * below as its own corpus: raw text, whitespace-normalised, with
 * placeholder-aware anchors — the withdrawn-claim direction reuses
 * `WITHDRAWN_CLAIM` verbatim.
 *
 * [#13859] ONE hard-coded path is a population of one, and the sentence does
 * not stay inside it: the withdrawn automatic-rewrite claim reached the
 * PUBLISHED skill catalog — the artifacts a customer agent actually loads, and
 * that land in codebases this repo cannot see — with nothing scanning them.
 * The corpus below is therefore DISCOVERED rather than named: this playbook
 * plus every `.md` file under `skills/`. Discovery is the input, so it is
 * itself asserted non-vacuous — a walk that silently reached zero published
 * files would leave this pin exactly as narrow as it was, while reading green.
 */
const RETIREMENT_SKILL_MD = path.resolve(HERE, '../../../../.claude/skills/spec-property-retirement/SKILL.md');
/**
 * [#13859] The published catalog root. Discovered, never listed: a checked-in
 * file list is a second population to keep in sync, and the one that rots is
 * always the list.
 */
const PUBLISHED_SKILLS_ROOT = path.resolve(HERE, '../../../../skills');

/** One scanned corpus: a root directory, plus its own out-of-scope exemptions. */
interface Corpus {
Expand DownExpand Up@@ -388,62 +403,185 @@ describe('`os migrate meta` sentences are the house sentence, across corpora (#6
});

/**
 * [#10848] Markdown-corpus anchors for the retirement skill (see the
 * `RETIREMENT_SKILL_MD` docblock). Same two legal shapes as
 * `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on exactly three axes: the
 * `--from` operand may be a placeholder (`<N-1>`, `<N>`) as well as a literal
 * major; the judged text is the whole file with runs of whitespace collapsed
 * (markdown wraps sentences mid-clause); and "last sentence of its literal"
 * becomes "last sentence of its double-backtick code span" (`.``), so prose
 * cannot bury the command mid-span either. The marker requires the leading
 * `Run` on purpose: the skill legitimately NAMES the command mid-prose
 * (`migrate meta --from <old>` in §3's `retiredFromLoadPath` bullet) without
 * prescribing a sentence — only taught sentence templates are judged.
 * [#10848] Markdown-corpus anchors (see the `RETIREMENT_SKILL_MD` docblock).
 * Same two legal shapes as `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on
 * exactly three axes: the `--from` operand may be a placeholder (`<N-1>`,
 * `<N>`) as well as a literal major; the judged text is the whole file with
 * runs of whitespace collapsed (markdown wraps sentences mid-clause); and
 * "last sentence of its literal" becomes "last sentence of its container".
 *
 * The marker requires the leading `Run` on purpose: a skill legitimately NAMES
 * the command mid-prose (`migrate meta --from <old>` in the playbook's §3
 * `retiredFromLoadPath` bullet; a dozen bare `os migrate meta …` invocations in
 * the published upgrade skill) without prescribing a sentence — only sentences
 * are judged. That property is what makes the widened population safe, so it
 * stays.
 *
 * [#13859] "Container" is TWO things once the corpus is more than the playbook,
 * and one anchor for both would be a false-positive machine. A taught TEMPLATE
 * is written as a double-backtick code span, so its sentence must close that
 * span (`.` then the span's two backticks) — prose cannot bury the command
 * mid-span, which is the property the source corpus gets from its closing
 * quote. A sentence QUOTED in prose or inside a fenced transcript has no span
 * to close: the published upgrade skill quotes the house sentence verbatim
 * inside a rendered parse error, whose next line is `expected: never`. Under a
 * span-only anchor every such occurrence is an unconditional RED — not a
 * property, since nothing but a code span could ever satisfy it, but a trap
 * that fires the moment the corpus stops being one file. So a non-span
 * occurrence is judged for its WORDING, ending at its own period. What that
 * gives up, deliberately and only in prose: burying (`… apply them by hand.
 * Also do X.`) passes there, while it stays RED inside a template and in every
 * `.ts` prescription above — the two places a reader copies text from.
 */
const SKILL_FROM_OPERAND = /(?:\d+|<[^>`]+>)/.source;
const SKILL_MARKER = new RegExp(
 `Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\``,
 'g',
);
const SKILL_HOUSE_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.\`\``,
);
const SKILL_MIXED_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+\\.\`\``,
);
/** The two legal wordings, from the marker up to the sentence's final period. */
const SKILL_HOUSE_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.`;
const SKILL_MIXED_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+?\\.`;
/** Container-final anchors: a code span closes; a quoted sentence just ends. */
const SKILL_TEMPLATE_END = '``';
const SKILL_PROSE_END = '(?:\\s|$)';
const SKILL_HOUSE_TEMPLATE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_MIXED_TEMPLATE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_HOUSE_PROSE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_PROSE_END}`);
const SKILL_MIXED_PROSE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_PROSE_END}`);

/** Label prefix for the internal playbook, mirroring the `spec:`/`lint:` convention. */
const PLAYBOOK_LABEL = 'internal:spec-property-retirement/SKILL.md';

interface MarkdownFile {
 /** Corpus-prefixed path, e.g. `skills:objectstack-upgrade/SKILL.md`. */
 file: string;
 /** Whole file, runs of whitespace collapsed. */
 flat: string;
}

describe('the spec-property-retirement skill agrees with this pin (#10848 — the one-file population widening)', () => {
 const raw = fs.readFileSync(RETIREMENT_SKILL_MD, 'utf8');
 const flat = raw.replace(/\s+/g, ' ');

 it('every prescription sentence the skill teaches is house-form or MIXED two-clause', () => {
 const bad = [...flat.matchAll(SKILL_MARKER)]
 .filter((m) => {
 const rest = flat.slice(m.index ?? 0);
 return !(SKILL_HOUSE_AT_MARKER.test(rest) || SKILL_MIXED_AT_MARKER.test(rest));
 })
 .map((m) => flat.slice(m.index ?? 0, (m.index ?? 0) + 120));
 expect(bad, bad.join('\n')).toEqual([]);
/**
 * [#13859] Deterministic markdown discovery. `readdirSync(…).sort()` so the
 * corpus order — and therefore every failure message — is identical on every
 * machine and every run. `lstatSync` so a symlink is SKIPPED rather than
 * followed: a link pointing outside the repo would make the corpus depend on
 * the checkout around it, and following one back inside would judge a file
 * twice. Nothing else is skipped, and nothing is skipped silently — the
 * discovery anti-vacuity assertion below is what notices when it is.
 */
function* walkMarkdown(dir: string): Generator<string> {
 for (const name of fs.readdirSync(dir).sort()) {
 const p = path.join(dir, name);
 const st = fs.lstatSync(p);
 if (st.isSymbolicLink()) continue;
 if (st.isDirectory()) yield* walkMarkdown(p);
 else if (st.isFile() && p.endsWith('.md')) yield p;
 }
}

/** The internal retirement playbook, then the published catalog in sorted order. */
function markdownCorpus(): MarkdownFile[] {
 const files: Array<{ file: string; abs: string }> = [{ file: PLAYBOOK_LABEL, abs: RETIREMENT_SKILL_MD }];
 for (const abs of walkMarkdown(PUBLISHED_SKILLS_ROOT)) {
 files.push({ file: `skills:${path.relative(PUBLISHED_SKILLS_ROOT, abs)}`, abs });
 }
 return files.map(({ file, abs }) => ({ file, flat: fs.readFileSync(abs, 'utf8').replace(/\s+/g, ' ') }));
}

interface MarkdownSite {
 file: string;
 /** True when the marker opens a double-backtick code span — a taught template. */
 template: boolean;
 excerpt: string;
 ok: boolean;
}

/** Every `os migrate meta` prescription sentence in one markdown file, with its verdict. */
function judgeMarkdownSentences({ file, flat }: MarkdownFile): MarkdownSite[] {
 return [...flat.matchAll(SKILL_MARKER)].map((m) => {
 const at = m.index ?? 0;
 const rest = flat.slice(at);
 const template = flat.slice(0, at).endsWith('``');
 const ok = template
 ? SKILL_HOUSE_TEMPLATE.test(rest) || SKILL_MIXED_TEMPLATE.test(rest)
 : SKILL_HOUSE_PROSE.test(rest) || SKILL_MIXED_PROSE.test(rest);
 return { file, template, excerpt: rest.slice(0, 120), ok };
 });
}

describe('the retirement playbook and the published skill catalog agree with this pin (#10848, corpus widened #13859)', () => {
 const corpus = markdownCorpus();

 it('[#13859] anti-vacuity for the DISCOVERY: the playbook and at least one published skill', () => {
 // The corpus is this suite's input, so a walk that reached zero published
 // files would restore the exact one-file blindness #13859 is about — and
 // every assertion below would stay green while it did. Assert the shape of
 // the population itself, not a count that ages out with the catalog.
 expect(corpus.length).toBeGreaterThan(0);
 const labels = corpus.map((c) => c.file);
 expect(labels).toContain(PLAYBOOK_LABEL);
 const published = labels.filter((l) => l.startsWith('skills:'));
 expect(published.length).toBeGreaterThanOrEqual(1);
 // Deterministic ordering: the published tail is sorted, so a failure names
 // its sites in a stable order rather than in readdir order.
 expect(published).toEqual([...published].sort());
 });

 it('every prescription sentence in the markdown corpus is house-form or MIXED two-clause', () => {
 const bad = corpus.flatMap(judgeMarkdownSentences).filter((s) => !s.ok);
 expect(
 bad,
 bad.map((s) => `${s.file} [${s.template ? 'template' : 'prose'}] — "${s.excerpt}"`).join('\n'),
 ).toEqual([]);
 });

 it('anti-vacuity: the skill teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Convention 5 carries the house template and its one allowed variant
 // (the partial-conversion two-clause shape). Zero or one marker means the
 // skill stopped teaching a shape — or this scan went blind on the file.
 const rests = [...flat.matchAll(SKILL_MARKER)].map((m) => flat.slice(m.index ?? 0));
 it('anti-vacuity: the PLAYBOOK teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Pinned to the one file that OWNS both templates — never a whole-corpus
 // claim. Convention 5 carries the house template and its one allowed
 // variant (the partial-conversion two-clause shape); zero or one marker
 // means the playbook stopped teaching a shape, or this scan went blind on
 // it. A published skill that names the command once is not a regression,
 // so widening this floor to the corpus would assert nothing and fail on
 // the catalog's own editorial choices.
 const playbook = corpus.find((c) => c.file === PLAYBOOK_LABEL);
 expect(playbook, PLAYBOOK_LABEL).toBeDefined();
 const rests = [...playbook!.flat.matchAll(SKILL_MARKER)].map((m) => playbook!.flat.slice(m.index ?? 0));
 expect(rests.length).toBeGreaterThanOrEqual(2);
 expect(rests.some((r) => SKILL_HOUSE_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_HOUSE_TEMPLATE.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_TEMPLATE.test(r))).toBe(true);
 });

 it('[#9529] the withdrawn automatic-rewrite claim is absent from the skill, in every spelling', () => {
 it('[#9529] the withdrawn automatic-rewrite claim is absent from the corpus, in every spelling', () => {
 // Judged over the raw text rather than reconstruct(): a markdown line
 // opening with `*`/`**` would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only, so the skill's Chinese prose
 // cannot fabricate a match; a hit is a real regression of the ruling.
 const claims = [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60),
 // opening with an asterisk would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only BY DESIGN — the playbook's
 // Chinese prose (and any that follows it into the catalog) cannot
 // fabricate a match, so a hit is a real regression of the ruling. This is
 // the direction #13859 widened for: the claim is red wherever it appears,
 // and "wherever" now includes the published catalog.
 const claims = corpus.flatMap(({ file, flat }) =>
 [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => `${file} — "${flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60)}"`,
 ),
 );
 expect(claims, claims.join('\n')).toEqual([]);
 });

 it('[#13859] the markdown judge is not vacuous — template and prose anchors each hold', () => {
 const judge = (flat: string): MarkdownSite[] => judgeMarkdownSentences({ file: 'synthetic.md', flat });
 const house = 'Run `os migrate meta --from <N-1>` to list the mechanical edits for existing sources; apply them by hand.';
 const mixed = 'Run `os migrate meta --from 16` to list the mechanical edits for the `1y` case; the rest are reported.';
 // A taught template must close its code span, in both legal shapes.
 expect(judge(`5. \`\`${house}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 expect(judge(`\`\`${mixed}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 // …and a template that buries the command mid-span stays RED.
 expect(judge(`\`\`${house} Also do X.\`\``).map((s) => s.ok)).toEqual([false]);
 // A sentence quoted in prose or a transcript is judged on wording alone.
 expect(judge(`error text … ${house} expected: never`).map((s) => [s.template, s.ok])).toEqual([[false, true]]);
 expect(judge('Run `os migrate meta --from 16` to rewrite it automatically.').map((s) => s.ok)).toEqual([false]);
 expect(judge('Run `os migrate meta --from 16` to remove it.').map((s) => s.ok)).toEqual([false]);
 // Naming the command mid-prose without the leading `Run` is not a sentence.
 expect(judge('Stored flows convert with `os migrate meta --from 16`.')).toEqual([]);
 // The withdrawn claim trips wherever it appears, fence or prose.
 expect([...'``os migrate meta --from 16` rewrites your source files.``'.matchAll(WITHDRAWN_CLAIM)]).not.toEqual([]);
 });
});
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
test(spec): discover the migrate-sentence markdown corpus instead of naming one file#14557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

os-sam
merged 1 commit into

from

Sep 2, 2026
 Merged


test(spec): discover the migrate-sentence markdown corpus instead of naming one file
#14557
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


228 changes: 183 additions & 45 deletions
packages/spec/src/shared/retired-key-migrate-sentence.test.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -70,21 +70,36 @@ const SPEC_SRC_ROOT = path.resolve(HERE, '..');
/** #7030: `packages/lint/src`, the one other corpus carrying this sentence. */
const LINT_SRC_ROOT = path.resolve(HERE, '../../../lint/src');
/**
 * [#10848] The population widens by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/**`): the retirement
 * [#10848] The population widened by EXACTLY ONE governed file (maintainer
 * ruling 2026-08-22, deliberately not all of `.claude/`): the retirement
 * playbook every new tombstone's guidance string is authored from. It sat
 * outside both corpora and prescribed the withdrawn sentence, so the skill
 * taught authors to red this very pin — and a red pin over a skill-taught
 * sentence invites weakening the PIN rather than the skill. It cannot ride
 * the corpus walk: it is markdown (the walk yields `.ts` only), its `--from`
 * operand is a placeholder like `<N-1>` (never `\d+`), its sentences end at a
 * code-span close (never at a string-literal quote), and `reconstruct()`
 * would drop every markdown line that opens with `*`/`**`. So it is judged
 * would drop every markdown line that opens with an asterisk. So it is judged
 * below as its own corpus: raw text, whitespace-normalised, with
 * placeholder-aware anchors — the withdrawn-claim direction reuses
 * `WITHDRAWN_CLAIM` verbatim.
 *
 * [#13859] ONE hard-coded path is a population of one, and the sentence does
 * not stay inside it: the withdrawn automatic-rewrite claim reached the
 * PUBLISHED skill catalog — the artifacts a customer agent actually loads, and
 * that land in codebases this repo cannot see — with nothing scanning them.
 * The corpus below is therefore DISCOVERED rather than named: this playbook
 * plus every `.md` file under `skills/`. Discovery is the input, so it is
 * itself asserted non-vacuous — a walk that silently reached zero published
 * files would leave this pin exactly as narrow as it was, while reading green.
 */
const RETIREMENT_SKILL_MD = path.resolve(HERE, '../../../../.claude/skills/spec-property-retirement/SKILL.md');
/**
 * [#13859] The published catalog root. Discovered, never listed: a checked-in
 * file list is a second population to keep in sync, and the one that rots is
 * always the list.
 */
const PUBLISHED_SKILLS_ROOT = path.resolve(HERE, '../../../../skills');

/** One scanned corpus: a root directory, plus its own out-of-scope exemptions. */
interface Corpus {
Expand DownExpand Up@@ -388,62 +403,185 @@ describe('`os migrate meta` sentences are the house sentence, across corpora (#6
});

/**
 * [#10848] Markdown-corpus anchors for the retirement skill (see the
 * `RETIREMENT_SKILL_MD` docblock). Same two legal shapes as
 * `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on exactly three axes: the
 * `--from` operand may be a placeholder (`<N-1>`, `<N>`) as well as a literal
 * major; the judged text is the whole file with runs of whitespace collapsed
 * (markdown wraps sentences mid-clause); and "last sentence of its literal"
 * becomes "last sentence of its double-backtick code span" (`.``), so prose
 * cannot bury the command mid-span either. The marker requires the leading
 * `Run` on purpose: the skill legitimately NAMES the command mid-prose
 * (`migrate meta --from <old>` in §3's `retiredFromLoadPath` bullet) without
 * prescribing a sentence — only taught sentence templates are judged.
 * [#10848] Markdown-corpus anchors (see the `RETIREMENT_SKILL_MD` docblock).
 * Same two legal shapes as `HOUSE_AT_MARKER`/`MIXED_AT_MARKER`, adapted on
 * exactly three axes: the `--from` operand may be a placeholder (`<N-1>`,
 * `<N>`) as well as a literal major; the judged text is the whole file with
 * runs of whitespace collapsed (markdown wraps sentences mid-clause); and
 * "last sentence of its literal" becomes "last sentence of its container".
 *
 * The marker requires the leading `Run` on purpose: a skill legitimately NAMES
 * the command mid-prose (`migrate meta --from <old>` in the playbook's §3
 * `retiredFromLoadPath` bullet; a dozen bare `os migrate meta …` invocations in
 * the published upgrade skill) without prescribing a sentence — only sentences
 * are judged. That property is what makes the widened population safe, so it
 * stays.
 *
 * [#13859] "Container" is TWO things once the corpus is more than the playbook,
 * and one anchor for both would be a false-positive machine. A taught TEMPLATE
 * is written as a double-backtick code span, so its sentence must close that
 * span (`.` then the span's two backticks) — prose cannot bury the command
 * mid-span, which is the property the source corpus gets from its closing
 * quote. A sentence QUOTED in prose or inside a fenced transcript has no span
 * to close: the published upgrade skill quotes the house sentence verbatim
 * inside a rendered parse error, whose next line is `expected: never`. Under a
 * span-only anchor every such occurrence is an unconditional RED — not a
 * property, since nothing but a code span could ever satisfy it, but a trap
 * that fires the moment the corpus stops being one file. So a non-span
 * occurrence is judged for its WORDING, ending at its own period. What that
 * gives up, deliberately and only in prose: burying (`… apply them by hand.
 * Also do X.`) passes there, while it stays RED inside a template and in every
 * `.ts` prescription above — the two places a reader copies text from.
 */
const SKILL_FROM_OPERAND = /(?:\d+|<[^>`]+>)/.source;
const SKILL_MARKER = new RegExp(
 `Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\``,
 'g',
);
const SKILL_HOUSE_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.\`\``,
);
const SKILL_MIXED_AT_MARKER = new RegExp(
 `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+\\.\`\``,
);
/** The two legal wordings, from the marker up to the sentence's final period. */
const SKILL_HOUSE_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for existing sources; apply them by hand\\.`;
const SKILL_MIXED_BODY = `^Run \`os migrate meta --from ${SKILL_FROM_OPERAND}\` to list the mechanical edits for the [^;]+ case[^;]*; [^;]+?\\.`;
/** Container-final anchors: a code span closes; a quoted sentence just ends. */
const SKILL_TEMPLATE_END = '``';
const SKILL_PROSE_END = '(?:\\s|$)';
const SKILL_HOUSE_TEMPLATE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_MIXED_TEMPLATE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_TEMPLATE_END}`);
const SKILL_HOUSE_PROSE = new RegExp(`${SKILL_HOUSE_BODY}${SKILL_PROSE_END}`);
const SKILL_MIXED_PROSE = new RegExp(`${SKILL_MIXED_BODY}${SKILL_PROSE_END}`);

/** Label prefix for the internal playbook, mirroring the `spec:`/`lint:` convention. */
const PLAYBOOK_LABEL = 'internal:spec-property-retirement/SKILL.md';

interface MarkdownFile {
 /** Corpus-prefixed path, e.g. `skills:objectstack-upgrade/SKILL.md`. */
 file: string;
 /** Whole file, runs of whitespace collapsed. */
 flat: string;
}

describe('the spec-property-retirement skill agrees with this pin (#10848 — the one-file population widening)', () => {
 const raw = fs.readFileSync(RETIREMENT_SKILL_MD, 'utf8');
 const flat = raw.replace(/\s+/g, ' ');

 it('every prescription sentence the skill teaches is house-form or MIXED two-clause', () => {
 const bad = [...flat.matchAll(SKILL_MARKER)]
 .filter((m) => {
 const rest = flat.slice(m.index ?? 0);
 return !(SKILL_HOUSE_AT_MARKER.test(rest) || SKILL_MIXED_AT_MARKER.test(rest));
 })
 .map((m) => flat.slice(m.index ?? 0, (m.index ?? 0) + 120));
 expect(bad, bad.join('\n')).toEqual([]);
/**
 * [#13859] Deterministic markdown discovery. `readdirSync(…).sort()` so the
 * corpus order — and therefore every failure message — is identical on every
 * machine and every run. `lstatSync` so a symlink is SKIPPED rather than
 * followed: a link pointing outside the repo would make the corpus depend on
 * the checkout around it, and following one back inside would judge a file
 * twice. Nothing else is skipped, and nothing is skipped silently — the
 * discovery anti-vacuity assertion below is what notices when it is.
 */
function* walkMarkdown(dir: string): Generator<string> {
 for (const name of fs.readdirSync(dir).sort()) {
 const p = path.join(dir, name);
 const st = fs.lstatSync(p);
 if (st.isSymbolicLink()) continue;
 if (st.isDirectory()) yield* walkMarkdown(p);
 else if (st.isFile() && p.endsWith('.md')) yield p;
 }
}

/** The internal retirement playbook, then the published catalog in sorted order. */
function markdownCorpus(): MarkdownFile[] {
 const files: Array<{ file: string; abs: string }> = [{ file: PLAYBOOK_LABEL, abs: RETIREMENT_SKILL_MD }];
 for (const abs of walkMarkdown(PUBLISHED_SKILLS_ROOT)) {
 files.push({ file: `skills:${path.relative(PUBLISHED_SKILLS_ROOT, abs)}`, abs });
 }
 return files.map(({ file, abs }) => ({ file, flat: fs.readFileSync(abs, 'utf8').replace(/\s+/g, ' ') }));
}

interface MarkdownSite {
 file: string;
 /** True when the marker opens a double-backtick code span — a taught template. */
 template: boolean;
 excerpt: string;
 ok: boolean;
}

/** Every `os migrate meta` prescription sentence in one markdown file, with its verdict. */
function judgeMarkdownSentences({ file, flat }: MarkdownFile): MarkdownSite[] {
 return [...flat.matchAll(SKILL_MARKER)].map((m) => {
 const at = m.index ?? 0;
 const rest = flat.slice(at);
 const template = flat.slice(0, at).endsWith('``');
 const ok = template
 ? SKILL_HOUSE_TEMPLATE.test(rest) || SKILL_MIXED_TEMPLATE.test(rest)
 : SKILL_HOUSE_PROSE.test(rest) || SKILL_MIXED_PROSE.test(rest);
 return { file, template, excerpt: rest.slice(0, 120), ok };
 });
}

describe('the retirement playbook and the published skill catalog agree with this pin (#10848, corpus widened #13859)', () => {
 const corpus = markdownCorpus();

 it('[#13859] anti-vacuity for the DISCOVERY: the playbook and at least one published skill', () => {
 // The corpus is this suite's input, so a walk that reached zero published
 // files would restore the exact one-file blindness #13859 is about — and
 // every assertion below would stay green while it did. Assert the shape of
 // the population itself, not a count that ages out with the catalog.
 expect(corpus.length).toBeGreaterThan(0);
 const labels = corpus.map((c) => c.file);
 expect(labels).toContain(PLAYBOOK_LABEL);
 const published = labels.filter((l) => l.startsWith('skills:'));
 expect(published.length).toBeGreaterThanOrEqual(1);
 // Deterministic ordering: the published tail is sorted, so a failure names
 // its sites in a stable order rather than in readdir order.
 expect(published).toEqual([...published].sort());
 });

 it('every prescription sentence in the markdown corpus is house-form or MIXED two-clause', () => {
 const bad = corpus.flatMap(judgeMarkdownSentences).filter((s) => !s.ok);
 expect(
 bad,
 bad.map((s) => `${s.file} [${s.template ? 'template' : 'prose'}] — "${s.excerpt}"`).join('\n'),
 ).toEqual([]);
 });

 it('anti-vacuity: the skill teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Convention 5 carries the house template and its one allowed variant
 // (the partial-conversion two-clause shape). Zero or one marker means the
 // skill stopped teaching a shape — or this scan went blind on the file.
 const rests = [...flat.matchAll(SKILL_MARKER)].map((m) => flat.slice(m.index ?? 0));
 it('anti-vacuity: the PLAYBOOK teaches BOTH shapes, so the scan judges at least two sites', () => {
 // Pinned to the one file that OWNS both templates — never a whole-corpus
 // claim. Convention 5 carries the house template and its one allowed
 // variant (the partial-conversion two-clause shape); zero or one marker
 // means the playbook stopped teaching a shape, or this scan went blind on
 // it. A published skill that names the command once is not a regression,
 // so widening this floor to the corpus would assert nothing and fail on
 // the catalog's own editorial choices.
 const playbook = corpus.find((c) => c.file === PLAYBOOK_LABEL);
 expect(playbook, PLAYBOOK_LABEL).toBeDefined();
 const rests = [...playbook!.flat.matchAll(SKILL_MARKER)].map((m) => playbook!.flat.slice(m.index ?? 0));
 expect(rests.length).toBeGreaterThanOrEqual(2);
 expect(rests.some((r) => SKILL_HOUSE_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_AT_MARKER.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_HOUSE_TEMPLATE.test(r))).toBe(true);
 expect(rests.some((r) => SKILL_MIXED_TEMPLATE.test(r))).toBe(true);
 });

 it('[#9529] the withdrawn automatic-rewrite claim is absent from the skill, in every spelling', () => {
 it('[#9529] the withdrawn automatic-rewrite claim is absent from the corpus, in every spelling', () => {
 // Judged over the raw text rather than reconstruct(): a markdown line
 // opening with `*`/`**` would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only, so the skill's Chinese prose
 // cannot fabricate a match; a hit is a real regression of the ruling.
 const claims = [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60),
 // opening with an asterisk would be dropped as a comment line, hiding a
 // claim. WITHDRAWN_CLAIM is English-only BY DESIGN — the playbook's
 // Chinese prose (and any that follows it into the catalog) cannot
 // fabricate a match, so a hit is a real regression of the ruling. This is
 // the direction #13859 widened for: the claim is red wherever it appears,
 // and "wherever" now includes the published catalog.
 const claims = corpus.flatMap(({ file, flat }) =>
 [...flat.matchAll(WITHDRAWN_CLAIM)].map(
 (m) => `${file} — "${flat.slice(Math.max(0, (m.index ?? 0) - 60), (m.index ?? 0) + 60)}"`,
 ),
 );
 expect(claims, claims.join('\n')).toEqual([]);
 });

 it('[#13859] the markdown judge is not vacuous — template and prose anchors each hold', () => {
 const judge = (flat: string): MarkdownSite[] => judgeMarkdownSentences({ file: 'synthetic.md', flat });
 const house = 'Run `os migrate meta --from <N-1>` to list the mechanical edits for existing sources; apply them by hand.';
 const mixed = 'Run `os migrate meta --from 16` to list the mechanical edits for the `1y` case; the rest are reported.';
 // A taught template must close its code span, in both legal shapes.
 expect(judge(`5. \`\`${house}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 expect(judge(`\`\`${mixed}\`\``).map((s) => [s.template, s.ok])).toEqual([[true, true]]);
 // …and a template that buries the command mid-span stays RED.
 expect(judge(`\`\`${house} Also do X.\`\``).map((s) => s.ok)).toEqual([false]);
 // A sentence quoted in prose or a transcript is judged on wording alone.
 expect(judge(`error text … ${house} expected: never`).map((s) => [s.template, s.ok])).toEqual([[false, true]]);
 expect(judge('Run `os migrate meta --from 16` to rewrite it automatically.').map((s) => s.ok)).toEqual([false]);
 expect(judge('Run `os migrate meta --from 16` to remove it.').map((s) => s.ok)).toEqual([false]);
 // Naming the command mid-prose without the leading `Run` is not a sentence.
 expect(judge('Stored flows convert with `os migrate meta --from 16`.')).toEqual([]);
 // The withdrawn claim trips wherever it appears, fence or prose.
 expect([...'``os migrate meta --from 16` rewrites your source files.``'.matchAll(WITHDRAWN_CLAIM)]).not.toEqual([]);
 });
});
Loading