Skip to content

fix(scripts): create the SDUI manifest output directory, and stop blaming Playwright for every dump failure#10162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
os-zhuang merged 1 commit into from
Aug 20, 2026
+348 −8
Merged

fix(scripts): create the SDUI manifest output directory, and stop blaming Playwright for every dump failure #10162

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
238 changes: 238 additions & 0 deletions packages/spec/scripts/gen-sdui-manifest-write-target.test.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,238 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
//
// Pins two contracts of `scripts/gen-sdui-manifest.sh` that were measured
// failing together, in one run, on a tree whose console build was broken.
//
// ## What was measured
//
// `packages/console/dist/` is gitignored and exists only after a successful
// `scripts/build-console.sh`. Nothing created it, and objectui's dumper calls
// `writeFileSync` straight out, so with the directory absent the run died:
//
// [dump] enumerated registry over http://localhost:5180 (HTTP 200)
// Error: ENOENT: no such file or directory, open '.../sdui.manifest.json'
// ✗ manifest generation failed (exit 1).
// If Playwright reported a missing browser, install it and retry:
// pnpm exec playwright install chromium-headless-shell
//
// Two separate defects in those five lines:
//
// 1. The ENOENT arrived AFTER a vite dev server and a chromium launch had
// been paid for, and the ratchet never needed the built dist in the first
// place — this script drives a vite DEV server over `.cache/objectui-*`,
// and `dist/` is only where the manifest lands. So a broken console build
// made the ADR-0082 D4 ratchet unrunnable for want of one `mkdir`.
//
// 2. The remedy printed was Playwright's, for a failure in which the browser
// had already done its job. Following it costs a round on the wrong layer,
// and inside an agent dispatch container it cannot even be followed —
// `PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1` is set there.
//
// ## Why these are executed assertions and not greps
//
// A grep for `mkdir -p` passes against one written after the dump, which fixes
// nothing about (1) — the cost is in the ORDER, so the order is what is
// measured: the failing-precondition case asserts the dev server was spawned
// ZERO times. And a grep for the remedy string cannot tell a remedy that is
// printed always from one printed on evidence, which is the whole of (2), so
// the real classifier is sourced and fed real failure text.
//
// The vacuity guards carry as much weight as the assertions. `DUMP_REACHED_
// SERVER` proves the stub dumper genuinely talked to the server the script
// spawned (otherwise "the run completed" could mean nothing ran), `DIST_EXISTED_
// BEFORE` proves the directory was really absent, and the Playwright case proves
// the classifier still CAN print the install remedy — without it, "never prints
// Playwright advice" would be green for a script that prints no advice at all.
//
// No vite, no chromium and no console build: vite stands in as a one-line http
// server and the dumper as a node stub that writes OUT with `writeFileSync`
// exactly as objectui's does. The script is COPIED into a temp tree, which is
// what relocates it — `FRAMEWORK_ROOT` is derived from the script's own path,
// so the copy moves `TARGET` into the temp tree and this test cannot write into
// the real `packages/console/dist/`. Ports come from the script's own picker,
// so this cannot collide with a concurrent agent.

import { describe, it, expect, afterAll } from 'vitest';
import { execFileSync, spawnSync } from 'node:child_process';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

const HERE = path.dirname(fileURLToPath(import.meta.url));
const SCRIPT = path.resolve(HERE, '..', '..', '..', 'scripts', 'gen-sdui-manifest.sh');

function have(bin: string): boolean {
try {
execFileSync('sh', ['-c', `command -v ${bin}`], { stdio: 'ignore' });
return true;
} catch {
return false;
}
}

// Linux-only by construction, like the cleanup and collision tests beside it:
// the failure being pinned is an agent-container one, and the script's lifecycle
// helpers read process sessions.
const RUNNABLE =
process.platform === 'linux' && ['setsid', 'pgrep', 'curl', 'node'].every(have);

const FAKE_SHA = '0123456789abcdef0123456789abcdef01234567';
const trees: string[] = [];

/** Stands in for objectui's chromium dumper: reach the dev server, then write OUT. */
const DUMPER_STUB = [
"import { writeFileSync } from 'node:fs';",
'const res = await fetch(process.env.BASE_URL);',
'console.log(`[dump] enumerated registry over ${process.env.BASE_URL} (HTTP ${res.status})`);',
"writeFileSync(process.env.OUT, JSON.stringify({ blocks: [] }));",
].join('\n');

/** Stands in for `pnpm --filter @object-ui/console exec vite dev --port N --strictPort`. */
const PNPM_STUB = [
'#!/usr/bin/env bash',
'port=""; prev=""',
'for a in "$@"; do [[ "$prev" == "--port" ]] && port="$a"; prev="$a"; done',
'if [[ -n "$port" ]]; then',
' printf "spawned %s\\n" "$port" >> "$SDUI_TEST_SPAWN_LOG"',
' exec node -e \'require("node:http").createServer((_q,r)=>{r.writeHead(200);r.end("DEV")}).listen(Number(process.argv[1]),"127.0.0.1")\' "$port"',
'fi',
'exit 0',
].join('\n');

interface Run {
exit: number;
output: string;
spawns: number;
manifest: boolean;
distExistedBefore: boolean;
}

/**
* Build an isolated framework root around a COPY of the real script and run it.
* `blockTarget` makes `TARGET`'s parent a regular FILE, so `mkdir -p` cannot
* succeed even for root — chmod would prove nothing in a container running as
* root, which is where this script's failures are diagnosed.
*/
function runScript(opts: { blockTarget?: boolean } = {}): Run {
const T = fs.mkdtempSync(path.join(os.tmpdir(), 'sdui-write-target-'));
trees.push(T);

fs.mkdirSync(path.join(T, 'scripts'), { recursive: true });
fs.mkdirSync(path.join(T, 'bin'), { recursive: true });
fs.copyFileSync(SCRIPT, path.join(T, 'scripts', 'gen-sdui-manifest.sh'));
fs.writeFileSync(path.join(T, '.objectui-sha'), `${FAKE_SHA}\n`);

const build = path.join(T, '.cache', `objectui-${FAKE_SHA.slice(0, 12)}`);
fs.mkdirSync(path.join(build, 'apps', 'console', 'dev'), { recursive: true });
fs.mkdirSync(path.join(build, 'scripts'), { recursive: true });
fs.writeFileSync(path.join(build, 'apps', 'console', 'dev', 'manifest-dump.html'), '');
fs.writeFileSync(path.join(build, 'scripts', 'dump-public-manifest.mjs'), DUMPER_STUB);

const pnpm = path.join(T, 'bin', 'pnpm');
fs.writeFileSync(pnpm, PNPM_STUB, { mode: 0o755 });

if (opts.blockTarget) {
fs.mkdirSync(path.join(T, 'packages'), { recursive: true });
fs.writeFileSync(path.join(T, 'packages', 'console'), 'not a directory');
}

const manifest = path.join(T, 'packages', 'console', 'dist', 'sdui.manifest.json');
const spawnLog = path.join(T, 'spawned.log');
fs.writeFileSync(spawnLog, '');
const distExistedBefore = fs.existsSync(path.dirname(manifest));

const res = spawnSync('bash', [path.join(T, 'scripts', 'gen-sdui-manifest.sh')], {
encoding: 'utf8',
timeout: 120_000,
env: {
...process.env,
PATH: `${path.join(T, 'bin')}:${process.env.PATH ?? ''}`,
SDUI_TEST_SPAWN_LOG: spawnLog,
},
});

return {
exit: res.status ?? -1,
output: `${res.stdout ?? ''}${res.stderr ?? ''}`,
spawns: fs.readFileSync(spawnLog, 'utf8').split('\n').filter(Boolean).length,
manifest: fs.existsSync(manifest),
distExistedBefore,
};
}

/** Source the real script and run its real classifier over `text`. */
function advice(text: string, out: string): string {
const T = fs.mkdtempSync(path.join(os.tmpdir(), 'sdui-advice-'));
trees.push(T);
const log = path.join(T, 'dump.log');
fs.writeFileSync(log, text);
return execFileSync(
'bash',
[
'-c',
`source ${JSON.stringify(SCRIPT)}; sdui_dump_failure_advice ${JSON.stringify(log)} ${JSON.stringify(out)}`,
],
{ encoding: 'utf8', timeout: 30_000 },
);
}

afterAll(() => {
for (const t of trees) fs.rmSync(t, { recursive: true, force: true });
});

const INSTALL_REMEDY = 'playwright install chromium-headless-shell';
const OUT_PATH = '/tmp/whatever/packages/console/dist/sdui.manifest.json';

describe.skipIf(!RUNNABLE)('gen-sdui-manifest.sh output-directory precondition', () => {
const ok = RUNNABLE ? runScript() : ({} as Run);

it('creates its output directory instead of dying on it', () => {
// Vacuity first: absent beforehand, and the dump really reached the server.
expect(ok.distExistedBefore).toBe(false);
expect(ok.output).toContain('[dump] enumerated registry over');

expect(ok.exit).toBe(0);
expect(ok.manifest).toBe(true);
expect(ok.output).not.toContain('ENOENT');
});

it('fails the precondition BEFORE paying for a dev server and a browser', () => {
const blocked = runScript({ blockTarget: true });
expect(blocked.exit).not.toBe(0);
expect(blocked.output).toContain('could not create the manifest output directory');
// The point of the whole card: the cost came before the diagnosis.
expect(blocked.spawns).toBe(0);
// And the successful run above did spawn one, so `0` here is a measurement
// rather than a harness that never spawns anything.
expect(ok.spawns).toBe(1);
});
});

describe.skipIf(!RUNNABLE)('gen-sdui-manifest.sh dump-failure diagnosis', () => {
it('prints the Playwright remedy when Playwright is what failed', () => {
const text = [
"browserType.launch: Executable doesn't exist at",
'/opt/pw-browsers/chromium_headless_shell-1234/chrome-headless-shell-linux64/chrome-headless-shell',
].join(' ');
expect(advice(text, OUT_PATH)).toContain(INSTALL_REMEDY);
});

it('does not blame Playwright for a write failure', () => {
const text = [
'[dump] enumerated registry over http://localhost:5180 (HTTP 200)',
`Error: ENOENT: no such file or directory, open '${OUT_PATH}'`,
' at writeFileSync (node:fs:2430:20)',
].join('\n');
const said = advice(text, OUT_PATH);
expect(said).not.toContain(INSTALL_REMEDY);
expect(said).toContain('could not WRITE its output');
expect(said).toContain(path.dirname(OUT_PATH));
});

it('says so plainly when it recognises neither, rather than defaulting to Playwright', () => {
const said = advice("TypeError: Cannot read properties of undefined (reading 'blocks')", OUT_PATH);
expect(said).not.toContain(INSTALL_REMEDY);
expect(said).toContain('NOT identified as a missing Playwright browser');
});
});
118 changes: 110 additions & 8 deletions scripts/gen-sdui-manifest.sh
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -276,6 +276,77 @@ sdui_on_signal() {
kill -"$sig" "$$"
}

# ---------------------------------------------------------------------------
# The remedy for a failed dump is CHOSEN from what failed, not fixed in advance.
#
# WHY. Every failure of the dump below used to print one remedy — `playwright
# install chromium-headless-shell` — because a missing browser was the failure
# the author had in hand. Measured with `packages/console/dist/` absent: the
# dump itself SUCCEEDED (the browser launched, the registry was enumerated) and
# only the final `writeFileSync` failed —
#
# Error: ENOENT: no such file or directory, open '.../sdui.manifest.json'
#
# — and the run still printed the Playwright remedy. A reader who follows it
# reinstalls a browser that was never the problem; in an agent dispatch
# container they cannot even do that, because `PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1`
# is set there and agents are instructed not to override it (see
# docs/releases-maintenance.md, "If the dispatch container's Playwright browser
# doesn't match the revision"). Advice that names the wrong layer with
# confidence costs MORE than no advice, because it is followed — that round was
# spent on the Playwright trail while the defect was one absent directory.
#
# So: the Playwright remedy is printed only on Playwright EVIDENCE, and the
# fallback is "unclassified", never "Playwright". The direction of that default
# is the whole fix; a classifier that guesses Playwright when it recognises
# nothing is the same defect wearing a conditional.
#
# The signatures are the ones this repo has actually measured, not invented:
# `browserType.launch: Executable doesn't exist at ...` is verbatim what the
# container's revision mismatch produced (docs/releases-maintenance.md), and the
# install banner is what playwright prints beside it.
#
# Reads the log as a FILE rather than piping text into grep: `grep -q` exits on
# the first match, and under `set -o pipefail` the SIGPIPE'd producer upstream
# would then set the pipeline non-zero — turning a match into a miss for exactly
# the long outputs that need classifying most.
#
# $1 file holding the dump's combined output
# $2 the OUT path the dump was given
sdui_dump_failure_advice() {
local log="${1:-}" out="${2:-}"

if [[ -n "$log" && -r "$log" ]] &&
grep -qaE "browserType\.launch|Executable doesn't exist|playwright install|download new browsers" -- "$log"; then
echo " Playwright could not start a browser — that IS this failure. Install the"
echo " matching one and retry:"
echo " pnpm exec playwright install chromium-headless-shell"
echo " Can't install here (e.g. an agent dispatch container)? See docs/releases-maintenance.md"
echo " 'If the dispatch container's Playwright browser doesn't match the revision'."
return 0
fi

# A write failure names the path it could not open, so requiring BOTH the
# errno and this run's own OUT path keeps an unrelated ENOENT elsewhere in the
# output from being read as one.
if [[ -n "$log" && -r "$log" && -n "$out" ]] &&
grep -qaE '(ENOENT|EACCES|EROFS|ENOSPC|EISDIR|EPERM)' -- "$log" &&
grep -qaF -- "$out" "$log"; then
echo " The dump could not WRITE its output. The browser side is not implicated"
echo " and 'playwright install' would change nothing here."
echo " output file: ${out}"
echo " its directory: $(dirname -- "$out")"
echo " Check that directory exists and is writable, then re-run."
return 0
fi

echo " This failure was NOT identified as a missing Playwright browser, so"
echo " 'pnpm exec playwright install ...' is not the indicated remedy for it."
echo " Read the dump's own output above first."
if [[ -n "$log" ]]; then echo " It is saved at ${log}"; fi
return 0
}

# Sourced rather than executed: publish the lifecycle helpers above and do
# nothing else, so the cleanup contract can be exercised without a console
# build. `${BASH_SOURCE[0]}` differs from `$0` exactly when this file is sourced.
Expand DownExpand Up@@ -308,6 +379,27 @@ if [[ ! -f "$DUMP_PAGE" || ! -f "$DUMP_SCRIPT" ]]; then
exit 0
fi

# The output directory, created HERE — with the other preconditions, before the
# dev server and the browser, rather than implicitly at write time.
#
# `packages/console/dist/` is gitignored and exists only after a successful
# `scripts/build-console.sh`, so on a tree whose console build is broken it is
# simply ABSENT. Nothing created it and objectui's dumper calls `writeFileSync`
# straight out, so the run died `ENOENT ... sdui.manifest.json` — after it had
# already paid for a vite dev server and a chromium launch. That cost is why
# this is a precondition and not a `mkdir` beside the write: a precondition that
# can only fail cheaply is one nobody has to debug at the end of a long run.
#
# It also removes a false coupling. The ratchet does not need the built dist at
# all — this script drives a vite DEV server over `.cache/objectui-<sha>`, and
# `dist/` is only where the manifest LANDS — so refusing to run for want of a
# directory left the ADR-0082 D4 ratchet unrunnable exactly on the trees that
# most want an independent read on the registry.
if ! mkdir -p "$TARGET"; then
echo "✗ could not create the manifest output directory: ${TARGET}"
exit 1
fi

echo "→ Generating SDUI public-tier manifest (ADR-0080) from objectui@${PINNED_SHA:0:12}..."
pushd "$BUILD_ROOT" > /dev/null

Expand DownExpand Up@@ -363,18 +455,28 @@ if ! sdui_wait_for_own_server "$DUMP_PID_FILE" "$DUMP_PORT" 90; then
exit 1
fi

if BASE_URL="http://localhost:${DUMP_PORT}" OUT="${TARGET}/sdui.manifest.json" node scripts/dump-public-manifest.mjs; then
# The dump's combined output goes to a per-run file as well as to the terminal:
# the failure branch classifies that text, and a remedy chosen from what
# actually happened is the point (see sdui_dump_failure_advice above).
DUMP_OUT_LOG="$(mktemp "${TMPDIR:-/tmp}/sdui-dump-out.XXXXXX.log")"

# `${PIPESTATUS[0]}`, never `$?`: after a pipeline `$?` is TEE's status, and tee
# does not fail, so `$?` here would read every failure of the dump as a success.
set +e
BASE_URL="http://localhost:${DUMP_PORT}" OUT="${TARGET}/sdui.manifest.json" \
node scripts/dump-public-manifest.mjs 2>&1 | tee "$DUMP_OUT_LOG"
dump_status="${PIPESTATUS[0]}"
set -e

if [[ "$dump_status" -eq 0 ]]; then
echo "✓ wrote ${TARGET}/sdui.manifest.json"
rm -f "$DUMP_OUT_LOG"
else
status=$?
echo "✗ manifest generation failed (exit ${status})."
echo " If Playwright reported a missing browser, install it and retry:"
echo " pnpm exec playwright install chromium-headless-shell"
echo " Can't install here (e.g. an agent dispatch container)? See docs/releases-maintenance.md"
echo " 'If the dispatch container's Playwright browser doesn't match the revision'."
echo "✗ manifest generation failed (exit ${dump_status})."
sdui_dump_failure_advice "$DUMP_OUT_LOG" "${TARGET}/sdui.manifest.json"
echo " The dev server log for THIS run is at ${DUMP_DEV_LOG}"
popd > /dev/null
exit "$status"
exit "$dump_status"
fi
popd > /dev/null

Expand Down
Loading