Skip to content

publish-smoke.sh smoke-tests whatever answers on its port — objectstack dev auto-shifts, so a concurrent run gets asserted against the wrong app #9647

Description

@os-steve

Filed unassigned by the dev seat working #9578 (the gen-sdui-manifest.sh fixed-port collision). Not fixed there — different file, different blast radius, and that card's surface was scoped to the SDUI dump script. Found by the concurrency sweep that card asked for, not from a code read alone.

The shape

scripts/publish-smoke.sh picks its dev-server port from a fixed default:

  • SMOKE_PORT="${SMOKE_PORT:-3210}" (line 67)
  • BASE_URL="http://localhost:$SMOKE_PORT" (line 78)
  • ... ./node_modules/.bin/objectstack dev --port "$SMOKE_PORT" --fresh (line 302)

Agent dispatch containers run several agents against one filesystem and one network namespace, so the default is shared state between concurrent runs.

This script is better off than the one in #9578 in two of three ways, which is why it is worth stating precisely rather than filing as "same bug":

  • its work dir and server log are mktemp-qualified, so there is no log clobber;
  • the port is env-overridable (SMOKE_PORT), so a caller who knows about the problem can dodge it;
  • but the default is still one shared literal, and BASE_URL still derives from it.

Why the port matters here specifically — verified

objectstack devauto-shifts off a busy port. packages/cli/src/commands/serve.ts:

const portAutoShiftAllowed = flags.dev || process.env.NODE_ENV === 'development';
if (portAutoShiftAllowed) {
try { port = await getAvailablePort(requestedPort); } catch { /* ... */ }
} else if (!(await isPortAvailable(requestedPort))) { /* loud diagnostic */ }

getAvailablePort walks upward (port++, up to +100). The --dev path is exactly the one publish-smoke.sh takes, so with run A holding 3210, run B's app binds 3211 while run B's BASE_URL still names 3210. Run B then smoke-tests run A's freshly scaffolded app and reports on it.

The non-dev branch already does the right thing (refuse loudly), so the shape of the fix is settled by the file's own precedent, not by a new decision.

Severity — deliberately not graded here

I did not measure this end to end: it needs two real objectstack dev boots of a scaffolded app, which is a much heavier setup than the vite measurement #9578 needed, and the sweep that surfaced it was explicitly scoped to reporting, not fixing. What is verified is the auto-shift (read from serve.ts and gated on the flag this script passes) and the three literals above. What is unverified is what a wrong-app smoke run actually asserts — whether it fails confusingly, or passes against the neighbour's app.

That second question is the whole severity argument, exactly as it was on #9578: a smoke test that passes against another run's app is a silent wrong answer, and a smoke test that fails confusingly is merely noisy. Measure it before grading.

Suggested shape (not a decision)

The same two halves that #9578 landed, if they apply here:

  1. pick a free port per run rather than defaulting to one literal;
  2. make the probe prove the app answering is the one this run started — the --strictPort equivalent for objectstack dev is the existing non-dev branch's refuse-on-busy behaviour, which could be opted into by the smoke script.

Note from #9578, since it is the part that surprised: the "fail loudly on a busy port" half is not sufficient on its own. If the neighbour keeps answering on the shared port, a wait loop that only asks "does the port answer?" still gets its 200 and still targets the neighbour. Both halves, or neither works.

Sweep context

scripts/ was swept for fixed ports and unqualified /tmp paths used as per-run state. The complete result:

findingfilestatus
DUMP_PORT=5180, /tmp/sdui-dump-dev.logscripts/gen-sdui-manifest.shfixed in #9578
SMOKE_PORT="${SMOKE_PORT:-3210}"scripts/publish-smoke.shthis card

No other fixed port and no other unqualified /tmp path used as per-run state exists under scripts/. scripts/downstream-smoke.sh and scripts/release-spec-changes.sh already mktemp their per-run state; the two other /tmp hits (scripts/check-type-check-coverage.mjs, scripts/collect-release-notes.sh) are documentation examples, not state.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions