Skip to content

fix(cli): make os validate --json --strict exit 1 where --strict already does - #11299

Merged
os-zhuang merged 2 commits into
mainfrom
claude/issue-11174-validate-json-strict-exit
Aug 23, 2026
Merged

fix(cli): make os validate --json --strict exit 1 where --strict already does#11299
os-zhuang merged 2 commits into
mainfrom
claude/issue-11174-validate-json-strict-exit

Conversation

@claude

@claudeclaudeBot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Fixes#11174

commands/validate.ts emitted the --json payload and returned above the only
flags.strict reader, which sat inside the text-rendering block. One config, one flag,
two answers — reproduced on this branch's base before touching anything:

$ os validate --strict ; echo "exit=$?"
⚠ No objects defined — this stack has no data model
⚠ No apps or plugins defined — this stack may not do much
⚠ Missing manifest.id — required for deployment
⚠ Missing manifest.namespace — required for multi-app hosting
✗ Strict mode: warnings treated as errors
exit=1
$ os validate --json --strict ; echo "exit=$?"
{ "valid": true, ..., "warnings": [ ...the same four... ] }
exit=0

--strict was accepted, documented and inert whenever --json was passed.
content/docs/deployment/cli.mdx spells os validate --json --strict twice in its
CI/CD section, once as a GitHub Actions step — that combination is the one audience the
flag exists for, and a pipeline gating on its exit status read 0 and called the stack
clean.

This is the second half of a pair. #10953 (PR #11175) made the four structural advisories
reachable in the payload, so a pipeline could at least gate on warnings.length; it
hoisted the advisories and did not touch the exit code, so a pipeline trusting the exit
status still could not.

Which set --strict gates on — measured, not chosen

The card warns against reflexively moving the strict check up, because #11175 changed what
is in scope at that point in the file. The two faces turn out to gate on sets that differ
by one collection:

collectiontext warningsJSON payload warnings
capProviderWarnings, unknownKeyWarnings, ruleAdvisories, docWarnings, structuralWarningsyesyes
conversionNotices (ADR-0087 D2)yesno — carried under its own conversions field
specGapno (explicitly never gated by --strict)no — own specVersionGap field

So gating the JSON face on the payload's warnings field would have left the identical
divergence in place, one collection narrower: a config whose only advisories are
conversion notices would still exit 0 under --json --strict and 1 without --json.
The gate therefore reads the text face's own list, and that list is now assembled once
above the if (flags.json) branch and consumed by both faces — the same "a single list
cannot drift from itself" move this file already made for unknownKeyWarnings and for
structuralWarnings. The push order is unchanged, so the text face's output is
byte-for-byte what it was.

This is a difference in payload shape, not a dropped advisory, and it is already
declared as such by test/validate-json-warning-parity.e2e.test.ts, which excludes
conversions and specVersionGap from its set equality and asserts both are empty for
its fixtures rather than ignoring them.

Two smaller points, both deliberate:

  • valid: true beside exit 1 is the text face verbatim. That path prints "Validation
    passed" and then fails for strict. The stack is schema-valid; --strict is what
    promotes its advisories to a failure. Pinned in the test, because the pairing is new
    here and reads like a bug otherwise.
  • The status rides in emitJson's CliExitCode slot, not a following this.exit(1).
    The failure paths above it use this.exit(1) because they must stop a fall-through into
    the text rendering; here the payload is complete and the return is right there. The
    slot is the declared channel for pairing a --json document with the status the shell
    reads (utils/format.ts, pinned by utils/format.exit-code.test.ts and
    test/migrate-exit-code.e2e.test.ts), and it emits the one document with no ExitError
    unwinding through the catch.

Reverse verification — both directions, quoted

The new test was run on the pre-fix tree (validate.ts unmodified, confirmed by an
empty git diff on it) before the fix was written, against a prediction recorded first:
test 1 red, tests 2-4 green.

Pre-fix — red, and it reproduces the card's measurement on this head:

AssertionError: --json --strict exited 0 where --strict exited 1, same config.
: expected +0 to be 1 // Object.is equality
Test Files 1 failed (1)
Tests 1 failed | 3 passed (4)

Post-fix — green:

 Test Files 1 passed (1)
Tests 4 passed (4)

The assertions are a parity matrix, not expect(code).toBe(1): both sides are read from
the real exit status of two real CLI runs and compared to each other, with an anti-vacuity
floor (the warning fixture's text run must exit non-zero) so equality is never asserted
over a run with nothing to fail on, a clean fixture pinning the zero end so neither face
can move to meet the other, and a run without --strict separating "gates on --strict"
from "fails whenever --json sees a warning".

Test placement

packages/cli/src/commands/validate-json-strict-exit.e2e.test.ts, not beside its siblings
in packages/cli/test/, because that directory was held by another in-flight card and was
read-only to this change. src/ is the stronger home anyway, for the reason
utils/format.exit-code.test.ts already gives: packages/cli/tsconfig.json includes
src, so pnpm typecheck compiles the file, while no tsc program reads
packages/cli/test/. tsconfig.build.json excludes src/**/*.test.ts, so nothing ships.

Verification

All gates below were run at head 79844ec3. A later commit (bb07f74f) changes the
changeset prose only — no source, test or bump change — so only the three changeset gates
were re-run at that head, after committing: check-adr-0087-registration
(1 declared-breaking changeset(s), each carrying an ADR-0087 disposition),
check-changeset-no-major (introduces no \major` bump) and check-empty-changeset (1 declaring changeset(s) added), plus check:changeset-gate-self-tests, check:objectui-changesetandcheck:nul-bytes`. All green.

  • pnpm lint (full repo, eslint . --no-inline-config) — green in 80s. No narrowing, so
    nothing to declare.
  • pnpm --filter @objectstack/cli typecheck> tsc --noEmit, clean.
  • Ten validate-reaching suites in @objectstack/cliTest Files 10 passed (10),
    Tests 74 passed (74), including the three existing validate-* suites,
    emit-json-pipe, commands, authoring-rule-command-parity and
    utils/format.exit-code.test.ts.
  • Gate families re-derived at final head with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack
    (no paths — it reads its own change set). All green: check:changeset-gate-self-tests,
    check:cross-package-test-inputs, check:objectui-changeset, check:published-files,
    check:slot-lookup, check:test-source-alias, check:type-source-resolution,
    check-adr-0087-registration, check-changeset-no-major, check-ci-filter-parity,
    check-empty-changeset, check-plugin-teardown-shape, docs-audit/check-affected-docs,
    plus the convention-triggered set the new test file moves —
    check:query-options-erasure, check:type-check-coverage, check:type-check-debt --re-measure,
    check:engine-double-contract, check:where-matcher — and check:nul-bytes.
  • check:type-check-debt --re-measure first refused outright (@objectstack/service-knowledge
    had no built type entry point), which means not measured rather than not applicable. That
    one package was built and the gate re-run to a real verdict:
    33 ledger entr(ies) re-measured in 553.6s, 1897 raw tsc error(s) total, none above its recorded number.

origin/main moved 5 commits during the work; none touch packages/cli or
packages/spec, so the build state and the numbers above stand.

Changeset

minor, argued in the body rather than defaulted. A pipeline running
os validate --json --strict over a stack that raises non-blocking advisories was green
and will now be red, so this must not ship in a release a CI system can take unattended —
but nothing is removed or renamed, the accept set is identical, and the new behaviour
restores what --strict declares and the docs already advertise rather than contradicting
a contract, so it is not a major either. Under this repo's launch-window convention
(breaking changes ship as minor while the stack versions in lockstep), minor is the
honest slot. Marked **BREAKING** with an ADR-0087 not-required (no-migration-prescription)
disposition; the gate accepted it and printed the reason back.

Follow-up filed

The conversions-only exit-code cell — a config whose only advisories are conversion
notices now exits 1 with warnings: [] and a populated conversions — is stated in the
changeset but not pinned by a test: no fixture in this repo raises a conversion. Adding
one needs a new fixture plus a full dependency-closure rebuild, so it is filed unassigned
as #11301 rather than stretched into this PR.


Generated by Claude Code


Generated by Claude Code

…already does (#11174)
The `--json` branch emitted its payload and returned above the only
`flags.strict` reader, which sat inside the text-rendering block. On one
config with one flag the two faces answered differently: text exited 1
with "Strict mode: warnings treated as errors", `--json --strict` exited 0.
The flag was accepted, documented for CI in
`content/docs/deployment/cli.mdx`, and inert.
Assemble the warning list once, above the `if (flags.json)` branch, and
have both faces gate on that same list, so the two exit codes cannot drift
apart again. The JSON status rides in `emitJson`'s `CliExitCode` slot — the
declared channel for pairing a `--json` document with the status the shell
reads — so the payload stays exactly one parseable document.
The gate reads the text face's list rather than the payload's `warnings`
field: the two differ by the ADR-0087 conversion notices, which the payload
carries under `conversions`, and gating on the field would have left the
same divergence for a conversion-only config.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019siH5jDmk5hrayvfyojUqR
@github-actions

github-actionsBot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 1 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • 1 anchor(s) matched too much of the corpus to be a work list: os validate (command, 42 pages)
  • 1 name(s) were too generic to anchor anything (single lowercase words)

Coarse fallback — 23 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 15d58dbf12ca850f10aab2763100aaa7d762db99packageMentionDocs.

@github-actionsgithub-actionsBot added documentation Improvements or additions to documentation tests tooling labels Aug 23, 2026
…` needs `conversions` too (#11174)
The changeset told a pipeline to "drop `--strict` and gate on the payload's
`warnings` array, which has carried the full advisory set since #10953".
That is false in exactly the dimension this PR makes load-bearing: the
payload's `warnings` field is the five-way spread WITHOUT the ADR-0087
conversion notices, which ride under `conversions`. A pipeline following
that advice would have got a strictly weaker gate than `--strict` — a
conversions-only config passes it and fails `--strict` — which is the same
silent under-reporting this change exists to remove, published as guidance.
Say `warnings.length > 0 || conversions.length > 0`, say plainly that
`warnings` alone is narrower and why, and state the resulting payload shape
outright: a conversions-only config exits 1 with `"warnings": []` and a
populated `conversions`, so predicting the exit code from `warnings.length`
is wrong for that config.
Changeset prose only — no source, test or bump change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019siH5jDmk5hrayvfyojUqR
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationsize/mteststooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] os validate --json --strict exits 0 on a config the text-mode --strict exits 1 for — the JSON path returns above the strict check

2 participants

@os-zhuang@claude