Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .changeset/cli-validate-author-gate.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
---
"@objectstack/cli": minor
"create-objectstack": minor
---

Make `os validate` the author-time verification gate and steer scaffolds toward it.

- **`os validate`** now runs the same CEL/predicate gate as `os build`/`os compile`
(ADR-0032): every `visible`/`disabled`/`requiredWhen`/validation/flow/sharing
predicate is checked for CEL syntax and `record.<field>` existence on the target
object. It already ran the protocol schema and widget-binding checks; the
expression gate closes the gap so a bare field ref (`done` instead of
`record.done`) — which silently hides an action on every record at runtime
(#2183/#2185) — fails validation instead of shipping. `os validate` is now a
read-only superset of the build's checks (no artifact emitted).
- **`create-objectstack`** now emits an `AGENTS.md` (and `.github/copilot-instructions.md`)
into every generated project instructing coding agents to run `npm run validate`
after editing metadata, aligns the blank template's `dev`/`start` scripts with the
example apps (`objectstack dev`/`objectstack start`), and sharpens the post-create
"Next steps" output.
31 changes: 27 additions & 4 deletions content/docs/getting-started/cli.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -47,7 +47,7 @@ Open [http://localhost:3000/_console/](http://localhost:3000/_console/) — you'
### Validate & Build

```bash
os validate # Check schema correctness
os validate # Check schema + CEL predicates + widget bindings (no artifact)
os compile # Build production artifact → dist/objectstack.json
```

Expand DownExpand Up@@ -303,7 +303,7 @@ shape they accept. See [Source vs Artifact](#source-vs-artifact) below.
| Command | Description |
|---------|-------------|
| `os compile [config]` | Compile configuration to a JSON artifact (`dist/objectstack.json`) |
| `os validate [config]` | Validate configuration against the ObjectStack Protocol schema |
| `os validate [config]` | Validate schema, CEL predicates, and widget bindings — the same gates as `os compile`/`os build`, no artifact emitted |
| `os info [config]` | Display metadata summary (objects, fields, apps, agents, etc.) |

#### `os compile`
Expand DownExpand Up@@ -346,7 +346,9 @@ over HTTP from another runtime. See [`os start`](#os-start) and

#### `os validate`

Standalone schema validation with rich error output. Use to check your configuration without compiling.
The fast, artifact-free verification gate. It runs the **same structural and
semantic checks as `os compile`/`os build`** but writes no `dist/`, so it is the
command to run after every metadata edit. Use it before reporting a change done.

```bash
os validate # Validate current directory
Expand All@@ -355,16 +357,37 @@ os validate --json # JSON output for CI
os validate path/to/config # Validate specific file
```

**Gates run (each exits non-zero with a located, corrective message):**
1. **Protocol schema** — the stack conforms to `ObjectStackDefinitionSchema`
(`@objectstack/spec`).
2. **CEL / predicate validation (ADR-0032)** — every `visible` / `disabled` /
`requiredWhen` / validation rule / flow condition / sharing rule is parsed
for CEL syntax **and** checked that each `record.<field>` reference exists on
the target object. This catches a **bare field ref** (`done` instead of
`record.done`) that would otherwise evaluate to `null` and silently hide an
action on every record (#2183/#2185).
3. **Widget-binding integrity (ADR-0021)** — every dashboard widget's
`dataset` / `dimensions` / `values` resolves to a declared dataset/field, so
a dangling binding fails here instead of rendering an empty chart.

**Options:**
- `--strict` — Treat warnings as errors (exit code 1)
- `--json` — Output results as JSON

**Warnings checked:**
**Warnings checked (advisory, non-blocking unless `--strict`):**
- Missing `manifest.id` (required for deployment)
- Missing `manifest.namespace` (required for multi-app hosting)
- No objects defined
- No apps or plugins defined

<Callout type="tip">
`os validate` and `os build` share one validator, so a config that passes
`os validate` will not fail the build on schema/predicate/binding grounds. In a
scaffolded project these are wired as `npm run validate` and `npm run build`;
your `AGENTS.md` tells coding agents to run `npm run validate` after editing
metadata. See [Validating metadata](/docs/guides/validating-metadata).
</Callout>

#### `os info`

Displays a summary of your metadata without compilation or validation:
Expand Down
1 change: 1 addition & 0 deletions content/docs/guides/meta.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,6 +12,7 @@
"seed-data",
"common-patterns",
"formula",
"validating-metadata",
"analytics-datasets",
"airtable-dashboard-analysis",
"---Building---",
Expand Down
104 changes: 104 additions & 0 deletions content/docs/guides/validating-metadata.mdx
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
---
title: Validating Metadata
description: Why ObjectStack metadata mistakes fail silently at runtime, and the one command that catches them at author time — run it after every metadata edit.
---

# Validating Metadata

ObjectStack metadata is data, not code paths — so most mistakes are **not** caught
by the TypeScript compiler. They pass `tsc`, load fine, and then fail **silently
at runtime**. The fix is one command you run after every metadata edit:

```bash
os validate # schema + CEL predicates + widget bindings — no artifact
```

In a scaffolded project this is wired as `npm run validate`. Your generated
`AGENTS.md` instructs coding agents (Claude Code, Cursor, Copilot) to run it
after editing metadata.

## Why typecheck isn't enough

Two classes of bug type-check cleanly but break at runtime:

### 1. Bare-field predicates

Predicates — an action's `visible`/`disabled`, a field's `requiredWhen`, a
validation rule, a flow condition, a sharing rule — are **CEL expressions**, and
they reference record fields through the `record.` scope:

```ts
// ✗ Wrong — `done` is a bare reference. It type-checks (it's just a string),
// but at runtime it resolves to null → the action is hidden on EVERY record.
{ name: 'mark_done', visible: '!done' }

// ✓ Right
{ name: 'mark_done', visible: '!record.done' }
```

This is the trap behind the recurring "the button never shows / the rule never
fires" bugs (#2183/#2185). `os validate` parses every predicate and checks that
each `record.<field>` exists on the target object, so the bare ref fails the
gate with a located, did-you-mean message instead of shipping.

### 2. Dangling widget bindings

A dashboard widget points at a `dataset` and reads `dimensions`/`values` from it.
If a name doesn't resolve, the chart renders **empty** — no error (ADR-0021).
`os validate` resolves every binding against the declared datasets and fails on a
dangling one.

## The one gate, two entry points

`os validate` and `os build` (alias `os compile`) run the **same** validator:

| | `os validate` | `os build` |
|---|---|---|
| Protocol schema (Zod) | ✓ | ✓ |
| CEL / predicate validation | ✓ | ✓ |
| Widget-binding integrity | ✓ | ✓ |
| Emits `dist/objectstack.json` | — | ✓ |

So `os validate` is the fast inner-loop check (no artifact); `os build` is what
you run when you need the deployable artifact. A config that passes `os validate`
will not fail `os build` on schema/predicate/binding grounds.

<Callout type="info">
`os lint` is a **separate** pass — style and convention checks (snake_case
naming, required labels, namespace prefixes, data-model patterns). Run it too,
but it does not replace `os validate`, and `os validate` does not replace it.
</Callout>

## The workflow

```bash
# after editing any *.object.ts / *.view.ts / *.action.ts / *.flow.ts / *.dashboard.ts
npm run validate # os validate — schema + predicates + bindings
npm run typecheck # tsc --noEmit — types against @objectstack/spec
```

**Rule of thumb: never report a metadata change as done until `npm run validate`
passes.** In the example apps the equivalent is `pnpm --filter <pkg> validate`
(`pnpm verify` in `app-showcase`, which chains validate + typecheck + test).

### Checking a single expression

To validate one CEL expression *before* you write it into a file — for example
inside an AI build loop — call the `validate_expression` agent tool, which runs
the same predicate validator inline. See the
[objectstack-formula skill](/docs/guides/formula).

## In CI

Both commands support `--json` and exit non-zero on failure:

```yaml
- name: Validate ObjectStack metadata
run: npx objectstack validate --strict --json
```

## See also

- [Command Line Interface](/docs/getting-started/cli) — full `os validate` / `os build` reference
- [Formula & Predicates](/docs/guides/formula) — the CEL surface and `record.` scope
- [Data Modeling](/docs/guides/data-modeling) — objects, fields, validations, sharing
33 changes: 33 additions & 0 deletions examples/AGENTS.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
# Examples — Agent Instructions

These are reference ObjectStack apps (`app-crm`, `app-todo`, `app-showcase`).
They double as the canonical shape of a user-scaffolded project, so the metadata
here should always be correct.

## Verify after every metadata change

ObjectStack metadata mistakes fail **silently at runtime**, not at edit time:

- a bare field ref in a predicate (`done` instead of `record.done`) evaluates to
`null` and silently hides the action/validation on every record (#2183/#2185);
- a dangling dashboard widget binding renders an empty chart (ADR-0021).

`objectstack validate` catches both at author time. It runs the same gates as
`objectstack build` — Zod protocol schema, CEL/predicate validation with
`record.<field>` existence checks, and widget-binding integrity — but emits no
`dist/`, so it is the fast inner-loop check. It exits non-zero with a located,
corrective message.

After editing any `*.object.ts` / `*.view.ts` / `*.action.ts` / `*.flow.ts` /
`*.dashboard.ts`, run the gate for that example:

```bash
pnpm --filter @objectstack/example-crm validate # + typecheck, test
pnpm --filter @objectstack/example-todo validate
pnpm --filter @objectstack/example-showcase verify # validate + typecheck + test
```

**Do not report a metadata change as done until `validate` passes.** When in
doubt about how to express a predicate or binding, consult the relevant
`objectstack-*` skill (e.g. `objectstack-formula` for CEL, `objectstack-ui` for
views/actions/dashboards).
1 change: 1 addition & 0 deletions examples/app-crm/package.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,7 @@
"dev": "objectstack dev",
"start": "objectstack start",
"build": "objectstack build",
"validate": "objectstack validate",
"typecheck": "tsc --noEmit",
"test": "vitest run"
},
Expand Down
3 changes: 2 additions & 1 deletion examples/app-showcase/package.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -15,9 +15,10 @@
"dev": "objectstack dev --seed-admin",
"start": "objectstack start",
"build": "objectstack build",
"validate": "objectstack validate",
"typecheck": "tsc --noEmit",
"test": "vitest run",
"verify": "pnpm typecheck && pnpm test",
"verify": "pnpm validate && pnpm typecheck && pnpm test",
"test:smoke": "playwright test --config=playwright.config.ts"
},
"dependencies": {
Expand Down
1 change: 1 addition & 0 deletions examples/app-todo/package.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,7 @@
"dev": "objectstack dev",
"start": "objectstack start",
"build": "objectstack build",
"validate": "objectstack validate",
"typecheck": "tsc --noEmit",
"test": "objectstack test",
"test:ai": "tsx test/ai.test.ts",
Expand Down
40 changes: 38 additions & 2 deletions packages/cli/src/commands/validate.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -5,6 +5,7 @@ import chalk from 'chalk';
import { ZodError } from 'zod';
import { ObjectStackDefinitionSchema, normalizeStackInput } from '@objectstack/spec';
import { loadConfig } from '../utils/config.js';
import { validateStackExpressions } from '../utils/validate-expressions.js';
import { validateWidgetBindings } from '../utils/validate-widget-bindings.js';
import {
printHeader,
Expand All@@ -19,7 +20,8 @@ import {
} from '../utils/format.js';

export default class Validate extends Command {
static override description = 'Validate ObjectStack configuration against the protocol schema';
static override description =
'Validate ObjectStack configuration against the protocol schema, CEL expressions, and widget bindings (no artifact emitted)';

static override args = {
config: Args.string({ description: 'Configuration file path', required: false }),
Expand DownExpand Up@@ -70,6 +72,37 @@ export default class Validate extends Command {
this.exit(1);
}

// 2b. Expression validation (ADR-0032 §1a/1b) — the same gate `os build`
// runs, brought to the read-only check so authors catch it without
// emitting an artifact. CEL predicates in actions/validations/flows/
// sharing/hooks are checked for syntax AND that `record.<field>`
// references resolve on the target object. This is what catches a
// BARE field ref (`done` instead of `record.done`) that would
// otherwise silently hide an action on every record (#2183/#2185).
if (!flags.json) printStep('Validating expressions (ADR-0032)...');
const exprIssues = validateStackExpressions(result.data as Record<string, unknown>);
const exprErrors = exprIssues.filter((i) => i.severity !== 'warning');
const exprWarnings = exprIssues.filter((i) => i.severity === 'warning');

if (exprErrors.length > 0) {
if (flags.json) {
console.log(JSON.stringify({
valid: false,
errors: exprErrors,
warnings: exprWarnings,
duration: timer.elapsed(),
}, null, 2));
this.exit(1);
}
console.log('');
printError(`Expression validation failed (${exprErrors.length} issue${exprErrors.length > 1 ? 's' : ''})`);
for (const i of exprErrors.slice(0, 50)) {
console.log(` • ${i.where}: ${i.message}`);
console.log(chalk.dim(` source: \`${i.source}\``));
}
this.exit(1);
}

// 3. Dashboard widget reference integrity (issue #1721) — a semantic
// cross-reference pass the protocol schema cannot express: every
// widget's `dataset`/`dimensions`/`values` and chartConfig
Expand DownExpand Up@@ -108,7 +141,7 @@ export default class Validate extends Command {
valid: true,
manifest: config.manifest,
stats,
warnings: widgetWarnings,
warnings: [...exprWarnings, ...widgetWarnings],
duration: timer.elapsed(),
}, null, 2));
return;
Expand All@@ -117,6 +150,9 @@ export default class Validate extends Command {
// 5. Warnings (non-blocking)
const warnings: string[] = [];

for (const i of exprWarnings) {
warnings.push(`${i.where}: ${i.message}`);
}
for (const f of widgetWarnings) {
warnings.push(`${f.where}: ${f.message}`);
}
Expand Down
46 changes: 45 additions & 1 deletion packages/create-objectstack/src/index.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -335,6 +335,49 @@ function rewriteProjectIdentity(
md = md.replace(/^#\s+.*$/m, `# ${title}`);
fs.writeFileSync(readmePath, md);
}

writeAgentGuides(targetDir, title, projectName);
}

// Emit the cross-agent guidance file (AGENTS.md) and the GitHub Copilot variant
// (.github/copilot-instructions.md) from the shared template. This is what tells
// the coding agent to run `npm run validate` after editing metadata — the gate
// that catches bare-field predicates and dangling bindings that otherwise fail
// silently at runtime. Skip either file if the template already shipped its own,
// so a curated template can override the default.
function writeAgentGuides(targetDir: string, title: string, projectName: string) {
const templatePath = path.join(BUNDLED_TEMPLATES_DIR, 'AGENTS.md');
let template: string;
try {
template = fs.readFileSync(templatePath, 'utf8');
} catch (err: any) {
if (err?.code === 'ENOENT') return; // bundled template absent — nothing to emit
throw err;
}

const rendered = template
.replace(/\{\{PROJECT_TITLE\}\}/g, title)
.replace(/\{\{PROJECT_NAME\}\}/g, projectName);

// Atomic exclusive-create (the `wx` flag) instead of existsSync()+writeFileSync():
// it fails with EEXIST if the file already exists, so a curated template that
// ships its own guide is preserved — without the check-then-write TOCTOU race a
// separate existence check introduces.
writeIfAbsent(path.join(targetDir, 'AGENTS.md'), rendered);

const copilotPath = path.join(targetDir, '.github', 'copilot-instructions.md');
fs.mkdirSync(path.dirname(copilotPath), { recursive: true });
writeIfAbsent(copilotPath, rendered);
}

// Create a file only if it does not already exist, atomically — no time-of-check
// to time-of-use gap between an existence test and the write.
function writeIfAbsent(filePath: string, contents: string) {
try {
fs.writeFileSync(filePath, contents, { flag: 'wx' });
} catch (err: any) {
if (err?.code !== 'EEXIST') throw err;
}
}

// ─── CLI Program ────────────────────────────────────────────────────
Expand DownExpand Up@@ -450,7 +493,8 @@ const program = new Command()
console.log(chalk.dim(' npm install'));
}
console.log(chalk.dim(' npm run dev # Start development server'));
console.log(chalk.dim(' npm run validate # Check configuration'));
console.log(chalk.dim(' npm run validate # Verify metadata: schema + predicates + bindings'));
console.log(chalk.dim(' # (run after every metadata edit — see AGENTS.md)'));
if (options.skipInstall || options.skipSkills) {
console.log('');
console.log(chalk.bold(' AI Skills (recommended):'));
Expand Down
Loading
Loading