From 90cb47458fc490c6d732a431252b20bddee037b0 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 01:58:47 +0000 Subject: [PATCH 1/4] skills(data): consolidate hooks into references/data-hooks.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DATA-B-01 / I-03 — rules/hooks.md called itself "a reference pointer" and then restated five sections of the canonical file. Delete it; references/data-hooks.md is the single hooks file and the catalog's hooks anchor. Its ceiling row is dropped from scripts/check-skills-token-ratchet.mjs (the only edit to that file). DATA-B-02 second H1 + "When to Use This Skill" inside a reference. DATA-B-04 three registration methods, compact -- keeping the four load-bearing facts (AppPlugin auto-binds; the `functions` map; `packageId`; and that programmatic binding gets NO declarative condition/retry/ timeout/onError/async semantics). DATA-B-05 10+10 generic DO/DON'T -> the compact 5+5 rules/hooks.md carried. DATA-C-03 vitest + LiteKernel harnesses -> one-line pointer to objectstack-platform, which claims that surface in its frontmatter. DATA-D-01/02/03 invented benchmark table, plain-JS "Advanced Topics", generic Troubleshooting and the Summary that repeated the DO list verbatim. The References block survives, and absorbs the two cross-package pointers rules/hooks.md carried so no routing is lost. DATA-D-04 patterns 1,2,4,5,7,9 (generic JS) deleted; 3,6,8,10 kept, renumbered. DATA-E-04 the file prescribed `body` and demonstrated `handler` ten times: two surviving patterns are now sandboxed `body` with declared capabilities. INC-03 (found while rewriting pattern 8, evidence pinned): "There is NO `updateMany`" is true of the HANDLER-side repo -- ObjectRepository, what ctx.api.object() returns, packages/objectql/src/engine.ts:13575-13600 -- but the SANDBOX repo does install updateMany/deleteMany (packages/runtime/src/sandbox/quickjs-runner.ts:544-545), exactly as this file's own capability table says. The comment now names the surface it is true of. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1 --- scripts/check-skills-token-ratchet.mjs | 1 - skills/objectstack-data/SKILL.md | 11 +- .../objectstack-data/references/data-hooks.md | 608 +++--------------- skills/objectstack-data/rules/hooks.md | 209 ------ 4 files changed, 77 insertions(+), 752 deletions(-) delete mode 100644 skills/objectstack-data/rules/hooks.md diff --git a/scripts/check-skills-token-ratchet.mjs b/scripts/check-skills-token-ratchet.mjs index 41e166e5b9..db0e70145b 100644 --- a/scripts/check-skills-token-ratchet.mjs +++ b/scripts/check-skills-token-ratchet.mjs @@ -404,7 +404,6 @@ export const CEILINGS = new Map([ ['skills/objectstack-data/references/data-hooks.md', 12611], ['skills/objectstack-data/rules/datasources.md', 911], ['skills/objectstack-data/rules/field-types.md', 3584], - ['skills/objectstack-data/rules/hooks.md', 2195], ['skills/objectstack-data/rules/indexing.md', 3241], ['skills/objectstack-data/rules/lifecycle.md', 1590], ['skills/objectstack-data/rules/naming.md', 773], diff --git a/skills/objectstack-data/SKILL.md b/skills/objectstack-data/SKILL.md index 46e2753888..528106be8f 100644 --- a/skills/objectstack-data/SKILL.md +++ b/skills/objectstack-data/SKILL.md @@ -157,7 +157,7 @@ fresh as whatever writes it. Cover both write paths: | A project is renamed | `afterUpdate` hook on `project` — re-stamp `project_name` on that project's tasks | Rows written by a path that bypasses hooks (bulk import, direct SQL) need a -one-off backfill. See [Lifecycle Hooks](./rules/hooks.md). +one-off backfill. See [Lifecycle Hooks](./references/data-hooks.md). **The errors an author sees for the dotted path** (grep either back to here). `os validate` → `searchable-field-unknown`: @@ -315,7 +315,7 @@ For comprehensive documentation with incorrect/correct examples: - **[Validation Rules](./rules/validation.md)** — All validation types, script inversion, severity levels - **[Index Strategy](./rules/indexing.md)** — btree/gin/gist/fulltext, composite indexes, partial indexes - **[Data Lifecycle & Retention](./rules/lifecycle.md)** — `lifecycle` classes (record/audit/telemetry/transient/event), retention/TTL/rotation/archive policies; ❗ append-only objects must declare one (distinct from lifecycle *hooks* below) -- **[Lifecycle Hooks](./rules/hooks.md)** — Hook quick reference (→ see [references/data-hooks.md](./references/data-hooks.md) for the full 8-event guide + the sandboxed `body` ctx/capability contract) +- **[Lifecycle Hooks](./references/data-hooks.md)** — the 8 lifecycle events, `handler` vs sandboxed `body` (ctx + capability contract), registration, canonical patterns - **[Datasources & Federation](./rules/datasources.md)** — `defineDatasource`, external/federated objects (`remoteName`/`columnMap`), auto-connect gating, credentials; ❌ no `field.columnName` on external objects --- @@ -530,10 +530,9 @@ export default defineHook({ The `handler` above is the inline (in-process) form. The **preferred**, metadata-native form is a sandboxed `body` — `{ language: 'js', source, capabilities }` run in an isolated VM, the shape that AI/Studio-authored hooks and every build -artifact carry. See [rules/hooks.md](./rules/hooks.md) for the quick reference, or -[references/data-hooks.md](./references/data-hooks.md) for complete documentation -of all 8 lifecycle events, both registration forms, the **sandboxed `body` ctx + -capability contract**, and patterns. +artifact carry. See [references/data-hooks.md](./references/data-hooks.md) for all 8 +lifecycle events, both registration forms, the **sandboxed `body` ctx + capability +contract**, and the canonical patterns. --- diff --git a/skills/objectstack-data/references/data-hooks.md b/skills/objectstack-data/references/data-hooks.md index ec2dcd30fd..5974b02269 100644 --- a/skills/objectstack-data/references/data-hooks.md +++ b/skills/objectstack-data/references/data-hooks.md @@ -1,29 +1,11 @@ # Data Lifecycle Hooks — Reference -Reference companion to `objectstack-data/SKILL.md`. Comprehensive guide to -the 8 data lifecycle events, registration modes (inline `handler` **and** -sandboxed `body`), the `HookContext` API, and common patterns (validation, -defaults, audit logging, workflows). +Reference companion to `objectstack-data/SKILL.md`, and the catalog's anchor for +data hooks. Covers the 8 lifecycle events, the inline `handler` and sandboxed +`body` forms, the `HookContext` API, registration, and the canonical patterns +(delete guards, audit trails, cross-object writes, read masking). -# Writing Hooks — ObjectStack Data Lifecycle - -Expert instructions for third-party developers to write data lifecycle hooks in ObjectStack. -Hooks are the primary extension point for adding custom business logic, validation rules, -side effects, and data transformations to CRUD operations. - ---- - -## When to Use This Skill - -- You need to **add custom validation** beyond declarative rules. -- You want to **enrich data** (set defaults, calculate fields, normalize values). -- You need to trigger **side effects** (send emails, update external systems, publish events). -- You want to **enforce business rules** that span multiple fields or objects. -- You need to **transform data** before or after database operations. -- You want to **integrate with external APIs** during data operations. -- You need to **implement audit trails** or compliance requirements. - --- ## Core Concepts @@ -764,205 +746,56 @@ handler: async (ctx: HookContext) => { ## Common Patterns -### 1. Setting Default Values - -```typescript -const setAccountDefaults = defineHook({ - name: 'account_defaults', - object: 'account', - events: ['beforeInsert'], - handler: async (ctx) => { - // Set default industry - if (!ctx.input.industry) { - ctx.input.industry = 'Other'; - } - - // Set created timestamp - ctx.input.created_at = new Date().toISOString(); - - // Set owner to current user - if (!ctx.input.owner_id && ctx.session?.userId) { - ctx.input.owner_id = ctx.session.userId; - } - }, -}); -``` - -### 2. Data Validation - -```typescript -const validateAccount = defineHook({ - name: 'account_validation', - object: 'account', - events: ['beforeInsert', 'beforeUpdate'], - handler: async (ctx) => { - // Validate email format - if (ctx.input.email && !ctx.input.email.includes('@')) { - throw new Error('Invalid email format'); - } - - // Validate website URL - if (ctx.input.website && !ctx.input.website.startsWith('http')) { - throw new Error('Website must start with http or https'); - } - - // Check annual revenue - if (ctx.input.annual_revenue && ctx.input.annual_revenue < 0) { - throw new Error('Annual revenue cannot be negative'); - } - }, -}); -``` - -### 3. Preventing Deletion +### 1. Preventing Deletion (sandboxed `body`) ```typescript const protectStrategicAccounts = defineHook({ name: 'protect_strategic_accounts', object: 'account', events: ['beforeDelete'], - handler: async (ctx) => { - // ctx.previous contains the record being deleted - if (ctx.previous?.type === 'Strategic') { - throw new Error('Cannot delete Strategic accounts'); - } - - // Check for active opportunities - const oppCount = await ctx.api?.object('opportunity').count({ - filter: { - account_id: ctx.input.id, - stage: { $in: ['Prospecting', 'Negotiation'] } - } - }); - - if (oppCount && oppCount > 0) { - throw new Error(`Cannot delete account with ${oppCount} active opportunities`); - } - }, -}); -``` - -### 4. Data Enrichment - -```typescript -const enrichLeadScore = defineHook({ - name: 'lead_scoring', - object: 'lead', - events: ['beforeInsert', 'beforeUpdate'], - handler: async (ctx) => { - let score = 0; - - // Email domain scoring - if (ctx.input.email?.endsWith('@enterprise.com')) { - score += 50; - } - - // Phone number bonus - if (ctx.input.phone) { - score += 20; - } - - // Company size scoring - if (ctx.input.company_size === 'Enterprise') { - score += 30; - } - - // Industry scoring - if (ctx.input.industry === 'Technology') { - score += 25; - } - - ctx.input.score = score; - }, -}); -``` - -### 5. Triggering Workflows - -```typescript -const notifyOnStatusChange = defineHook({ - name: 'notify_status_change', - object: 'opportunity', - events: ['afterUpdate'], - async: true, // Fire-and-forget - handler: async (ctx) => { - // Detect status change - const oldStatus = ctx.previous?.stage; - const newStatus = ctx.input.stage; - - if (oldStatus !== newStatus) { - // Send notification (async, doesn't block transaction) - console.log(`Opportunity ${ctx.input.id} moved from ${oldStatus} to ${newStatus}`); - - // Could trigger email, Slack notification, etc. - // await sendEmail({ - // to: ctx.user?.email, - // subject: `Opportunity stage changed to ${newStatus}`, - // body: `...` - // }); - } + body: { + language: 'js', + source: ` + // ctx.previous carries the record being deleted (it is undefined on insert). + if (ctx.previous && ctx.previous.type === 'Strategic') + throw new Error('Cannot delete Strategic accounts'); + const open = await ctx.api.object('opportunity').count({ + where: { account_id: ctx.input.id, stage: { $in: ['Prospecting', 'Negotiation'] } }, + }); + if (open > 0) + throw new Error('Cannot delete an account with ' + open + ' open opportunities'); + `, + capabilities: ['api.read'], // count() is a read; an undeclared token throws in the VM }, }); ``` -### 6. Creating Related Records +### 2. Creating Related Records (audit trail, sandboxed `body`) ```typescript const createAuditTrail = defineHook({ name: 'audit_trail', object: ['account', 'contact', 'opportunity'], events: ['afterInsert', 'afterUpdate', 'afterDelete'], - async: false, // Must run in transaction - handler: async (ctx) => { - const action = ctx.event.replace('after', '').toLowerCase(); - - await ctx.api?.object('audit_log').insert({ - object_type: ctx.object, - record_id: String(ctx.input.id || ''), - action, - user_id: ctx.session?.userId, - timestamp: new Date().toISOString(), - changes: ctx.event === 'afterUpdate' ? { - before: ctx.previous, - after: ctx.result, - } : undefined, - }); - }, -}); -``` - -### 7. External API Integration - -```typescript -const syncToExternalCRM = defineHook({ - name: 'sync_external_crm', - object: 'account', - events: ['afterInsert', 'afterUpdate'], - async: true, // Don't block the main transaction - timeout: 10000, // 10 second timeout - retryPolicy: { - maxRetries: 3, - backoffMs: 2000, - }, - handler: async (ctx) => { - try { - // Call external API - // await fetch('https://external-crm.com/api/accounts', { - // method: 'POST', - // headers: { 'Authorization': 'Bearer ...' }, - // body: JSON.stringify(ctx.result), - // }); - - console.log(`Synced account ${ctx.input.id} to external CRM`); - } catch (error) { - // Error is logged but doesn't abort the operation - console.error('Failed to sync to external CRM', error); - } + async: false, // must run inside the transaction + body: { + language: 'js', + source: ` + await ctx.api.object('audit_log').insert({ + object_type: ctx.object, + record_id: String(ctx.input.id || ''), + action: ctx.event.replace('after', '').toLowerCase(), + user_id: ctx.session && ctx.session.userId, + timestamp: new Date().toISOString(), + changes: ctx.event === 'afterUpdate' ? { before: ctx.previous, after: ctx.result } : undefined, + }); + `, + capabilities: ['api.write'], }, }); ``` -### 8. Multi-Object Logic +### 3. Multi-Object Logic (inline `handler`) ```typescript const cascadeAccountUpdate = defineHook({ @@ -970,8 +803,10 @@ const cascadeAccountUpdate = defineHook({ object: 'account', events: ['afterUpdate'], handler: async (ctx) => { - // If account industry changed, update all contacts. - // There is NO `updateMany` — bulk updates use update(data, { where, multi: true }). + // If account industry changed, update all contacts. The handler-side repo + // (`ObjectRepository`) exposes NO `updateMany` — a bulk update is + // update(data, { where, multi: true }). Only the sandbox `ctx.api` adds + // `updateMany`/`deleteMany` (see the capability table above). if (ctx.input.industry && ctx.previous?.industry !== ctx.input.industry) { await ctx.api?.object('contact').update( { account_industry: ctx.input.industry }, @@ -982,24 +817,7 @@ const cascadeAccountUpdate = defineHook({ }); ``` -### 9. Conditional Execution - -```typescript -const highValueAccountAlert = defineHook({ - name: 'high_value_alert', - object: 'account', - events: ['afterInsert'], - // Only run for high-value accounts (CEL) - condition: P`record.annual_revenue > 10000000`, - async: true, - handler: async (ctx) => { - console.log(`🚨 High-value account created: ${ctx.result.name}`); - // Send alert to sales leadership - }, -}); -``` - -### 10. Data Masking (Read Operations) +### 4. Data Masking on Read > For **static** field masking (a field is always hidden/masked for a role), > prefer declarative **field-level metadata** (secret/masked fields) — it applies @@ -1049,115 +867,51 @@ const maskSensitiveData = defineHook({ ## Registration Methods -### Method 1: Declarative (Stack Definition) — RECOMMENDED - -**Best for:** Application-level hooks defined as metadata. The `AppPlugin` -auto-binds these onto the ObjectQL engine at startup — **no `register*Hook` -boilerplate is required**, and all declarative fields (`condition`, -`async`, `retryPolicy`, `timeout`, `onError`, `priority`) are honoured by -the runtime. +**1. Declarative — `defineStack({ hooks })`, the default.** `AppPlugin` auto-binds +these at startup: no `register*Hook` boilerplate, and the declarative fields +(`condition`, `async`, `retryPolicy`, `timeout`, `onError`, `priority`) are honoured +**only** on this path. A string-named `handler` resolves through the stack's +`functions` map. ```typescript // objectstack.config.ts -import { defineStack } from '@objectstack/spec'; -import taskHook from './objects/task.hook'; - export default defineStack({ - manifest: { /* ... */ }, - objects: [/* ... */], - hooks: [taskHook], // ← AppPlugin auto-binds; no manual registration needed + hooks: [taskHook, { name: 'h', object: 'account', events: ['beforeInsert'], handler: 'normalize' }], + functions: { normalize: async (ctx) => { /* ... */ } }, }); ``` -For string-named handlers, declare them under `functions` so the binder -can resolve them: +**2. Programmatic — `ctx.ql.registerHook()`, the plugin escape hatch.** Pass +`packageId` so the hook can be unregistered cleanly. ⚠️ Hooks bound this way get +**none** of the declarative `condition` / `retryPolicy` / `timeout` / `onError` / +`async` semantics — those apply only through `defineStack({ hooks })` or +`ql.bindHooks([...])`. ```typescript -export default defineStack({ - hooks: [ - { name: 'h', object: 'account', events: ['beforeInsert'], handler: 'normalize' }, - ], - functions: { - normalize: async (ctx) => { /* ... */ }, - }, -}); +// in your plugin's onEnable() +ctx.ql.registerHook('beforeInsert', async (hookCtx) => { /* ... */ }, + { object: 'account', priority: 100, packageId: 'my-plugin' }); ``` -### Method 2: Programmatic (Runtime) — escape hatch - -**Best for:** Plugins that need to register hooks dynamically based on -runtime state. Prefer Method 1 unless you actually need imperative -control. +**3. Hook files — `src/objects/{object}.hook.ts`.** One `defineHook({ ... })` per +file, default-exported, then listed in the stack's `hooks` array (method 1). This is +the layout both real hook modules in the repo use. -```typescript -// In your plugin's onEnable() -export const onEnable = async (ctx: { ql: ObjectQL }) => { - ctx.ql.registerHook('beforeInsert', async (hookCtx) => { - // Handler logic - }, { - object: 'account', - priority: 100, - packageId: 'my-plugin', // enables clean unregister later - }); -}; -``` - -> Note: hooks registered this way **do not** get the declarative -> `condition` / `retry` / `timeout` / `onError` / `async` semantics — -> those only apply when binding through `defineStack({ hooks })` or -> calling `ql.bindHooks([...])` directly. - -### Method 3: Hook Files (Convention) - -**Best for:** Organized codebases, per-object hooks. - -```typescript -// src/objects/account.hook.ts -import { defineHook, HookContext } from '@objectstack/spec/data'; - -const accountHook = defineHook({ - name: 'account_logic', - object: 'account', - events: ['beforeInsert', 'beforeUpdate'], - handler: async (ctx: HookContext) => { - // Validation logic - }, -}); - -export default accountHook; - -// Then import and register in objectstack.config.ts -``` --- ## Best Practices -### ✅ DO - -1. **Use specific events** — Don't subscribe to all events if you only need one. -2. **Keep handlers focused** — One hook = one responsibility. -3. **Use `condition` for filtering** — Avoid unnecessary handler execution. -4. **Set appropriate priorities** — Ensure correct execution order. -5. **Use `async: true` for side effects** — Don't block transactions for non-critical operations. -6. **Validate early** — Use `before*` hooks for validation. -7. **Handle errors gracefully** — Provide meaningful error messages. -8. **Use `ctx.api` for cross-object operations** — Maintains transaction consistency. -9. **Document your hooks** — Use `description` and comments. -10. **Test thoroughly** — Unit test hooks in isolation. - -### ❌ DON'T - -1. **Don't mutate immutable properties** — `ctx.object`, `ctx.event`, `ctx.id` are read-only. -2. **Don't perform expensive operations in `before*` hooks** — Use `after*` + `async: true` instead. -3. **Don't create infinite loops** — Be careful when hooks modify data that triggers other hooks. -4. **Don't ignore `ctx.previous`** — Essential for detecting changes. -5. **Don't use `object: '*'` unless necessary** — Performance impact. -6. **Don't block on external APIs** — Use `async: true` and proper timeouts. -7. **Don't assume `ctx.session` exists** — System operations may have no user context. -8. **Don't throw in `after*` hooks unless critical** — Use `onError: 'log'` for non-critical errors. -9. **Don't duplicate validation** — Use declarative validation rules when possible. -10. **Don't forget transaction boundaries** — `async: true` runs outside transaction. +✅ **DO** — use `before*` for validation and `after*` for side effects; set +`async: true` for non-critical background work; go through `ctx.api` for +cross-object operations; give a thrown error a message the caller can act on; +test hooks in isolation. + +❌ **DON'T** — do expensive work in `before*` (it blocks the transaction); let a +hook re-trigger itself; use `object: '*'` unless you mean every object; throw in +`after*` unless the failure must abort the operation; assume `ctx.session` exists +— system operations carry no user. + --- @@ -1208,237 +962,19 @@ handler: async (ctx) => { ## Testing Hooks -### Unit Testing - -```typescript -import { describe, it, expect } from 'vitest'; -import { HookContext } from '@objectstack/spec/data'; -import accountHook from './account.hook'; - -describe('accountHook', () => { - it('sets default industry', async () => { - const ctx: Partial = { - object: 'account', - event: 'beforeInsert', - input: { name: 'Acme Corp' }, - }; - - await accountHook.handler(ctx as HookContext); - - expect(ctx.input.industry).toBe('Other'); - }); - - it('validates website URL', async () => { - const ctx: Partial = { - object: 'account', - event: 'beforeInsert', - input: { website: 'invalid-url' }, - }; - - await expect( - accountHook.handler(ctx as HookContext) - ).rejects.toThrow('Website must start with http'); - }); -}); -``` - -### Integration Testing - -```typescript -import { LiteKernel } from '@objectstack/core'; -import { ObjectQLPlugin } from '@objectstack/objectql'; -import { DriverPlugin } from '@objectstack/runtime'; -import { InMemoryDriver } from '@objectstack/driver-memory'; - -describe('Hook Integration', () => { - it('executes hook on insert', async () => { - const kernel = new LiteKernel(); - kernel.use(new ObjectQLPlugin()); - kernel.use(new DriverPlugin(new InMemoryDriver())); - - // Register hook - const ql = kernel.getService('objectql'); - ql.registerHook('beforeInsert', async (ctx) => { - ctx.input.created_at = '2026-04-13T10:00:00Z'; - }, { object: 'account' }); - - // Test insert - const result = await ql.object('account').insert({ - name: 'Test Account', - }); - - expect(result.created_at).toBe('2026-04-13T10:00:00Z'); - - await kernel.shutdown(); - }); -}); -``` +Hook test harnesses — vitest units and `LiteKernel` integration setups — are the +**objectstack-platform** skill's surface (its frontmatter claims "test harnesses via +LiteKernel"): see [objectstack-platform/SKILL.md](../../objectstack-platform/SKILL.md). --- -## Performance Considerations - -### Hook Execution Overhead - -``` -Single Record Insert: -┌─────────────────┬──────────────┐ -│ Hook Count │ Overhead │ -├─────────────────┼──────────────┤ -│ 0 hooks │ ~1ms │ -│ 5 hooks │ ~5ms │ -│ 20 hooks │ ~20ms │ -└─────────────────┴──────────────┘ -``` - -### Optimization Tips - -1. **Use `condition` to filter** — Avoid executing handlers unnecessarily. -2. **Use `async: true` for non-critical side effects** — Don't block transactions. -3. **Batch operations in `after*` hooks** — Reduce database round-trips. -4. **Cache expensive lookups** — Use kernel cache service. -5. **Use specific `object` targets** — Avoid `object: '*'`. - -### Anti-Patterns - -```typescript -// ❌ BAD: Expensive synchronous operation -{ - events: ['beforeInsert'], - async: false, - handler: async (ctx) => { - await slowExternalAPI(ctx.input); // Blocks transaction - } -} - -// ✅ GOOD: Async background operation -{ - events: ['afterInsert'], - async: true, // Fire-and-forget - handler: async (ctx) => { - await slowExternalAPI(ctx.result); - } -} -``` - ---- - -## Advanced Topics - -### Dynamic Hook Registration - -```typescript -// Register hooks based on configuration -export const onEnable = async (ctx: { ql: ObjectQL }) => { - const config = await loadConfig(); - - config.objects.forEach(objectName => { - ctx.ql.registerHook('beforeInsert', async (hookCtx) => { - // Dynamic logic - }, { object: objectName }); - }); -}; -``` - -### Hook Composition - -```typescript -// Compose multiple validators -const validators = [ - validateEmail, - validatePhone, - validateWebsite, -]; - -const composedHook = defineHook({ - name: 'validation_suite', - object: 'account', - events: ['beforeInsert', 'beforeUpdate'], - handler: async (ctx) => { - for (const validator of validators) { - await validator(ctx); - } - }, -}); -``` - -### Conditional Hook Execution - -```typescript -const conditionalHook = defineHook({ - name: 'enterprise_only', - object: 'account', - events: ['afterInsert'], - handler: async (ctx) => { - // Check runtime condition - if (process.env.FEATURE_FLAG_ENTERPRISE !== 'true') { - return; // Skip execution - } - - // Enterprise-specific logic - }, -}); -``` - ---- - -## Troubleshooting - -### Common Issues - -**Issue:** Hook not executing - -**Solutions:** -1. Check `object` matches target object name -2. Verify `events` includes the expected event -3. Check `condition` doesn't filter out all records -4. Ensure hook is registered before operations - -**Issue:** Transaction rollback on `after*` hook error - -**Solution:** Set `onError: 'log'` or `async: true` - -**Issue:** Infinite loop (hook triggers itself) - -**Solution:** Use conditional checks, track execution state - -**Issue:** `ctx.api` is undefined - -**Solution:** Ensure ObjectQL engine is initialized with API support - -**Issue:** Performance degradation - -**Solutions:** -1. Use `async: true` for non-critical operations -2. Add `condition` to filter executions -3. Reduce number of global (`object: '*'`) hooks - --- ## References - `node_modules/@objectstack/spec/src/data/hook.zod.ts` — Hook schema definition, HookContext interface - [Project hooks pattern](../SKILL.md#lifecycle-hooks) — Hook integration in the data skill +- [objectstack-platform/references/plugin-hooks.md](../../objectstack-platform/references/plugin-hooks.md) — plugin/kernel hooks (a different extension point) +- [objectstack-automation](../../objectstack-automation/SKILL.md) — Flows and Workflows --- - -## Summary - -Hooks are the **primary extension mechanism** in ObjectStack. They enable you to: - -- ✅ Add custom validation and business rules -- ✅ Enrich data with calculated fields -- ✅ Trigger side effects and integrations -- ✅ Enforce security and compliance -- ✅ Implement audit trails -- ✅ Transform data in/out - -**Golden Rules:** - -1. Use `before*` for validation, `after*` for side effects -2. Set `async: true` for non-critical background work -3. Use `ctx.api` for cross-object operations -4. Handle errors gracefully with meaningful messages -5. Test hooks in isolation and integration - -For more advanced patterns, see the **objectstack-automation** skill for Flows and Workflows. diff --git a/skills/objectstack-data/rules/hooks.md b/skills/objectstack-data/rules/hooks.md deleted file mode 100644 index 59e58ffcc3..0000000000 --- a/skills/objectstack-data/rules/hooks.md +++ /dev/null @@ -1,209 +0,0 @@ -# Data Lifecycle Hooks (Reference) - -> **Note:** This document is a reference pointer. Complete documentation has been moved to the canonical hooks skill. - ---- - -## Complete Documentation - -For comprehensive data lifecycle hooks documentation, see: - -**→ [objectstack-data/references/data-hooks.md](../../objectstack-data/references/data-hooks.md)** - -The canonical reference includes: -- All 8 lifecycle events (beforeFind, afterFind, beforeInsert, afterInsert, beforeUpdate, afterUpdate, beforeDelete, afterDelete) -- Complete Hook definition schema -- HookContext API reference -- Registration methods (declarative, programmatic, file-based) -- 10+ common patterns with full examples -- Performance considerations and optimization tips -- Testing strategies (unit and integration) -- Best practices and anti-patterns - ---- - -## Quick Reference - -### Hook Definition - -```typescript -import { P } from '@objectstack/spec'; -import { defineHook, HookContext } from '@objectstack/spec/data'; - -const hook = defineHook({ - name: 'my_hook', // Required: unique identifier - object: 'account', // Required: target object(s) - events: ['beforeInsert'], // Required: lifecycle events - handler: async (ctx: HookContext) => { - // Your logic here - }, - priority: 100, // Optional: execution order - async: false, // Optional: background execution (after* only) - condition: P`record.status == 'active'`, // Optional: conditional execution (CEL) -}); -``` - -Prefer `defineHook()` over a bare `: Hook` literal (the same rule as -`defineDatasource`): it validates when the module is imported, so -constraint-level mistakes a bare annotation can't catch — a non-`snake_case` -`name`, a misspelled key routed through a spread — fail while you author -instead of at deploy, and the export carries defaults already materialized. - -### Logic: `body` (preferred) or `handler` (deprecated) - -A hook's logic comes from **either** an inline `handler` function **or** a -metadata-native `body`. Prefer **`body`** for new code — it is what a -metadata-only runtime executes, and it ships as plain JSON inside the build -artifact. `handler` (inline function) is deprecated; when both are present the -runtime uses `body`. - -```typescript -// Sandboxed body: `source` is the function body, run in an isolated QuickJS VM. -{ - name: 'fill_position_on_hire', - object: 'candidate', - events: ['afterUpdate'], - body: { - language: 'js', // 'js' (sandboxed) | 'expression' (pure CEL) - source: ` - if (!ctx.result || ctx.result.stage !== 'hired') return; - // afterUpdate ctx.result is PARTIAL — re-query for the lookup FK. - const rec = await ctx.api.object('candidate').findOne({ where: { id: ctx.result.id } }); - if (rec && rec.position_id) - await ctx.api.object('position').update({ id: rec.position_id, status: 'filled' }); - `, - capabilities: ['api.read', 'api.write'], // declare every ctx API the body touches - }, -} -``` - -Sandbox essentials (full contract in -[references/data-hooks.md → Sandboxed Hook Bodies](../references/data-hooks.md#sandboxed-hook-bodies-body--what-the-sandbox-ctx-can-call)): - -- **`ctx`** exposes `input`, `previous` (`undefined` on insert → `!ctx.previous` - detects *create*), `result` (⚠️ **partial** on afterUpdate — re-query for - unwritten fields), `user`, `session`, `event`, `object`, `api`, - `log` (`ctx.log.info(msg)`), `crypto` (`randomUUID`). -- **`ctx.api.object(n)`** repo: `find` / `findOne` / `count` / `insert` / - `update({ id, ...fields })` / `upsert` / `delete`. Query key is **`where`** - (object + `$`-operators) — **not** `filter: [[…]]`. -- **`capabilities`** (declare what the body uses, else it throws) — the five legal - tokens: `api.read`, `api.write`, `api.transaction`, `crypto.uuid`, `log`. - There is **no hashing capability**: `crypto.hash` was removed in spec 17 - because the sandbox never implemented it. -- Cross-object writes obey the **target's** sharing model — a `public_read` - target rejects the write with `FORBIDDEN`, and **admin is not exempt**. -- No `console` (use `ctx.log`), no `fetch` (use Connectors), no `import` / - `require` / module-scope helpers — a `body` must be self-contained. - -### 8 Lifecycle Events - -| Event | When Fires | Use Case | -|:------|:-----------|:---------| -| `beforeFind` | Before any read (`find` **and** `findOne`) | Filter queries, log access | -| `afterFind` | After any read (`find` **and** `findOne`) | Transform results, enrich data | -| `beforeInsert` | Before creating a record | Set defaults, validate | -| `afterInsert` | After creating a record | Send notifications | -| `beforeUpdate` | Before updating a record (single **or** bulk `multi:true`) | Validate changes | -| `afterUpdate` | After updating a record (single **or** bulk) | Trigger workflows | -| `beforeDelete` | Before deleting a record (single **or** bulk `multi:true`) | Check dependencies | -| `afterDelete` | After deleting a record (single **or** bulk) | Clean up related data | - -> **One read event, one write event per kind.** `beforeFind`/`afterFind` fire for -> `findOne` too (the event attaches to record materialization, not the method), and -> the write events fire on bulk `multi:true` operations as well. A bulk write hands -> hooks **no** row-scoping predicate: it lives on the engine-internal -> `OperationContext.ast`, so the RLS / sharing filters composed onto it bind -> the driver call itself, where no handler can widen them — scope a batch through -> `options.where` at the caller. The `after*` events instead dispatch **once per -> matched row**, each on a single-record-shaped context whose `input.id` names that -> row. There is no `beforeFindOne`, `beforeCount`, `beforeAggregate`, or -> `*Many` event. -> -> **Don't reach for a hook when a declarative mechanism already fits:** -> - Read authorization / row filtering → **RLS / permission rules**, not a `beforeFind` hook. -> - Field masking → **field-level metadata** (secret/masked fields), not an `afterFind` hook. -> - Delete guards → a **`beforeDelete`** hook (this is the right tool). - -### Common Patterns - -See the full documentation for complete examples of: - -1. **Setting Default Values** — Auto-populate fields on insert -2. **Data Validation** — Custom validation rules beyond declarative -3. **Preventing Deletion** — Block deletes based on conditions -4. **Data Enrichment** — Calculate and set derived fields -5. **Triggering Workflows** — Fire notifications and integrations -6. **Creating Related Records** — Maintain referential integrity -7. **External API Integration** — Sync with external systems -8. **Multi-Object Logic** — Cascade updates across objects -9. **Conditional Execution** — Use `condition` property -10. **Data Masking** — PII protection in read operations - ---- - -## Registration - -Three methods available: - -### 1. Declarative (in Stack) - -```typescript -// objectstack.config.ts -export default defineStack({ - hooks: [accountHook, contactHook], -}); -``` - -### 2. Programmatic (in Plugin) - -```typescript -ctx.ql.registerHook('beforeInsert', async (hookCtx) => { - // Handler logic -}, { object: 'account', priority: 100 }); -``` - -### 3. Hook Files (Convention) - -```typescript -// src/objects/account.hook.ts -export default { - name: 'account_logic', - object: 'account', - events: ['beforeInsert'], - handler: async (ctx) => { /* ... */ }, -}; -``` - ---- - -## Best Practices - -✅ **DO:** -1. Use `before*` for validation, `after*` for side effects -2. Set `async: true` for non-critical background work -3. Use `ctx.api` for cross-object operations -4. Handle errors gracefully with meaningful messages -5. Test hooks in isolation and integration - -❌ **DON'T:** -1. Don't perform expensive operations in `before*` hooks -2. Don't create infinite loops (hooks triggering themselves) -3. Don't use `object: '*'` unless absolutely necessary -4. Don't throw in `after*` hooks unless critical -5. Don't assume `ctx.session` exists - ---- - -## See Also - -- **[objectstack-data/SKILL.md#lifecycle-hooks](../../objectstack-data/SKILL.md#lifecycle-hooks)** — Complete hooks system overview -- **[objectstack-data/references/data-hooks.md](../../objectstack-data/references/data-hooks.md)** — Full data hooks documentation -- **[objectstack-platform/references/plugin-hooks.md](../../objectstack-platform/references/plugin-hooks.md)** — Plugin hook system -- **[objectstack-automation](../../objectstack-automation/SKILL.md)** — Flows and Workflows for advanced automation - ---- - -**For complete documentation with detailed examples, context API reference, testing strategies, and performance optimization, see the canonical reference:** - -→ **[objectstack-data/references/data-hooks.md](../../objectstack-data/references/data-hooks.md)** From 9c7988af4527d7b00bb7015db9fb50e25c9bc4c5 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 02:03:00 +0000 Subject: [PATCH 2/4] skills(data): SKILL.md -- teach sharingModel/Field face, correct INC-01/02, cut restatement Additions (all paid by deletions in this same file; SKILL.md 13892 -> 12535): DATA-F-01 `sharingModel` row + the four canonical values. Used by 31/31 real objects and absent from both property tables. A publish with no authored sharingModel is refused (`security-owd-unset`, 422) -- maintainer ruling 2026-08-13 recorded at packages/lint/src/authoring-rules.ts:1293-1296; enum at packages/spec/src/data/object.zod.ts:2177. DATA-F-02 `required` (write contract) vs `storage: { notNull: true }` (the DDL), ADR-0113 -- never mentioned in the package before. DATA-F-03 `icon` / `pluralLabel` / `highlightFields` rows. DATA-F-04 four security lint rows; three are publish-blocking errors. DATA-E-05 the Quick-Start Template now carries the face every real object has (pluralLabel/description/icon/highlightFields) and drops `indexes`, which appears in 1 of 26 corpus files. INC-01 (HIGH) -- the schema-evolution table named `required` as the NOT NULL dial and claimed dev auto-heals it. Both wrong: schema-drift.ts:795-798 compares `field.storage?.notNull`, explicitly NOT `required`; the emitted op is severity 'warning' / category 'needs_confirm' / 'relax_not_null' and the comment says "never auto-applied"; sql-driver.ts:10746 applies only category 'safe'. INC-02 (MED) -- the tell-tale promised a clean 400 " is required". schema-drift.ts:800-802: that configuration dies as a RAW DRIVER ERROR instead. Deletions: DATA-A-03 (intro + when-to-use restating the frontmatter), DATA-C-01 (two verbatim error transcripts; the query skill carries the same 400 text), DATA-C-04 (standalone tombstones -- each key is refused by a parse error carrying its own prescription, and objectstack-upgrade owns that path), DATA-C-05 (protection: schema + lock table + ONE example survive), DATA-D-12, DATA-D-13, DATA-D-15, DATA-F-06, DATA-G-01 (one CLI spelling: `os`). DATA-F-05 is a SHRINK, not a removal (retirement deferred pending #14296): `defineObjectExtension` and `readScope`/`writeScope` each keep one schema-pointing row -- zero corpus usage is evidence, not proof. DATA-A-01/A-02/F-06 rewrite the frontmatter description: it never said datasource / external / federated though rules/datasources.md is entirely that surface, never said lifecycle or protection, and fired on `*.seed.ts` -- a pattern that matches nothing in the repo. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1 --- skills/objectstack-data/SKILL.md | 318 ++++++++----------------------- 1 file changed, 84 insertions(+), 234 deletions(-) diff --git a/skills/objectstack-data/SKILL.md b/skills/objectstack-data/SKILL.md index 528106be8f..5335d9af74 100644 --- a/skills/objectstack-data/SKILL.md +++ b/skills/objectstack-data/SKILL.md @@ -3,13 +3,14 @@ name: objectstack-data description: > Design ObjectStack data schemas — objects, fields, field conditional rules, relationships, validations, indexes, lifecycle hooks, permissions, - row-level security — - and the seeds (`defineSeed()`) that load fixtures and - reference data alongside them. Use when the user is creating or - modifying `*.object.ts` / `*.seed.ts` files, picking field types, - modelling relationships, writing `beforeInsert`/`afterUpdate` hooks, - configuring per-object access control, or authoring bootstrap / demo - data. Use for `visibleWhen` / `readonlyWhen` / `requiredWhen` rules that + row-level security, data `lifecycle` retention/TTL/rotation, metadata + `protection` locks, and external / federated datasources + (`defineDatasource`) — and the seeds (`defineSeed()`) that load fixtures + and reference data alongside them. Use when the user is creating or + modifying `*.object.ts` files or `src/data/*.ts` seed modules, picking + field types, modelling relationships, writing `beforeInsert`/`afterUpdate` + hooks, configuring per-object access control, pointing an object at an + existing external database, or authoring bootstrap / demo data. Use for `visibleWhen` / `readonlyWhen` / `requiredWhen` rules that belong on fields. Do not use for querying data (see objectstack-query) or for plugin / kernel hooks (see objectstack-platform). CEL expressions in formulas / validations / sharing rules / dynamic seed values: load @@ -25,12 +26,6 @@ metadata: # Data Modeling — ObjectStack Data Protocol -Expert instructions for designing business data schemas using the ObjectStack -specification. This skill covers Object definitions, Field type selection, -relationship modelling, validation rules, index strategy, and lifecycle hooks. - ---- - ## Skill Boundaries | Need | Use instead | @@ -42,18 +37,6 @@ relationship modelling, validation rules, index strategy, and lifecycle hooks. --- -## When to Use This Skill - -- You are creating a **new business object** (e.g., `account`, `project_task`) -- You need to **choose the right field type** from the 49 supported types -- You are configuring **lookup / master-detail relationships** between objects -- You need to add **validation rules** (cross-field, state machine, format, etc.) -- You are optimising **query performance with indexes** -- You are extending an existing object with new fields or capabilities -- You need to **implement data lifecycle hooks** for business logic - ---- - ## Core Concepts ### Object Definition @@ -67,13 +50,27 @@ database table and exposes automatic CRUD APIs. |:---------|:-------|:-----------|:------------| | `name` | string | `snake_case` | Immutable machine identifier (`/^[a-z_][a-z0-9_]*$/`) | | `fields` | map | keys in `snake_case` | Field definitions | +| `sharingModel` | enum | one of the four below | Org-wide default record visibility (OWD). Zod marks it optional, but **a publish with no authored `sharingModel` is refused** with the 422 lint envelope `security-owd-unset` — absence is not a decision (maintainer ruling 2026-08-13). Author it on every object | + +**`sharingModel` — the four canonical values** (ADR-0090 D4; legacy aliases +removed). A *custom* object that omits it resolves to `private` at runtime, but +the publish door rejects it before that: + +| Value | Who can read / write | +|:--|:--| +| `private` | owner only (widen with sharing rules, RLS, or `readScope`/`writeScope`) | +| `public_read` | everyone reads; the owner writes | +| `public_read_write` | everyone reads and writes | +| `controlled_by_parent` | inherited from the master record (`master_detail` children) | **Important optional properties:** | Property | Default | Description | |:---------|:--------|:------------| | `label` | Auto from `name` | Human-readable singular label | -| `pluralLabel` | — | Plural form (e.g., "Accounts") | +| `pluralLabel` | — | Plural form (e.g., "Accounts") — on 31/31 objects in the reference apps; author it alongside `label` | +| `icon` | — | Icon name for nav, list headers and lookup pickers — on 31/31 objects in the reference apps | +| `highlightFields` | derived | Ordered field keys used as a record's compact face: the columns a **related list** renders on the parent's detail page, and what a lookup picker shows. Declare it on the CHILD object (see [Relationships](./rules/relationships.md)) | | `namespace` | — | **Not a schema key** — `ObjectSchema.create()` rejects unknown keys, so authoring it is a build error. Embed the prefix directly in `name` instead (e.g. `name: 'crm_account'`) | | `datasource` | `'default'` | Target datasource ID for virtualized data | | `nameField` | derived (e.g. `'name'`/`'title'`) | **Canonical** record-title field — the stored field used as the record's display name. Use a single text/email field, or a formula field (`returnType: 'text'`) for a composite title | @@ -159,31 +156,9 @@ fresh as whatever writes it. Cover both write paths: Rows written by a path that bypasses hooks (bulk import, direct SQL) need a one-off backfill. See [Lifecycle Hooks](./references/data-hooks.md). -**The errors an author sees for the dotted path** (grep either back to here). -`os validate` → `searchable-field-unknown`: - -```text -searchableFields entry "project_id.name" is not a field on object "task". The -declaration is stale: searching it can never match, and the engine silently -drops it — leaving a narrower search than declared, or the auto-default set once -every entry is dropped. - -hint: 'search' scans this object's own columns, so a related record's column -cannot be a search target — expand the relation and search the related object, -or copy the value onto a stored text field here. Clients echo this declaration -verbatim as the '$searchFields' override, so a stale entry becomes a 400 -INVALID_FIELD on list search, not just a quietly narrowed one. -``` - -A request carrying the dotted path is `400 INVALID_FIELD`: - -```text -Unknown field 'project_id.name' on object 'task'. '$searchFields' narrows which -columns 'search' scans, so a name the object does not declare cannot narrow -anything — and the engine used to drop it and scan the default columns instead, -answering a NARROWER search with a WIDER one. 'search' scans this object's own -columns; a related record's column cannot be a search target. -``` +Both spellings are refused loudly: `os validate` reports +`searchable-field-unknown`, and a request naming the dotted path is `400 +INVALID_FIELD`. Each message carries its own prescription. Cross-object search paths are rejected by design, not pending. Do not invent a per-project convention for this — the mirror field is the answer. @@ -329,7 +304,11 @@ import { ObjectSchema } from '@objectstack/spec/data'; export default ObjectSchema.create({ name: 'support_case', label: 'Support Case', - sharingModel: 'private', + pluralLabel: 'Support Cases', + description: 'A customer-reported issue tracked to resolution.', + icon: 'life-buoy', + sharingModel: 'private', // required in practice — see above + highlightFields: ['subject', 'status', 'priority'], enable: { trackHistory: true, feeds: true, @@ -371,13 +350,12 @@ export default ObjectSchema.create({ message: 'Invalid status transition.', }, ], - indexes: [ - { fields: ['status', 'priority'] }, - { fields: ['account'] }, - ], }); ``` +Declared `indexes` are a separate decision — see +[Index Strategy](./rules/indexing.md). + --- ## Schema evolution on an existing database @@ -390,16 +368,26 @@ schema, and the **database column wins at write time**: | Change | Existing DB on restart | |--------|------------------------| | add object / field / index | ✅ applied automatically (additive) | -| `required: true → false` (relax `NOT NULL`) | dev auto-heals (`autoMigrate:'safe'`); otherwise `os migrate apply` | +| `storage: { notNull: true }` removed (relax `NOT NULL`) | ⚠️ **never auto-applied** — the drift is `category: 'needs_confirm'` (`relax_not_null`), so `os migrate apply` confirms it. Relaxing `required` alone changes no column | | `unique` re-scoped global → per-tenant | dev auto-heals; otherwise `os migrate apply` (`replace_unique_index`) | | type / length change, drop field, rename | `os migrate apply` (`--allow-destructive` for drops / tightenings) | | declared index removed, or its columns changed | `os migrate apply` (`--allow-destructive` when it drops, or rebuilds as `UNIQUE`) | -Tell-tale: `/meta` reports a field optional but a write still 400s -`" is required"` — that is a stale `NOT NULL` column (physical drift), -**not** a validator bug. `os dev` reconciles loosening automatically; otherwise -`os migrate plan` to preview and `os migrate apply` to reconcile. CLI details: -see **objectstack-platform**. +**`required` is not the `NOT NULL` dial (ADR-0113).** `required` is the +**write contract** — the engine refuses an insert that omits the value — and it +implies nothing about the column. The physical constraint is a separate explicit +opt-in, `storage: { notNull: true }`, and it is what drift detection compares +against. So tightening `required` on a deployed object is safe (existing null +rows stay readable), while declaring `storage.notNull` over null rows is a +destructive migration. The two cannot be combined with `requiredWhen` — a +conditional contract cannot be an unconditional column constraint. + +Tell-tale: `/meta` reports a field optional (no `required`, no `storage.notNull`) +but writes that omit it fail with a **raw driver error** rather than a clean +validation 400 — that is a stale `NOT NULL` column, not a validator bug. Run +`os migrate plan` to preview and `os migrate apply` to reconcile, or ratify the +column by declaring `storage: { notNull: true }`. CLI details: see +**objectstack-platform**. --- @@ -536,48 +524,17 @@ contract**, and the canonical patterns. --- -## CRM Schema Blueprint (Production Pattern) - -Mirror these CRM-style patterns when designing enterprise metadata objects: - -| Pattern | Typical Location | Implementation Cue | -|:--|:--|:--| -| Object layout via field groups | `src/objects/*.object.ts` | Use `fieldGroups[]` + per-field `group` for deterministic form structure | -| Capability gating | `src/objects/*.object.ts` | Use `enable` flags (`trackHistory`, `apiMethods`, `files`, `feeds`, `activities`) per object | -| Index + validation pairing | `src/objects/*.object.ts` | Keep `indexes[]` aligned to common filters and enforce invariants with `validations[]` | -| Relationship constraints | `src/objects/*.object.ts` | Use `lookup` + `lookupFilters` (`[{ field, operator, value }]`) for constrained child selection | -| Lifecycle automation | `src/objects/*.hook.ts` | Use a lifecycle **hook** (authored with `defineHook()`, registered via `defineStack({ hooks })` or the `*.hook.ts` convention scan) or a top-level `record_change` flow for field updates triggered by record changes. There is **no** object-level `workflows[]` field — authoring one is a build error. | -| State transitions | `src/objects/*.object.ts` | Prefer explicit `state_machine` validation rules (one per state field) — there is **no** separate `stateMachines` map | - -For metadata authoring, keep expressions in CEL (`P\`...\``, `F\`...\``, -`cel\`...\``) and avoid legacy formula-string syntax. - --- ## Object Extension Model -When extending an object you do not own, author an extension with -`defineObjectExtension()` and register it on the stack's `objectExtensions` -array: - -```typescript -import { defineObjectExtension } from '@objectstack/spec'; - -export const accountExtension = defineObjectExtension({ - extend: 'account', // target object name - fields: { custom_score: { type: 'number' } }, - priority: 300, // higher = applied later -}); - -// objectstack.config.ts -// defineStack({ objectExtensions: [accountExtension], ... }) -``` - -- `priority` controls merge order (default `200`; range `0–999`) -- Extensions can add fields, validations, and indexes — but cannot remove them -- Do **not** author `ownership: 'extend'` on an object schema — the object-level - `ownership` property is the *record-ownership* enum - (`'user' | 'business_unit' | 'org' | 'none'`), unrelated to extensions +To add fields/validations/indexes to an object you do not own, author +`defineObjectExtension({ extend, fields, priority })` and register it on +`defineStack({ objectExtensions: [...] })`. `priority` sets merge order (default +`200`, range `0–999`); an extension can add but never remove. ⛔ Not the same key +as object-level `ownership` (the record-ownership enum +`'user' | 'business_unit' | 'org' | 'none'`). Schema: +`node_modules/@objectstack/spec/src/data/object.zod.ts` (`ObjectExtensionSchema`). --- @@ -648,43 +605,17 @@ is `active: false`; the validity window has passed; `organization_id` mismatch. `GET /api/v1/security/explain?object=&operation=&userId=` answers from the enforcing code path (explaining another user needs `manage_users`). -### Access depth (scope-depth) — the ERP "see my unit / my unit and below" axis - -For owner-scoped (`private`) objects, a per-object grant on a permission set can -carry **`readScope` / `writeScope`** that *widens the owner-match declaratively* — -the ERP "my own / my reports / my unit / my unit and below / whole org" axis -(ADR-0057 D1). It saves hand-writing one RLS policy per object. - -```typescript -// in a permission set's `objects` map -objects: { - account: { - allowRead: true, allowEdit: true, - readScope: 'unit_and_below', // see accounts owned by my BU + descendant BUs - writeScope: 'own', // but only edit my own - }, -} -``` +### Access depth (`readScope` / `writeScope`) -| Scope | Who you can see / write | -|:--|:--| -| `own` | `owner == me` (baseline; unset = this) | -| `own_and_reports` | me + everyone below me on the `sys_user.manager_id` chain | -| `unit` | owners in my business unit (`sys_business_unit`) | -| `unit_and_below` | my BU + all descendant BUs (BFS) | -| `org` | the whole tenant (≈ `viewAllRecords` / `modifyAllRecords`) | - -Resolves at request time into an `owner_id IN (…)` set and AND-injects like RLS -(no compiler change; ADR-0055). Sharing rules still widen on top. - -> ⚠️ **Open-core boundary (ADR-0016).** `own` and `org` work in open-source. The -> **hierarchy-relative** scopes — `own_and_reports` / `unit` / `unit_and_below` — -> need the **paid** `@objectstack/security-enterprise` plugin (BU-subtree + -> manager-chain resolver). Without it they **fail closed to `own`** (never -> fail-open), and `defineStack` errors if a grant uses one without -> `requires: ['hierarchy-security']`. In an open-source app, author `own` / `org` -> + explicit sharing rules; reach for `unit*` only when the enterprise plugin is -> present. +On an owner-scoped (`private`) object, a per-object grant in a permission set may +carry `readScope` / `writeScope` to widen the owner match declaratively instead of +hand-writing an RLS policy (ADR-0057 D1): `own` (default) · `own_and_reports` · +`unit` · `unit_and_below` · `org`. It resolves at request time into an +`owner_id IN (…)` set and AND-injects like RLS. ⚠️ Open-core boundary (ADR-0016): +only `own` and `org` work in open source — the hierarchy-relative three need the +paid `@objectstack/security-enterprise` plugin, **fail closed to `own`** without +it, and `defineStack` errors unless the grant declares +`requires: ['hierarchy-security']`. Schema: `PermissionSetSchema.objects.*`. ### Row-Level Security (RLS) @@ -738,16 +669,10 @@ A legacy SQL-style `=` / `IN (...)` predicate still compiles via a **deprecated* is the declarative way to set the org-wide default — prefer those over hand-written policies for the common cases. -> **Removed:** a former object-level `rls` config (`RLSConfigSchema`, a free-form -> CEL `predicate` on the object) was **removed** from the spec (ADR-0056 D8, -> "design+enforce or remove"). Permission-set `rowLevelSecurity` policies are the -> only RLS surface — author them as shown above. - ### Sensitive fields — `secret` type + `requiredPermissions` -The former `encryptionConfig` field key was **pruned from `FieldSchema`** — it -had no runtime consumer. `maskingRule` is **live** (plugin-security's -FieldMasker enforces it). The real channels are: +`maskingRule` is **live** (plugin-security's FieldMasker enforces it). The real +channels are: **Encrypted-at-rest values — `type: 'secret'` (ADR-0100).** For reversible machine credentials (DB passwords, API keys, tokens): the engine encrypts the @@ -793,9 +718,6 @@ tenancy: { } ``` -- The former `shared` / `isolated` / `hybrid` mode key (`tenancy.strategy`) was - **retired** — an unknown `tenancy` key is now a loud parse error with - upgrade guidance, never silently stripped. - **Database-per-tenant isolation is not object metadata** — it is an environment/deployment choice (each environment carries its own database URL). - Platform/env-global objects declare `tenancy: { enabled: false }` to opt out @@ -888,101 +810,30 @@ unknown keys are rejected. | `no-delete` | ✅ | ❌ | Tenant may customize fields but the object itself must exist | | `full` | ❌ | ❌ | Core admin UI / platform identity (e.g. `sys_user`, `app/setup`) | -### Example — fully locked platform object +### Example -```ts +```typescript // src/objects/sys-user.object.ts import { ObjectSchema } from '@objectstack/spec/data'; export const SysUserObject = ObjectSchema.create({ name: 'sys_user', label: 'User', + sharingModel: 'public_read', protection: { lock: 'full', reason: 'Core identity object — see ADR-0010.', docsUrl: 'https://objectstack.ai/docs/references/shared/protection', }, - fields: { /* ... */ }, -}); -``` - -### Example — schema-locked but deletable - -```ts -// src/objects/sys-role.object.ts -import { ObjectSchema } from '@objectstack/spec/data'; - -export const SysRoleObject = ObjectSchema.create({ - name: 'sys_role', - label: 'Role', - protection: { - lock: 'no-overlay', - reason: 'RBAC schema is platform-defined — see ADR-0010.', - docsUrl: 'https://objectstack.ai/docs/references/shared/protection', - }, - fields: { /* ... */ }, + fields: { username: { type: 'text', required: true } }, }); ``` -### Example — locking a shipped app - The same block works on non-object metadata (apps, views, dashboards, flows, -agents, tools, skills, reports, email-templates): - -```ts -// src/apps/setup.app.ts -import { defineApp } from '@objectstack/spec'; - -export const SetupApp = defineApp({ - name: 'setup', - label: 'Setup', - protection: { - lock: 'full', - reason: 'Core admin UI shipped by @objectstack/platform-objects — see ADR-0010.', - docsUrl: 'https://objectstack.ai/docs/references/shared/protection', - }, - // ... -}); -``` - -### Enforcement - -- **REST**: `PUT /api/v1/meta/:type/:name` and `DELETE` return `403 item_locked` - for any operation the lock forbids. Layered-read endpoints - (`GET ?layers=true`) include `lock`, `lockReason`, `lockDocsUrl`, `lockSource`, - and `packageId` so Studio can render the banner. -- **Studio**: `ResourceEditPage` renders a banner with the lock reason and the - "View docs" link (from `docsUrl`); edit + delete buttons are hidden according - to the lock. -- **Package vs Artifact source**: `_lockSource: 'package'` when the lock comes - from a code-shipped schema, `'artifact'` when set by a workspace artifact. - Artifact locks override package locks (workspace wins). - -### Authoring guidance - -- Default to **no `protection` block** for tenant-authored metadata. -- Use `full` for anything Studio editing would break at runtime (core identity, - platform admin UIs, system flows). -- Use `no-overlay` for schemas that platform owns but a tenant may legitimately - not need (then they can delete it). -- Always include `reason` — it is the only thing the end-user sees first. -- Prefer pointing `docsUrl` to an ADR or onboarding doc, not a marketing page. - ---- - -## Advanced Features Checklist - -| Feature | When to Consider | -|:--------|:-----------------| -| `tenancy` | Multi-tenant SaaS — `{ enabled: true, tenantField: 'tenant_id' }` row-level isolation (DB-per-tenant is an environment/deployment choice, not object metadata) | -| `lifecycle` | Append-only / high-write-rate objects — retention / rotation / archival contract; see [rules/lifecycle.md](./rules/lifecycle.md) | -| per-field `trackHistory` | Render a field's value changes as human-readable activity-timeline entries (pair with `enable.trackHistory`, ADR-0052 §5b) | - -> The former `softDelete` / `versioning` object keys were **removed** from the -> spec (ADR-0049 enforce-or-remove) — authoring them is now a build -> error with upgrade guidance. `partitioning` / `cdc` were never schema keys, -> and the `encryptionConfig` field key was pruned (see -> [Sensitive fields](#sensitive-fields--secret-type--requiredpermissions)). +agents, tools, skills, reports, email-templates). Enforcement: `PUT`/`DELETE` on +`/api/v1/meta/:type/:name` return `403 item_locked`, and an artifact lock +overrides a package lock. Default to **no** `protection` block for +tenant-authored metadata. --- @@ -1125,13 +976,12 @@ changes must produce byte-identical `dist/objectstack.json`. CEL + pinned | Scope demo data with `env: ['dev','test']` | Keep noise out of prod | | Order seeds parent → child in the exported array | References resolve at load time | | Use `replace` only on cache/lookup tables, with comments | Data-loss footgun | -| One `{object}.seed.ts` file per object | Readability at scale | --- ## Linting & Generation Quality -`objectstack lint` checks the data model against the conventions in this skill — +`os lint` checks the data model against the conventions in this skill — not just naming/labels but the relationship/master-detail/roll-up patterns. Run it after authoring or generating metadata. Severities: `error` (structural, fails the command), `warning` (likely-wrong choice), `suggestion` (nudge). @@ -1149,6 +999,10 @@ Data-model rules (in addition to naming/label/i18n): | `rollup/missing-summary` | suggestion | a parent of numeric master_detail children with no roll-up `summary` | | `field/select-missing-options` | warning | a `select`/`multiselect`/`radio` with no `options` (or options source) | | `object/missing-name-field` | suggestion | an object with no `nameField` (ADR-0079's canonical title pointer) and no name-like field (`name`/`title`/`subject`/`label`/`full_name`/`display_name`/`code`) | +| `security-owd-unset` | error | an object published with no authored `sharingModel` (422 lint envelope; absence is not a decision) | +| `security-owd-alias` | error | a legacy OWD spelling instead of the canonical four (ADR-0090 D4) | +| `security-external-wider-than-internal` | error | `externalSharingModel` wider than `sharingModel` (ADR-0090 D11) | +| `security-master-detail-ungranted` | warning | a `master_detail` child whose master carries no matching grant | > **`code` counts for R9, but is NOT a title-derivation key.** R9's name-like > list above is the *looser* of two "name-like" sets, and the difference is @@ -1167,17 +1021,13 @@ Data-model rules (in addition to naming/label/i18n): These same rules are the **rubric for AI-generated metadata** — a generation is "good" exactly when it is schema-valid and lint-clean: -- `objectstack lint --score` — print a 0–100 metadata-quality score (+ letter +- `os lint --score` — print a 0–100 metadata-quality score (+ letter grade and severity breakdown) for the current project. Schema errors and lint errors weigh most; suggestions barely move it. -- `objectstack lint --eval` — run the generation eval over a bundled golden +- `os lint --eval` — run the generation eval over a bundled golden corpus (invoice+lines, project+tasks, blog+comments, expense+lines, account+contacts) offline; each case must clear the pass bar (`--eval-min`, default 75). Deterministic, no API key. -- `objectstack lint --eval --generator ./gen.mjs` — **live** eval: the module - default-exports `(prompt, id) => stack`; wire it to your agent / - `AIService.generateObject` (+ blueprint→metadata expansion) - to benchmark a real model against the same rubric. When generating object metadata, target a lint-clean model: master_detail (with `required` + `deleteBehavior` + `inlineEdit` for line items), roll-up summaries From c2ddad7299c3b8b5c2fe247b0ebb2dd044022c32 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 02:24:53 +0000 Subject: [PATCH 3/4] skills(data): rules/ diet -- naming, indexing, validation, relationships, field-types MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DATA-D-11 naming.md six ❌/✅ pairs + Critical Rules restatements + Rationale for a rule the regex in the table already states. 773 -> 248. DATA-D-05/06/07 indexing.md: generic RDBMS 101 ("when to add indexes"), four ❌/✅ pairs + three subsections teaching one textbook fact (left-to-right) collapsed to one paragraph, the unsourced `Table Size -> Max Indexes` table, the pg_stat_user_indexes / SHOW INDEX SQL, and a 10-item Best Practices restating the file. The naming block survives -- `name` is a declared key. DATA-C-04 indexing.md's standalone `type`/`partial` tombstone clause; the in-line ❌ block at "Retired and Redundant Keys" stays, because there a wrong key sits beside the right one. DATA-C-02 validation CEL-vs-SQL is objectstack-formula's anchor and this package's description already says to load it alongside. The INVERSION rule stays -- it is validation-specific and lives nowhere else. DATA-D-09/D-10 validation.md: five patterns restating four examples already on the page; 10 best practices + 4 unsourced performance claims about two rules that lower to the same CEL evaluator. DATA-B-07 the "no `unique` validation type" rule was stated at 7 sites; the restatements in validation.md and relationships.md go. DATA-D-08 relationships.md Best Practices + Performance Considerations. DATA-D-14/E-03/G-02 field-types.md: three ❌/✅ pairs duplicating naming.md, the type table and a lint rule (the autonumber pair survives) pay for the `Field.*` factory preamble -- the corpus is ~92% factory (~250 calls vs 22 literals) and this file had zero `Field.*`. DATA-E-02 two blocks carried an `import` and then opened `{` / `validations: [` at statement position, which TS parses as a labelled-statement block. Both are now real declarations. DATA-E-01 `os:check` markers on the two self-contained SKILL.md blocks that compile (definePermissionSet, the protection example). Verified: `check:skill-examples` type-checks 261 marked blocks. Two liveness-ledger bindings needed handling, both fallout of deleting rules/hooks.md and the F-05 shrink: - the `### Access depth (scope-depth) …` heading is ledger-bound, so the shrink's incidental rename is reverted -- the ledger is right. - `hook-lifecycle-events-rule` binds a heading in the deleted file. The gate's own message prescribes "Re-point `heading`, or delete the row"; re-pointing would duplicate `hook-lifecycle-events-reference`, which already binds the same symbol (HookEvent) to the surviving canonical file, so the row is dropped. This is the one edit outside the declared file surface and it is reported as such. `check:skill-docs` DEMANDED regeneration (it runs in CI, lint.yml:3986): the frontmatter description changed, so `pnpm --filter @objectstack/spec gen:skill-docs` regenerated skills/README.md and content/docs/ai/skills-reference.mdx. Generated `references/_index.md` is untouched and `check:skill-refs` is green. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1 --- content/docs/ai/skills-reference.mdx | 6 +- scripts/check-skill-identifier-liveness.mjs | 8 - skills/README.md | 2 +- skills/objectstack-data/SKILL.md | 4 +- skills/objectstack-data/rules/field-types.md | 72 +++----- skills/objectstack-data/rules/indexing.md | 160 +----------------- skills/objectstack-data/rules/naming.md | 86 ---------- .../objectstack-data/rules/relationships.md | 20 +-- skills/objectstack-data/rules/validation.md | 115 +------------ 9 files changed, 40 insertions(+), 433 deletions(-) diff --git a/content/docs/ai/skills-reference.mdx b/content/docs/ai/skills-reference.mdx index 657b24c9bb..f01daf3702 100644 --- a/content/docs/ai/skills-reference.mdx +++ b/content/docs/ai/skills-reference.mdx @@ -44,7 +44,7 @@ ObjectStack ships **11 skills** — one per authoring domain, plus process skill | # | Skill | Domain | Path | What it covers | | :--- | :--- | :--- | :--- | :--- | | 1 | [Platform](#platform) | `platform` | `skills/objectstack-platform/` | Bootstrap, configure, extend, and operate ObjectStack runtimes. Covers project setup (`defineStack`, drivers, adapters, scaffolding), plugin and service development (PluginContext, DI, kernel hooks like `kernel:ready`), and operations (CLI commands, migrations, deployment, test harnesses via LiteKernel). | -| 2 | [Data](#data) | `data` | `skills/objectstack-data/` | Design ObjectStack data schemas — objects, fields, field conditional rules, relationships, validations, indexes, lifecycle hooks, permissions, row-level security — and the seeds (`defineSeed()`) that load fixtures and reference data alongside them. | +| 2 | [Data](#data) | `data` | `skills/objectstack-data/` | Design ObjectStack data schemas — objects, fields, field conditional rules, relationships, validations, indexes, lifecycle hooks, permissions, row-level security, data `lifecycle` retention/TTL/rotation, metadata `protection` locks, and external / federated datasources (`defineDatasource`) — and the seeds (`defineSeed()`) that load fixtures and reference data alongside them. | | 3 | [Query](#query) | `query` | `skills/objectstack-query/` | Construct ObjectQL queries — filters, sorting, pagination, aggregation, relation expansion, and full-text search. | | 4 | [UI](#ui) | `ui` | `skills/objectstack-ui/` | Author ObjectStack UI metadata — Views (list/form/kanban/calendar/gantt), Apps (navigation), Pages (structured plus the HTML and React source-authoring tiers, ADR-0080/0081), Dashboards, Reports, Charts, Actions, and package Docs (`src/docs/*.md`). | | 5 | [Automation](#automation) | `automation` | `skills/objectstack-automation/` | Design ObjectStack automation — Flows (visual logic), Triggers, Approvals, state machines, scheduled jobs, and webhooks. | @@ -75,9 +75,9 @@ Do not use for data schema design (see objectstack-data) or query patterns (see **Domain** `data` · **Path** `skills/objectstack-data/` -Design ObjectStack data schemas — objects, fields, field conditional rules, relationships, validations, indexes, lifecycle hooks, permissions, row-level security — and the seeds (`defineSeed()`) that load fixtures and reference data alongside them. +Design ObjectStack data schemas — objects, fields, field conditional rules, relationships, validations, indexes, lifecycle hooks, permissions, row-level security, data `lifecycle` retention/TTL/rotation, metadata `protection` locks, and external / federated datasources (`defineDatasource`) — and the seeds (`defineSeed()`) that load fixtures and reference data alongside them. -Use when the user is creating or modifying `*.object.ts` / `*.seed.ts` files, picking field types, modelling relationships, writing `beforeInsert`/`afterUpdate` hooks, configuring per-object access control, or authoring bootstrap / demo data. Use for `visibleWhen` / `readonlyWhen` / `requiredWhen` rules that belong on fields. +Use when the user is creating or modifying `*.object.ts` files or `src/data/*.ts` seed modules, picking field types, modelling relationships, writing `beforeInsert`/`afterUpdate` hooks, configuring per-object access control, pointing an object at an existing external database, or authoring bootstrap / demo data. Use for `visibleWhen` / `readonlyWhen` / `requiredWhen` rules that belong on fields. Do not use for querying data (see objectstack-query) or for plugin / kernel hooks (see objectstack-platform). CEL expressions in formulas / validations / sharing rules / dynamic seed values: load objectstack-formula alongside. diff --git a/scripts/check-skill-identifier-liveness.mjs b/scripts/check-skill-identifier-liveness.mjs index 92de491967..b99571ea6c 100644 --- a/scripts/check-skill-identifier-liveness.mjs +++ b/scripts/check-skill-identifier-liveness.mjs @@ -306,14 +306,6 @@ const BINDINGS = [ source: 'packages/spec/src/data/hook.zod.ts', why: 'The reference table of lifecycle events. A missing event is a hook an author never learns exists.', }, - { - id: 'hook-lifecycle-events-rule', - file: 'skills/objectstack-data/rules/hooks.md', - heading: '### 8 Lifecycle Events', - symbol: 'HookEvent', - source: 'packages/spec/src/data/hook.zod.ts', - why: 'The heading states the count, so the section claims exhaustiveness in its own words — and the count goes stale silently when the enum grows.', - }, { id: 'lifecycle-classes', file: 'skills/objectstack-data/rules/lifecycle.md', diff --git a/skills/README.md b/skills/README.md index eba5382b3b..dd1f67036e 100644 --- a/skills/README.md +++ b/skills/README.md @@ -30,7 +30,7 @@ apps too). | Skill | Domain | What it covers | |:------|:-------|:---------------| | [Platform](./objectstack-platform/SKILL.md) | `platform` | Bootstrap, configure, extend, and operate ObjectStack runtimes. Covers project setup (`defineStack`, drivers, adapters, scaffolding), plugin and service development (PluginContext, DI, kernel hooks like `kernel:ready`), and operations (CLI commands, migrations, deployment, test harnesses via LiteKernel). | -| [Data](./objectstack-data/SKILL.md) | `data` | Design ObjectStack data schemas — objects, fields, field conditional rules, relationships, validations, indexes, lifecycle hooks, permissions, row-level security — and the seeds (`defineSeed()`) that load fixtures and reference data alongside them. | +| [Data](./objectstack-data/SKILL.md) | `data` | Design ObjectStack data schemas — objects, fields, field conditional rules, relationships, validations, indexes, lifecycle hooks, permissions, row-level security, data `lifecycle` retention/TTL/rotation, metadata `protection` locks, and external / federated datasources (`defineDatasource`) — and the seeds (`defineSeed()`) that load fixtures and reference data alongside them. | | [Query](./objectstack-query/SKILL.md) | `query` | Construct ObjectQL queries — filters, sorting, pagination, aggregation, relation expansion, and full-text search. | | [UI](./objectstack-ui/SKILL.md) | `ui` | Author ObjectStack UI metadata — Views (list/form/kanban/calendar/gantt), Apps (navigation), Pages (structured plus the HTML and React source-authoring tiers, ADR-0080/0081), Dashboards, Reports, Charts, Actions, and package Docs (`src/docs/*.md`). | | [Automation](./objectstack-automation/SKILL.md) | `automation` | Design ObjectStack automation — Flows (visual logic), Triggers, Approvals, state machines, scheduled jobs, and webhooks. | diff --git a/skills/objectstack-data/SKILL.md b/skills/objectstack-data/SKILL.md index 5335d9af74..a7c732dc7a 100644 --- a/skills/objectstack-data/SKILL.md +++ b/skills/objectstack-data/SKILL.md @@ -548,6 +548,7 @@ schema. There is no object-level `permissions` key (and no `hooks` key either) Grant CRUD access per object with boolean bits on a permission set: + ```typescript import { definePermissionSet } from '@objectstack/spec'; @@ -605,7 +606,7 @@ is `active: false`; the validity window has passed; `organization_id` mismatch. `GET /api/v1/security/explain?object=&operation=&userId=` answers from the enforcing code path (explaining another user needs `manage_users`). -### Access depth (`readScope` / `writeScope`) +### Access depth (scope-depth) — the ERP "see my unit / my unit and below" axis On an owner-scoped (`private`) object, a per-object grant in a permission set may carry `readScope` / `writeScope` to widen the owner match declaratively instead of @@ -812,6 +813,7 @@ unknown keys are rejected. ### Example + ```typescript // src/objects/sys-user.object.ts import { ObjectSchema } from '@objectstack/spec/data'; diff --git a/skills/objectstack-data/rules/field-types.md b/skills/objectstack-data/rules/field-types.md index a9f2967f3c..80bbe19a41 100644 --- a/skills/objectstack-data/rules/field-types.md +++ b/skills/objectstack-data/rules/field-types.md @@ -2,6 +2,24 @@ Quick reference for choosing the right field type from 49 available options. +## Two spellings: `Field.*` factory or object literal + +The tables below give the **object-literal** spelling. Real apps overwhelmingly +use the equivalent **`Field.*` factory** (`@objectstack/spec`) — same schema, plus +per-helper JSDoc and inference (`Field.lookup`'s const generic is what lets +`defineSeed()` type its reference values). A helper exists for every type below +except a few authored as literals (`tags`, `progress`, `repeater`, `vector`). + +```typescript +import { Field } from '@objectstack/spec'; + +fields: { + name: Field.text({ required: true, maxLength: 255 }), + stage: Field.select(['new', 'won'], { required: true }), // options 1st + account_id: Field.lookup('account', { required: true }), // reference 1st +} +``` + > **Config columns list only real `FieldSchema` keys.** Per-type display knobs > beyond these do **not** exist — an unknown field key is REFUSED at parse > (`unrecognized_keys`), so don't invent `theme`, `rows`, or @@ -298,11 +316,11 @@ grouped number (never a hardcoded `$`). The same chain backs analytics measures ```typescript import { F } from '@objectstack/spec'; -{ +const total = { type: 'formula', expression: F`record.amount * record.tax_rate`, // CEL — `record.` prefixes required returnType: 'number', // 'number' | 'text' | 'boolean' | 'date' (no 'currency') -} +}; ``` ### Summary (Roll-up) @@ -359,56 +377,6 @@ parse. `dimensions` is the flat field-level key. ## Incorrect vs Correct -### ❌ Incorrect — Wrong Type for Email - -```typescript -{ - type: 'text', // ❌ No built-in email validation - maxLength: 255, -} -``` - -### ✅ Correct — Use email Type - -```typescript -{ - type: 'email', // ✅ Built-in validation + UI affordances -} -``` - -### ❌ Incorrect — Uppercase Option Value - -```typescript -options: [ - { label: 'Done', value: 'Done' }, // ❌ Uppercase -] -``` - -### ✅ Correct — Lowercase Option Value - -```typescript -options: [ - { label: 'Done', value: 'done' }, // ✅ Lowercase -] -``` - -### ❌ Incorrect — Missing Reference - -```typescript -{ - type: 'lookup', // ❌ No reference specified -} -``` - -### ✅ Correct — Specify Reference - -```typescript -{ - type: 'lookup', - reference: 'account', // ✅ Target object specified -} -``` - ### ❌ Incorrect — Autonumber interpolating an optional / adjacent field ```typescript diff --git a/skills/objectstack-data/rules/indexing.md b/skills/objectstack-data/rules/indexing.md index 96d718239c..c73464928f 100644 --- a/skills/objectstack-data/rules/indexing.md +++ b/skills/objectstack-data/rules/indexing.md @@ -78,32 +78,6 @@ Notes an author has to know: tenancy posture — state the business boundary, not the deployment shape. - **`'tenant'` and `'org'` are rejected.** The word is `'organization'`. -## When to Add Indexes - -### ✅ Always Index - -1. **Foreign keys** — declare them; never automatic -2. **Filter fields** — Columns used in WHERE clauses -3. **Sort fields** — Columns used in ORDER BY -4. **Unique constraints** — Enforce uniqueness at DB level -5. **Composite filters** — Fields commonly filtered together - -### ⚠️ Consider Indexing - -1. **Join columns** — Non-foreign-key join fields -2. **Frequent aggregations** — GROUP BY columns -3. **Range queries** — Date ranges, numeric ranges -4. **Subset queries** — a partial index can help, but it is a database-layer - migration, not a declaration (see below) - -### ❌ Avoid Indexing - -1. **Low cardinality** — Boolean fields (unless combined with others) -2. **Rarely queried** — Fields almost never filtered/sorted -3. **High write volume** — Every insert/update maintains indexes -4. **Large text** — Full-text index only when needed -5. **Calculated fields** — Index source fields instead - ## Examples ### Composite Index (Multi-Column) @@ -158,86 +132,17 @@ indexes: [ ] ``` -### ❌ Incorrect — Over-Indexing - -```typescript -indexes: [ - { fields: ['is_active'] }, // ❌ Boolean, low cardinality - { fields: ['is_deleted'] }, // ❌ Boolean, low cardinality - { fields: ['is_verified'] }, // ❌ Boolean, low cardinality - { fields: ['status'] }, // ❌ Already indexed elsewhere - { fields: ['created_at'] }, // ❌ Already indexed elsewhere -] -``` - -### ✅ Correct — Strategic Indexing - -```typescript -indexes: [ - // Composite for common query pattern - { fields: ['is_active', 'created_at'] }, - - // Single index covers multiple queries - { fields: ['status', 'priority'] }, -] -``` - -### ❌ Incorrect — Wrong Order in Composite - -```typescript -indexes: [ - // Querying by created_at with status filter - { fields: ['created_at', 'status'] }, // ❌ Wrong order -] -``` - -### ✅ Correct — Most Selective First - -```typescript -indexes: [ - // Status is more selective (filters more), goes first - { fields: ['status', 'created_at'] }, // ✅ Correct order -] -``` - -## Composite Index Strategy - -### Order Matters - -```typescript -// Index: ['status', 'priority', 'created_at'] - -// ✅ Can use index -WHERE status = 'active' -WHERE status = 'active' AND priority = 'high' -WHERE status = 'active' AND priority = 'high' ORDER BY created_at - -// ❌ Cannot use index efficiently -WHERE priority = 'high' // Skips first column -WHERE created_at > '2026-01-01' // Skips first two columns -``` - -### Left-to-Right Rule - -Composite indexes are used **left-to-right**. Querying only the second or third column doesn't use the index. - -### Selectivity Rule - -Place most **selective** (unique) fields first, then range/sort fields last. +### Composite index order -```typescript -// Good order: selective → range -{ fields: ['tenant_id', 'status', 'created_at'] } - -// Bad order: range → selective -{ fields: ['created_at', 'status', 'tenant_id'] } -``` +A composite index is used **left-to-right**: `['status', 'priority', 'created_at']` +serves `status`, `status + priority`, and `status + priority ORDER BY created_at`, +but not a query that filters on `priority` alone. Put the most selective column +first and the range/sort column last. ## Access methods and partial indexes -Both are real database capabilities. Neither is part of the **declaration** -surface, and the keys that used to pretend otherwise (`type`, `partial`) were -retired at protocol 17 precisely because nothing consumed them. +Both are real database capabilities, and neither is part of the **declaration** +surface — issue them from a database-layer migration. **Access method (`btree` / `hash` / `gin` / `gist` / `fulltext`).** The driver and dialect decide. Postgres defaults to B-tree, which is the right choice for @@ -274,28 +179,6 @@ form in a migration. > index you create in a migration is not reported as drift and is never > targeted by `os migrate apply --allow-destructive`. -## Performance Trade-offs - -### Index Benefits -- ✅ Faster SELECT queries -- ✅ Faster ORDER BY operations -- ✅ Faster JOIN operations -- ✅ Enforce uniqueness at DB level - -### Index Costs -- ❌ Slower INSERT/UPDATE/DELETE (index maintenance) -- ❌ Increased storage (each index duplicates data) -- ❌ Query planner overhead (more indexes = more choices) - -### General Guidelines - -| Table Size | Max Indexes | Reasoning | -|:-----------|:------------|:----------| -| < 1K rows | 2-3 | Low volume, indexes may not help | -| 1K - 100K rows | 3-5 | Balance read/write performance | -| 100K - 1M rows | 5-8 | Read optimization critical | -| > 1M rows | 8-12 | Consider partitioning + indexes | - ## Index Naming Convention ObjectStack auto-generates index names. To specify custom names: @@ -309,35 +192,6 @@ ObjectStack auto-generates index names. To specify custom names: **Auto-generated pattern:** `idx_{object}_{field1}_{field2}_{...}` -## Monitoring Index Usage - -Use database tools to monitor index usage: - -```sql --- PostgreSQL: Find unused indexes -SELECT - schemaname, tablename, indexname, idx_scan -FROM pg_stat_user_indexes -WHERE idx_scan = 0 -ORDER BY schemaname, tablename; - --- MySQL: Check index cardinality -SHOW INDEX FROM your_table; -``` - -## Best Practices - -1. **Index foreign keys** — always; declare each one -2. **Composite for common queries** — Combine frequently filtered columns -3. **Order matters** — Most selective field first -4. **Partial for subsets** — but build it in a migration, not a declaration -5. **Unique for constraints** — Enforce at DB level, and always state the scope -6. **Monitor usage** — Remove unused indexes -7. **Limit total indexes** — Balance read/write performance -8. **Avoid over-indexing** — More indexes ≠ better performance -9. **Test with production data** — Index effectiveness depends on data volume -10. **Use EXPLAIN** — Verify query plans before deploying indexes - ## Common Query Patterns ### Filter by Status + Sort by Date diff --git a/skills/objectstack-data/rules/naming.md b/skills/objectstack-data/rules/naming.md index 0e78ddaf77..1123d9acf0 100644 --- a/skills/objectstack-data/rules/naming.md +++ b/skills/objectstack-data/rules/naming.md @@ -12,86 +12,6 @@ ObjectStack enforces strict naming conventions to ensure consistency and machine | Option `value` | lowercase machine ID | lowercase | `in_progress` | | Option `label` | Any case | — | `"In Progress"` | -## Incorrect vs Correct - -### ❌ Incorrect — Object Name - -```typescript -export default ObjectSchema.create({ - name: 'ProjectTask', // ❌ PascalCase not allowed - fields: { /* ... */ } -}); -``` - -### ✅ Correct — Object Name - -```typescript -export default ObjectSchema.create({ - name: 'project_task', // ✅ snake_case - fields: { /* ... */ } -}); -``` - -### ❌ Incorrect — Field Keys - -```typescript -fields: { - firstName: { type: 'text' }, // ❌ camelCase not allowed - 'Due-Date': { type: 'datetime' }, // ❌ kebab-case not allowed - Status: { type: 'select' }, // ❌ PascalCase not allowed -} -``` - -### ✅ Correct — Field Keys - -```typescript -fields: { - first_name: { type: 'text' }, // ✅ snake_case - due_date: { type: 'datetime' }, // ✅ snake_case - status: { type: 'select' }, // ✅ snake_case -} -``` - -### ❌ Incorrect — Schema Properties - -```typescript -{ - type: 'lookup', - reference: 'account', - lookup_filters: [], // ❌ snake_case not allowed for TS config - max_length: 255, // ❌ snake_case not allowed for TS config -} -``` - -### ✅ Correct — Schema Properties - -```typescript -{ - type: 'lookup', - reference: 'account', - lookupFilters: [{ field: 'status', operator: 'eq', value: 'active' }], // ✅ camelCase - maxLength: 255, // ✅ camelCase -} -``` - -### ❌ Incorrect — Select Option Values - -```typescript -options: [ - { label: 'In Progress', value: 'In Progress' }, // ❌ space/caps in value - { label: 'Done', value: 'Done' }, // ❌ uppercase in value -] -``` - -### ✅ Correct — Select Option Values - -```typescript -options: [ - { label: 'In Progress', value: 'in_progress' }, // ✅ lowercase, snake_case - { label: 'Done', value: 'done' }, // ✅ lowercase -] -``` - ## Critical Rules 1. **Never** use `camelCase` or `PascalCase` for object names or field keys @@ -99,9 +19,3 @@ options: [ 3. **Option values** must be lowercase machine identifiers (use snake_case for multi-word) 4. **Option labels** can use any case for display purposes 5. **Machine names are immutable** — changing them requires data migration - -## Rationale - -- **snake_case for data**: Database-friendly, SQL-compatible, cross-platform consistency -- **camelCase for config**: TypeScript/JavaScript convention for object properties -- **Lowercase option values**: Case-sensitive database comparisons, URL-safe, API-friendly diff --git a/skills/objectstack-data/rules/relationships.md b/skills/objectstack-data/rules/relationships.md index e2656dda09..6fa8663bbe 100644 --- a/skills/objectstack-data/rules/relationships.md +++ b/skills/objectstack-data/rules/relationships.md @@ -123,8 +123,7 @@ export default ObjectSchema.create({ hours_allocated: { type: 'number' }, }, indexes: [ - // One assignment per (project, employee) pair — uniqueness is an index - // concern; there is no 'unique' validation type. + // One assignment per (project, employee) pair. { fields: ['project_id', 'employee_id'], unique: 'organization' }, ], }); @@ -394,20 +393,3 @@ export default ObjectSchema.create({ }, }); ``` - -## Best Practices - -1. **Use lookup by default** — Only use master_detail when lifecycle coupling is required -2. **Unique constraints on junctions** — Prevent duplicate many-to-many entries -3. **Meaningful junction names** — Use descriptive names like `project_assignment` not `project_employee` -4. **deleteBehavior on master_detail** — Always specify `cascade` or `restrict` -5. **Required on master_detail** — Child should always require parent -6. **Roll-ups for aggregation** — Use summary fields on parent for counts/sums -7. **lookupFilters for scoping** — Limit lookup options to relevant records (`lookupFilters: [{ field, operator: 'eq', value }]`) - -## Performance Considerations - -- **Index foreign keys** — Always create indexes on lookup/master_detail fields -- **Avoid deep hierarchies** — tree relationships > 5 levels can impact query performance -- **Junction table indexes** — Composite index on both foreign keys in junction tables -- **Summary field caching** — Roll-up summaries are cached and updated on child changes diff --git a/skills/objectstack-data/rules/validation.md b/skills/objectstack-data/rules/validation.md index dc8de38238..180e6c0f8a 100644 --- a/skills/objectstack-data/rules/validation.md +++ b/skills/objectstack-data/rules/validation.md @@ -21,15 +21,15 @@ There is no other type. In particular: with a **unique index** ([see below](#uniqueness--use-unique-indexes)). - **No `async` / `custom` type** — external checks and arbitrary validation code belong in a `beforeInsert` / `beforeUpdate` **lifecycle hook** - (see [hooks.md](./hooks.md)). + (see [references/data-hooks.md](../references/data-hooks.md)). ## Expression Syntax `condition` / `when` are **CEL predicates** (ADR-0032). Author them with the `P` tag from `@objectstack/spec`; a plain string is also accepted and parsed as CEL. Record fields are addressed as `record.`; on update the prior -row is available as `previous.`. CEL uses `==`, `!=`, `&&`, `||`, -`!` — not SQL's `=`, `AND`, `IS NULL`. +row is available as `previous.`. CEL operator and null-handling rules: +see **objectstack-formula**. **⚠️ CRITICAL:** For `script` **and** `cross_field`, the predicate expresses the **failure** condition — validation **fails** when it evaluates to `true`. @@ -39,7 +39,7 @@ the **failure** condition — validation **fails** when it evaluates to `true`. ```typescript import { P } from '@objectstack/spec'; -validations: [ +const validations = [ { name: 'prevent_past_dates', type: 'script', @@ -48,7 +48,7 @@ validations: [ severity: 'error', events: ['insert', 'update'], }, -] +]; ``` ### Common Script Patterns @@ -310,26 +310,6 @@ Lower numbers execute **first**. } ``` -### ❌ Incorrect — SQL Syntax in a CEL Predicate - -```typescript -{ - type: 'script', - condition: "status = 'approved' AND approver_id IS NULL", // ❌ not CEL - message: 'Approved records need an approver', -} -``` - -### ✅ Correct — CEL Predicate - -```typescript -{ - type: 'script', - condition: P`record.status == 'approved' && isBlank(record.approver_id)`, - message: 'Approved records need an approver', -} -``` - ### ❌ Incorrect — Validation Fires Too Often ```typescript @@ -351,88 +331,3 @@ Lower numbers execute **first**. events: ['update'], // ✅ Only validate on update } ``` - -## Common Patterns - -### Prevent Backdating - -```typescript -{ - name: 'no_backdate', - type: 'script', - condition: P`record.effective_date < today()`, - message: 'Effective date cannot be in the past', - events: ['insert'], -} -``` - -### Require Approval for High Values - -```typescript -{ - name: 'high_value_approval', - type: 'conditional', - when: P`record.amount > 10000`, - message: 'High-value transaction validation', - then: { - name: 'approval_required', - type: 'script', - condition: P`isBlank(record.approved_by)`, - message: 'High-value transactions require approval', - }, -} -``` - -### Email Domain Whitelist - -```typescript -{ - name: 'email_domain', - type: 'format', - field: 'email', - regex: '^[a-zA-Z0-9._%+-]+@(company\\.com|partner\\.com)$', - message: 'Email must be from company.com or partner.com', -} -``` - -### Phone Number Format - -```typescript -{ - name: 'phone_format', - type: 'format', - field: 'phone', - regex: '^\\+?[1-9]\\d{1,14}$', // E.164 format - message: 'Phone must be in international format (+1234567890)', -} -``` - -### Composite Uniqueness (Tenant + Email) - -Not a validation — declare a unique index on the object: - -```typescript -indexes: [ - { fields: ['department', 'email'], unique: 'organization' }, -] -``` - -## Best Practices - -1. **Use declarative validation first** — Only use script validation when declarative rules don't fit -2. **Severity matters** — Use `warning` for soft rules, `error` for hard rules -3. **Events scope** — Only validate on relevant operations to avoid overhead -4. **Priority order** — System validations first (0-99), app validations second (100-999), user validations last (1000+) -5. **Clear error messages** — Tell users exactly what's wrong and how to fix it -6. **State machine for workflows** — Use state_machine instead of complex script logic -7. **Uniqueness is an index concern** — Declare `indexes: [{ fields, unique: 'organization' | 'global' }]` with the scope stated, never a script-based existence check -8. **External checks are hooks** — Call APIs from `beforeInsert`/`beforeUpdate` hooks, not validations -9. **Cross-field for comparisons** — More efficient than script validation -10. **Test thoroughly** — Validate edge cases, nulls, empty strings - -## Performance Considerations - -- **Script validations are expensive** — Use sparingly, prefer declarative rules -- **Priority affects order** — Lower priority = runs first -- **Unique indexes are enforced by the database** — no per-write query cost beyond index maintenance -- **State machine is optimized** — Better than complex conditional logic From 061e5f4c2019ec06bf6ecacaea0019a15497d0dd Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 02:46:44 +0000 Subject: [PATCH 4/4] skills(data): ratchet DOWN the role-word baseline for SKILL.md (4 -> 2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The DATA-C-04 / D-12 / D-13 deletions removed two role-word occurrences from skills/objectstack-data/SKILL.md, so `pnpm check:role-word` failed as a ratchet-DOWN: "role-word count improved 4 -> 2 — run `node scripts/check-role-word.mjs --update` and commit the baseline". That remedy is the author's own (baseline EXPANSION is the maintainer-only direction), so the baseline is updated here. The diff is one line, for this file only, and it shrinks. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1 --- scripts/role-word-baseline.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/role-word-baseline.json b/scripts/role-word-baseline.json index 119885f46c..37a4411672 100644 --- a/scripts/role-word-baseline.json +++ b/scripts/role-word-baseline.json @@ -36,7 +36,7 @@ "skills/objectstack-ai/SKILL.md": 5, "skills/objectstack-api/SKILL.md": 1, "skills/objectstack-automation/SKILL.md": 1, - "skills/objectstack-data/SKILL.md": 4, + "skills/objectstack-data/SKILL.md": 2, "skills/objectstack-data/rules/relationships.md": 1, "skills/objectstack-platform/SKILL.md": 2, "skills/objectstack-query/rules/filters.md": 9,