From 268b889e417e9595b2c21eb7436f78a70d13c49a Mon Sep 17 00:00:00 2001 From: coodos Date: Mon, 31 Aug 2026 21:40:32 +0800 Subject: [PATCH 1/4] docs: state that the eVault is the source of truth, and how to propose an ontology --- docs/docs/Getting Started/getting-started.md | 2 + docs/docs/Infrastructure/Ontology.md | 94 +++++++++++++++++- docs/docs/W3DS Basics/Access-Policy.md | 2 +- docs/docs/W3DS Basics/Binding-Documents.md | 2 +- docs/docs/W3DS Basics/Data-Ownership-Rules.md | 96 +++++++++++++++++++ docs/docs/W3DS Basics/Links.md | 2 +- docs/docs/W3DS Basics/W3ID.md | 2 +- docs/docs/W3DS Basics/eName.md | 2 +- 8 files changed, 195 insertions(+), 7 deletions(-) create mode 100644 docs/docs/W3DS Basics/Data-Ownership-Rules.md diff --git a/docs/docs/Getting Started/getting-started.md b/docs/docs/Getting Started/getting-started.md index 0e67ff356..c6595c7e0 100644 --- a/docs/docs/Getting Started/getting-started.md +++ b/docs/docs/Getting Started/getting-started.md @@ -203,4 +203,6 @@ The [Provisioner](/docs/W3DS%20Basics/Links) hosts `/provision`; the [Registry]( - Understand [Authentication](/docs/W3DS%20Protocol/Authentication) - How users authenticate with platforms - Learn about [Signing](/docs/W3DS%20Protocol/Signing) - Signature creation and verification - Explore [Signature Formats](/docs/W3DS%20Protocol/Signature-Formats) - Technical details on cryptographic signatures +- Read the [Data Ownership Rules](/docs/W3DS%20Basics/Data-Ownership-Rules) - Where data lives, and why the eVault is the source of truth - Build a platform with the [Post Platform Guide](/docs/Post%20Platform%20Guide/getting-started) - Step-by-step guide to creating a W3DS-compatible platform +- Building with an AI coding agent? Load the [W3DS agent skill](/docs/Post%20Platform%20Guide/ai-agent-skill) first - it keeps the agent grounded in these docs instead of guessing diff --git a/docs/docs/Infrastructure/Ontology.md b/docs/docs/Infrastructure/Ontology.md index f35a0a069..a6af37e5f 100644 --- a/docs/docs/Infrastructure/Ontology.md +++ b/docs/docs/Infrastructure/Ontology.md @@ -24,13 +24,16 @@ Returns a list of all available schemas. ```json [ - { "id": "550e8400-e29b-41d4-a716-446655440000", "title": "User" }, - { "id": "550e8400-e29b-41d4-a716-446655440001", "title": "SocialMediaPost" } + { "id": "550e8400-e29b-41d4-a716-446655440000", "title": "User", "domain": "identity" }, + { "id": "550e8400-e29b-41d4-a716-446655440001", "title": "SocialMediaPost", "domain": "social" } ] ``` - `id`: Schema W3ID (`schemaId`). - `title`: Human-readable schema title. +- `domain`: The domain the schema belongs to, or `null` if untagged. See [GET /domains](#get-domains). + +This endpoint is the only correct way to obtain a `schemaId`. Match on `title`, then confirm the field names with `GET /schemas/:id` before writing a mapping. Schema IDs are not derivable, not sequential, and not stable enough to recall from memory — a wrong `schemaId` means every awareness packet for that type is silently dropped by receiving platforms. ### GET /schemas/:id @@ -44,6 +47,30 @@ Returns the full JSON Schema for the given W3ID. Use this when you need the comp - **404**: Schema not found for the given W3ID. +### GET /domains + +Returns the domain list every schema is tagged with — the same list a platform is granted access to, one domain at a time. + +**Response** (200): + +```json +{ + "schemaId": "", + "domains": [ + { "id": "identity", "label": "Identity", "description": "..." }, + { "id": "social", "label": "Social", "description": "..." } + ] +} +``` + +The list is not a separate config file: it is read from the `Domain` schema's own enum, so it is versioned, browsable and fetchable like any other type. + +### GET /domains/:id/schemas + +Returns every schema under one domain: `{ "domain": { ... }, "schemas": [ { "id", "title" } ] }`. **404** if the domain does not exist. + +Use this when you know the subject area but not the type name — "what does W3DS already have for finance?" — before concluding that nothing fits and [proposing a new ontology](#proposing-a-new-ontology). + ### Human-facing viewer - **GET /** — Renders a viewer page that lists schemas and supports search. Optional query `?q=...` filters by title or ID; `?schema=` shows one schema. @@ -83,6 +110,68 @@ Example (conceptually): In eVault, a MetaEnvelope for a post would have `ontology: "550e8400-e29b-41d4-a716-446655440001"`, and its Envelopes would have `ontology` values such as `content`, `authorId`, `createdAt`. +## Proposing a new ontology + +Nothing in W3DS obliges you to squeeze your data into an existing type. If no schema fits what you are modelling, the correct move is to **propose a new one** — never to invent a `schemaId` and ship it. + +An invented `schemaId` does not fail loudly. The MetaEnvelope is written, the awareness packet fans out, and every receiving platform finds no mapping for that schema and drops it. The data becomes unreachable to the ecosystem while looking perfectly healthy on the platform that wrote it. + +### Before proposing + +1. `GET /schemas` and search the titles. +2. `GET /domains/:id/schemas` for the domain your data belongs to. +3. Read the near misses in full with `GET /schemas/:id`. + +Reuse beats addition, and **extending a near match by PR beats creating a parallel type**. Two schemas that mean the same thing split the ecosystem in half: platforms mapping one will not see data from platforms mapping the other. + +### Write the schema + +Schemas are ordinary files in the [prototype repository](https://github.com/MetaState-Prototype-Project/prototype), under `services/ontology/schemas/.json`. The service loads the directory into an in-memory index at boot; there is no database and no registration call. + +```json +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "schemaId": "", + "title": "Bookmark", + "domain": "productivity", + "type": "object", + "properties": { + "id": { + "type": "string", + "format": "uuid", + "description": "The unique identifier for the bookmark" + }, + "userId": { + "type": "string", + "format": "uuid", + "description": "The ID of the user who created the bookmark" + }, + "createdAt": { + "type": "string", + "format": "date-time", + "description": "When the bookmark was created" + } + }, + "required": ["id", "userId", "createdAt"], + "additionalProperties": false +} +``` + +Requirements: + +- **`schemaId`** — a freshly generated random UUIDv4 (`uuidgen`, `crypto.randomUUID()`). Never derive one from an existing ID, never continue a numeric sequence you notice in the directory, and never reuse an ID from another schema. +- **`title`** — the type name as other platforms will search for it. Singular, PascalCase. +- **`domain`** — one value from the `Domain` schema's enum, fetchable at `GET /domains`. Platforms are granted access domain by domain, so this decides who can consume the type. +- **`properties`** — every field with a `description`. Each property name becomes an Envelope's `ontology` value in the eVault, so **name fields for their cross-platform meaning, not after your local columns**. `authorId` is a W3DS field name; `fk_user_id_2` is not. +- **`required`** — the fields a consumer can rely on being present. +- **`additionalProperties`** — `false`, unless you have a specific reason. + +### Open the PR + +Add the file, open a pull request against the prototype repository, and say in the description what the type is for and which platform will write it. Reviewers will ask whether an existing schema could have carried the data — answer that question in the PR body and you will save a round trip. + +Until the PR merges and the service redeploys, **the type does not exist**. Do not ship a `mapping.json` referencing an unmerged `schemaId`; the sync will look fine locally and silently drop everywhere else. + ## Available schemas To see all available schemas, call `GET /schemas` on the [Ontology production service](/docs/W3DS%20Basics/Links) or browse the [viewer](https://ontology.w3ds.metastate.foundation/) at the production base URL. @@ -97,3 +186,4 @@ To see all available schemas, call `GET /schemas` on the [Ontology production se - [eVault](/docs/Infrastructure/eVault) — MetaEnvelopes, Envelopes, and the `ontology` field - [W3DS Basics](/docs/W3DS%20Basics/getting-started) — Ontology and schema concepts - [Links](/docs/W3DS%20Basics/Links) — Production Ontology base URL +- [Data Ownership Rules](/docs/W3DS%20Basics/Data-Ownership-Rules) — why every persisted entity needs an ontology diff --git a/docs/docs/W3DS Basics/Access-Policy.md b/docs/docs/W3DS Basics/Access-Policy.md index b0714f518..2af472586 100644 --- a/docs/docs/W3DS Basics/Access-Policy.md +++ b/docs/docs/W3DS Basics/Access-Policy.md @@ -1,5 +1,5 @@ --- -sidebar_position: 6 +sidebar_position: 7 --- # Access Policy diff --git a/docs/docs/W3DS Basics/Binding-Documents.md b/docs/docs/W3DS Basics/Binding-Documents.md index b33b55efb..37b500185 100644 --- a/docs/docs/W3DS Basics/Binding-Documents.md +++ b/docs/docs/W3DS Basics/Binding-Documents.md @@ -1,5 +1,5 @@ --- -sidebar_position: 5 +sidebar_position: 6 --- # Binding Documents diff --git a/docs/docs/W3DS Basics/Data-Ownership-Rules.md b/docs/docs/W3DS Basics/Data-Ownership-Rules.md new file mode 100644 index 000000000..140eaa057 --- /dev/null +++ b/docs/docs/W3DS Basics/Data-Ownership-Rules.md @@ -0,0 +1,96 @@ +--- +sidebar_position: 3 +--- + +# Data Ownership Rules + +This page is the rule set for anyone — human or coding agent — deciding **where a piece of data lives**. Everything else in the Post Platform Guide tells you how to move data. This tells you what belongs where, and why. + +## The rule + +W3DS states the principle in [Getting Started](/docs/Getting%20Started/getting-started#core-concept): + +> **Users, groups, and objects own their own eVaults**. All data about a person, group, or object is stored in their eVault, and platforms act as frontends that display and interact with this data, while also serving as caches and aggregators for improved performance and user experience. + +Stated as a rule you can apply while building: + +> **The eVault is the source of truth. Anything a platform stores is a projection of it.** + +Those two sentences are the same claim. "Cache and aggregator" is not a loophole that lets a platform own data — it is permission to keep a fast local copy of data that is authoritative somewhere else. A platform database is not forbidden. A platform database that is the *only* place some user data exists is. + +This is what separates a W3DS-native application from a conventional one with synchronisation bolted on. Both have a local database. Only one of them can be deleted without losing anything. + +## The reconstructability test + +One question decides almost every case: + +> **If the platform database were dropped and rebuilt by replaying the relevant eVaults, what would be lost?** + +- **Nothing that matters** — the database is a projection. This is correct, and it is how [Pictique, Blabsy and eCurrency](/docs/Post%20Platform%20Guide/getting-started) work. +- **Something a user would miss** — that data has no home but yours. You have taken ownership of it without meaning to. Fix the design before writing more code. + +Apply the test per entity type, not per application. A platform is usually correct about its posts and wrong about the one table someone added in a hurry. + +### Worked cases + +| Data | Reconstructable? | Verdict | +|---|---|---| +| A user's posts, mapped and synced to their eVault | Yes — replay from the author's eVault | Projection. Correct. | +| A draft the user never published, stored only in your Postgres | No | **Violation.** Drafts are the user's data; give them an ontology and an owner, or do not persist them. | +| A login session, a nonce, a job queue row | Nothing to reconstruct | Operational state. Correct — see below. | +| `(localId, globalId)` mapping rows | Rebuildable, but only by re-syncing | Operational state. Correct, and required. | +| A cached avatar URL resolved from a `w3ds://file` URI | Yes — re-dereference | Cache. Correct, if it can be re-derived. | + +## What every persisted entity needs + +Before a new entity type is persisted anywhere, three things must be true: + +1. **An ontology.** A `schemaId` resolved from the [Ontology service](/docs/Infrastructure/Ontology), not invented. If nothing fits, [propose one](/docs/Infrastructure/Ontology#proposing-a-new-ontology) — do not proceed with a made-up identifier. +2. **A resolvable owner.** An `ownerEnamePath` that resolves to an eName for *every* row, not most of them. As [Web3 Adapter](/docs/Infrastructure/Web3-Adapter) puts it: "Data is always written to that owner's eVault." The owner is the data subject — the person or group the data is *about* — not the platform that happened to receive the write. +3. **A write path to the eVault.** A named call site: a `handleChange` after the local write, or a direct eVault write for a stateless app. "We will add sync later" means the platform owns the data today. + +If any of the three is missing, the entity is platform-owned. That is the thing this page exists to prevent. + +## Legitimate local-only state + +The rule is about *user* data. These are operational and may live only on the platform: + +- Sessions, auth nonces, and the short-lived session IDs from the [`w3ds://auth`](/docs/W3DS%20Protocol/Authentication) flow. +- Job queues, retry state, outbox rows, and dead letters. +- Rate limits, feature flags, and request logs. +- The `(localId, globalId)` mapping table the [Web3 Adapter](/docs/Infrastructure/Web3-Adapter) needs to avoid duplicating entities. +- Cached Registry resolutions and platform profile data — explicitly sanctioned in [Platform eVault registration](/docs/Post%20Platform%20Guide/platform-evault-registration), which tells platforms to save `w3id` and `uri` locally and reuse them on every boot. +- Derived indexes, search indexes, aggregates and denormalised read models built *from* eVault-sourced records. + +The common thread: none of it is data about a user that a user would expect to take with them. + +## What a projection may not do + +- Be the only home for user data. +- Hold a field that has no counterpart in the entity's ontology. A column with nowhere to go in the mapping is data the platform has quietly claimed. +- Be read in preference to eVault-derived state when the eVault is reachable and current. +- Outlive the owner's decision to revoke access. Access policy is the owner's to set — see [Access Policy](/docs/W3DS%20Basics/Access-Policy) — and a projection that ignores a revocation is a copy the owner no longer consented to. +- Be treated as authoritative during a conflict. It is downstream by construction. + +## Do not mirror what you can already observe + +Duplication is its own failure. [File URIs](/docs/W3DS%20Protocol/File-URIs) makes the canonical version of this point: consuming the awareness packet is how a platform learns about a new blob — "there is no need to mirror the upload as a second envelope under the `File` ontology just to make it observable." The same reasoning applies generally. If a record is already observable through the Awareness Protocol, subscribing beats copying. + +## Bounds on how much a projection can be trusted + +Synchronisation is eventual, and the [Awareness Protocol](/docs/W3DS%20Protocol/Awareness-Protocol) is prototype-level. Design the projection to tolerate all of this: + +- **Last-write-wins.** No merge, no CRDT. +- **No ordering guarantee**, and no at-least-once delivery. +- **Fire-and-forget fanout** with no retries at the protocol level. The requesting platform is excluded from its own fanout. +- **A delay after creation** before fanout, to prevent ping-pong; updates fan out immediately. + +Consequences for your code: webhook handling must be **idempotent on the global `id`**, reads must tolerate a record that has not arrived yet, and nothing user-visible should depend on two platforms agreeing at the same instant. + +## Stateless applications + +An application that writes directly to eVaults and keeps no local database does not need a [Web3 Adapter](/docs/Infrastructure/Web3-Adapter) at all — the adapter exists to keep a database in sync, and there is nothing to sync. This is the simplest way to be W3DS-native, and it is the right default for small applications. + +## For coding agents + +If you are an AI agent building on W3DS, these rules are enforced by the [W3DS agent skill](/docs/Post%20Platform%20Guide/ai-agent-skill). The short version: run the reconstructability test before persisting anything new, resolve every identifier instead of recalling it, and stop and ask rather than designing a platform that owns its users' data. diff --git a/docs/docs/W3DS Basics/Links.md b/docs/docs/W3DS Basics/Links.md index 2d389ec09..81186090e 100644 --- a/docs/docs/W3DS Basics/Links.md +++ b/docs/docs/W3DS Basics/Links.md @@ -1,5 +1,5 @@ --- -sidebar_position: 7 +sidebar_position: 8 --- # Links diff --git a/docs/docs/W3DS Basics/W3ID.md b/docs/docs/W3DS Basics/W3ID.md index 6285e312c..fc678db5e 100644 --- a/docs/docs/W3DS Basics/W3ID.md +++ b/docs/docs/W3DS Basics/W3ID.md @@ -1,5 +1,5 @@ --- -sidebar_position: 3 +sidebar_position: 4 --- # W3ID diff --git a/docs/docs/W3DS Basics/eName.md b/docs/docs/W3DS Basics/eName.md index 4a6a4931a..91b56910e 100644 --- a/docs/docs/W3DS Basics/eName.md +++ b/docs/docs/W3DS Basics/eName.md @@ -1,5 +1,5 @@ --- -sidebar_position: 4 +sidebar_position: 5 --- # eName From a6ae708e3826968f0889748642bb58a505477250 Mon Sep 17 00:00:00 2001 From: coodos Date: Mon, 31 Aug 2026 21:40:38 +0800 Subject: [PATCH 2/4] feat(docs): publish llms.txt, llms-full.txt and the agent skill at /skill --- docs/docusaurus.config.ts | 3 + docs/plugins/llms-txt.js | 234 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 237 insertions(+) create mode 100644 docs/plugins/llms-txt.js diff --git a/docs/docusaurus.config.ts b/docs/docusaurus.config.ts index 616fc4c96..2d8c2544d 100644 --- a/docs/docusaurus.config.ts +++ b/docs/docusaurus.config.ts @@ -20,6 +20,9 @@ const config: Config = { themes: ['@docusaurus/theme-mermaid'], + // Publishes /llms.txt, /llms-full.txt and /skill/** for AI coding agents. + plugins: ['./plugins/llms-txt.js'], + // Production URL: hosted at docs.w3ds.metastate.foundation url: 'https://docs.w3ds.metastate.foundation', baseUrl: '/', diff --git a/docs/plugins/llms-txt.js b/docs/plugins/llms-txt.js new file mode 100644 index 000000000..2b73742f2 --- /dev/null +++ b/docs/plugins/llms-txt.js @@ -0,0 +1,234 @@ +/** + * Publishes machine-readable views of this site for AI coding agents: + * + * /llms.txt an index of every page, with URLs and one-line summaries + * /llms-full.txt the whole corpus in one file + * /skill/SKILL.md the W3DS agent skill, fetchable without installing anything + * /skill/reference/* its reference files + * /skill/w3ds-full.txt the skill concatenated, for agents that read a single file + * + * The docs site is the authoritative source for W3DS, so an agent that can fetch + * needs a way in that does not depend on having the repository checked out. + */ + +const fs = require('fs'); +const path = require('path'); + +const REPO_ROOT = path.resolve(__dirname, '../..'); +const DOCS_DIR = path.join(REPO_ROOT, 'docs/docs'); +const SKILL_DIR = path.join(REPO_ROOT, 'skills/w3ds'); + +/** Frontmatter is a leading `---` block; return it parsed shallowly, plus the body. */ +function splitFrontmatter(raw) { + const match = raw.match(/^---\r?\n([\s\S]*?)\r?\n---\r?\n?/); + if (!match) return { data: {}, body: raw }; + const data = {}; + for (const line of match[1].split(/\r?\n/)) { + const kv = line.match(/^([A-Za-z_][A-Za-z0-9_]*):\s*(.*)$/); + if (kv) data[kv[1]] = kv[2].trim().replace(/^["']|["']$/g, ''); + } + return { data, body: raw.slice(match[0].length) }; +} + +/** First real prose paragraph, collapsed to one line — the fallback summary. */ +function firstParagraph(body) { + const lines = body.split(/\r?\n/); + const buf = []; + let inFence = false; + for (const line of lines) { + if (line.startsWith('```')) inFence = !inFence; + if (inFence) continue; + const t = line.trim(); + if (!t) { + if (buf.length) break; + continue; + } + if (t.startsWith('#') || t.startsWith(':::') || t.startsWith('|') || t.startsWith('<')) { + if (buf.length) break; + continue; + } + buf.push(t); + } + return ( + buf + .join(' ') + // Links and emphasis are noise in a one-line summary; inline code is not, + // so backticks stay. + .replace(/!?\[([^\]]*)\]\([^)]*\)/g, '$1') + .replace(/\*\*([^*]+)\*\*/g, '$1') + .replace(/(^|\s)_([^_]+)_(?=\s|$)/g, '$1$2') + .replace(/\s+/g, ' ') + .trim() + ); +} + +function truncate(s, n) { + if (s.length <= n) return s; + const cut = s.slice(0, n); + const lastSpace = cut.lastIndexOf(' '); + return `${cut.slice(0, lastSpace > n * 0.6 ? lastSpace : cut.length).trimEnd()}…`; +} + +/** Category label and position come from `_category_.json`, as in the sidebar. */ +function readCategory(dir, name) { + const file = path.join(dir, '_category_.json'); + if (!fs.existsSync(file)) return { label: name, position: Number.MAX_SAFE_INTEGER }; + try { + const meta = JSON.parse(fs.readFileSync(file, 'utf8')); + return { + label: meta.label || name, + position: typeof meta.position === 'number' ? meta.position : Number.MAX_SAFE_INTEGER, + }; + } catch { + return { label: name, position: Number.MAX_SAFE_INTEGER }; + } +} + +/** Every doc page, grouped by category, in sidebar order. */ +function collectDocs(baseUrl) { + const categories = []; + + const readPage = (absPath, relPath) => { + const raw = fs.readFileSync(absPath, 'utf8'); + const { data, body } = splitFrontmatter(raw); + const h1 = body.match(/^#\s+(.+)$/m); + const title = (data.title || (h1 && h1[1]) || path.basename(relPath, '.md')).trim(); + const slug = relPath.replace(/\.md$/, ''); + return { + title, + url: `${baseUrl}/docs/${slug.split('/').map(encodeURIComponent).join('/')}`, + description: truncate(data.description || firstParagraph(body), 200), + position: + data.sidebar_position !== undefined + ? Number(data.sidebar_position) + : Number.MAX_SAFE_INTEGER, + body: body.trim(), + }; + }; + + const rootPages = []; + for (const entry of fs.readdirSync(DOCS_DIR, { withFileTypes: true })) { + if (entry.isDirectory()) { + const dir = path.join(DOCS_DIR, entry.name); + const meta = readCategory(dir, entry.name); + const pages = fs + .readdirSync(dir) + .filter((f) => f.endsWith('.md')) + .map((f) => readPage(path.join(dir, f), `${entry.name}/${f}`)) + .sort((a, b) => a.position - b.position || a.title.localeCompare(b.title)); + if (pages.length) categories.push({ ...meta, pages }); + } else if (entry.name.endsWith('.md')) { + rootPages.push(readPage(path.join(DOCS_DIR, entry.name), entry.name)); + } + } + + categories.sort((a, b) => a.position - b.position || a.label.localeCompare(b.label)); + if (rootPages.length) { + categories.unshift({ label: 'Documentation', position: -1, pages: rootPages }); + } + return categories; +} + +/** SKILL.md first, then reference files alphabetically. */ +function collectSkillFiles() { + if (!fs.existsSync(path.join(SKILL_DIR, 'SKILL.md'))) { + throw new Error( + `[llms-txt] expected the W3DS skill at ${SKILL_DIR}. The docs site publishes it at /skill/; ` + + 'shipping an empty /skill/ would silently break every zero-install agent.' + ); + } + const files = [{ rel: 'SKILL.md', abs: path.join(SKILL_DIR, 'SKILL.md') }]; + const refDir = path.join(SKILL_DIR, 'reference'); + if (fs.existsSync(refDir)) { + for (const f of fs.readdirSync(refDir).filter((n) => n.endsWith('.md')).sort()) { + files.push({ rel: `reference/${f}`, abs: path.join(refDir, f) }); + } + } + return files; +} + +module.exports = function llmsTxtPlugin() { + return { + name: 'llms-txt', + + async postBuild({ outDir, siteConfig }) { + const baseUrl = String(siteConfig.url).replace(/\/$/, ''); + const categories = collectDocs(baseUrl); + const skillFiles = collectSkillFiles(); + + // --- /llms.txt: the index ------------------------------------------------- + const index = [ + `# ${siteConfig.title}`, + '', + `> ${siteConfig.tagline}. W3DS lets users own their data in a personal eVault while platforms act as interchangeable frontends. This site is the authoritative source for the protocol, its services, and how to build on it.`, + '', + 'The eVault is the source of truth; anything a platform stores is a projection of it. Read Data Ownership Rules before designing a platform.', + '', + ]; + for (const cat of categories) { + index.push(`## ${cat.label}`, ''); + for (const p of cat.pages) { + index.push(`- [${p.title}](${p.url})${p.description ? `: ${p.description}` : ''}`); + } + index.push(''); + } + index.push( + '## Agent skill', + '', + 'The packaged W3DS skill, for coding agents. Fetch directly, or install with `npx skills add MetaState-Prototype-Project/prototype@w3ds`.', + '' + ); + for (const f of skillFiles) { + index.push(`- [${f.rel}](${baseUrl}/skill/${f.rel})`); + } + index.push( + `- [w3ds-full.txt](${baseUrl}/skill/w3ds-full.txt): the whole skill in one file`, + '', + '## Optional', + '', + `- [llms-full.txt](${baseUrl}/llms-full.txt): every page on this site concatenated`, + '' + ); + fs.writeFileSync(path.join(outDir, 'llms.txt'), index.join('\n')); + + // --- /llms-full.txt: the corpus ------------------------------------------ + const full = [ + `# ${siteConfig.title}`, + '', + `Every page of ${baseUrl}, concatenated. Generated at build time.`, + '', + ]; + for (const cat of categories) { + for (const p of cat.pages) { + full.push('---', '', `# ${p.title}`, '', `Source: ${p.url}`, '', p.body, ''); + } + } + fs.writeFileSync(path.join(outDir, 'llms-full.txt'), full.join('\n')); + + // --- /skill/**: the skill, fetchable --------------------------------------- + const skillOut = path.join(outDir, 'skill'); + fs.mkdirSync(path.join(skillOut, 'reference'), { recursive: true }); + const concat = []; + for (const f of skillFiles) { + const raw = fs.readFileSync(f.abs, 'utf8'); + fs.writeFileSync(path.join(skillOut, f.rel), raw); + concat.push(`---\n\n# ${f.rel}\n\n${splitFrontmatter(raw).body.trim()}\n`); + } + fs.writeFileSync( + path.join(skillOut, 'w3ds-full.txt'), + [ + '# W3DS agent skill', + '', + `Source: ${baseUrl}/skill/SKILL.md. Authoritative docs: ${baseUrl}.`, + '', + ...concat, + ].join('\n') + ); + + const pageCount = categories.reduce((n, c) => n + c.pages.length, 0); + console.log( + `[llms-txt] wrote llms.txt (${pageCount} pages), llms-full.txt, and /skill/ (${skillFiles.length} files)` + ); + }, + }; +}; From d44714e5ac65452228286b5a9ada30c8a9c39cba Mon Sep 17 00:00:00 2001 From: coodos Date: Mon, 31 Aug 2026 21:40:38 +0800 Subject: [PATCH 3/4] skill: make the docs site authoritative and enforce eVault-first design --- README.md | 2 + .../Post Platform Guide/ai-agent-skill.md | 105 ++++++---- skills/README.md | 25 ++- skills/w3ds/SKILL.md | 113 ++++++++--- skills/w3ds/reference/dev-setup.md | 8 +- skills/w3ds/reference/evault.md | 26 +-- skills/w3ds/reference/identity.md | 22 +-- skills/w3ds/reference/platform.md | 52 +++-- skills/w3ds/reference/protocols.md | 34 ++-- skills/w3ds/reference/registry.md | 75 +++++--- skills/w3ds/reference/w3ds-native.md | 179 ++++++++++++++++++ skills/w3ds/reference/wallet.md | 14 +- 12 files changed, 497 insertions(+), 158 deletions(-) create mode 100644 skills/w3ds/reference/w3ds-native.md diff --git a/README.md b/README.md index 524efd9ec..21d2fa877 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,8 @@ **[Documentation](https://docs.w3ds.metastate.foundation)** — Getting started with W3DS and the MetaState prototype. +**Building with an AI coding agent?** Load the [W3DS agent skill](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/ai-agent-skill) — `npx skills add MetaState-Prototype-Project/prototype@w3ds`, or point a fetch-capable agent at [`/skill/SKILL.md`](https://docs.w3ds.metastate.foundation/skill/SKILL.md) and [`/llms.txt`](https://docs.w3ds.metastate.foundation/llms.txt). + **Quick start (registry + evault-core + dev-sandbox):** see **[QUICKSTART.md](QUICKSTART.md)** — one script to run Postgres + Neo4j in Docker and the core services locally. Dev sandbox runs at **http://localhost:8080**. ## Docker Development Environment diff --git a/docs/docs/Post Platform Guide/ai-agent-skill.md b/docs/docs/Post Platform Guide/ai-agent-skill.md index 1fff42614..580ad38f8 100644 --- a/docs/docs/Post Platform Guide/ai-agent-skill.md +++ b/docs/docs/Post Platform Guide/ai-agent-skill.md @@ -6,6 +6,29 @@ sidebar_position: 8 This repo ships a packaged **W3DS knowledge skill** under `skills/w3ds/` that you can load into your AI coding assistant so it stops guessing ontology UUIDs, mapping directives, and GraphQL field names. It's grounded in the docs you're reading now. +## Zero install + +If your agent can fetch a URL, it needs nothing installed. Point it at: + +```text +https://docs.w3ds.metastate.foundation/skill/SKILL.md +``` + +Or hand it the whole skill in one file: + +```text +https://docs.w3ds.metastate.foundation/skill/w3ds-full.txt +``` + +Two companions are published alongside it, and an agent that can fetch should know about both: + +| File | What it is | +| --- | --- | +| [`/llms.txt`](https://docs.w3ds.metastate.foundation/llms.txt) | An index of every page on this site, with URLs and one-line summaries. The cheapest way for an agent to find the authoritative page for a question. | +| [`/llms-full.txt`](https://docs.w3ds.metastate.foundation/llms-full.txt) | The whole documentation corpus in a single file, for agents that would rather read everything once. | + +These are regenerated on every docs deploy, so a fetch is always current. Installing is still worth it for agents that support skills — the skill then loads automatically on the right questions, instead of only when someone remembers to paste a URL. + The easiest install for every supported agent is the [`npx skills`](https://skills.sh) CLI — it targets Claude Code, Codex, Cursor, GitHub Copilot, Windsurf, OpenCode, Cline, Gemini, and 60+ others. Manual per-tool instructions are further down if you'd rather bypass the CLI or your agent isn't supported yet. :::note Windows users @@ -18,9 +41,21 @@ Command blocks are labeled **macOS / Linux (bash)** and **Windows (PowerShell)** ::: +## What the skill enforces + +The skill is not only a reference. It changes how an agent behaves on W3DS work: + +- **The eVault is the source of truth.** The platform database is a projection of it. The skill applies the reconstructability test — *if this database were dropped and rebuilt by replaying the relevant eVaults, what would be lost?* — before agreeing to persist anything new. See [Data Ownership Rules](/docs/W3DS%20Basics/Data-Ownership-Rules). +- **Resolve, never recall.** Ontology IDs, endpoints, GraphQL field names and ACL verbs are looked up at the time of use. The skill deliberately contains no ontology UUIDs, so there is nothing stale to copy. Where it cannot verify something — no fetch tool, or the service is unreachable — it says so and marks the spot in code rather than substituting a plausible value. +- **Two hard stops.** The agent stops and asks, rather than writing code, when a design would make the local database authoritative for user data, or when a persisted entity type has no ontology. The second is a path rather than a wall: ontologies are ordinary JSON files, and the agent will draft the schema and offer to open the PR. See [Proposing a new ontology](/docs/Infrastructure/Ontology#proposing-a-new-ontology). +- **A definition of done.** `X-ENAME` on every call, `handleChange` on every write path, an idempotent webhook controller, no invented identifiers. + +If you want an agent that produces a conventional application with sync bolted on, do not install this skill. That is the outcome it exists to prevent. + ## What's in the skill -- `SKILL.md` — router and ecosystem map +- `SKILL.md` — router, authority rules, pre-flight gate, stop rules, definition of done +- `reference/w3ds-native.md` — where data lives: the reconstructability test, anti-patterns, proposing an ontology - `reference/evault.md` — GraphQL API, ACLs, `/whois`, `/logs` - `reference/identity.md` — W3ID, eName, Binding Documents - `reference/registry.md` — Registry endpoints, canonical ontology UUIDs @@ -29,6 +64,8 @@ Command blocks are labeled **macOS / Linux (bash)** and **Windows (PowerShell)** - `reference/wallet.md` — eID Wallet, wallet-sdk, key delegation - `reference/dev-setup.md` — `pnpm dev:core` + debugging playbook +Everything in it cites this site by URL, so an agent that gets stuck has somewhere authoritative to go. Where the skill and these docs disagree, the docs win. + ## Install with `npx skills` (all tools) The [skills CLI](https://skills.sh) auto-detects the AI coding agents you have installed and configures each of them. Works cross-platform (macOS / Linux / Windows PowerShell / WSL). @@ -132,49 +169,46 @@ Codex CLI reads `AGENTS.md` from the repo root and `~/.codex/AGENTS.md` for user ### Project-scoped -Copy the skill content into `AGENTS.md` at the root of the project you're building on W3DS. +Write the published skill into `AGENTS.md` at the root of the project you're building on W3DS. No clone needed. **macOS / Linux (bash):** ```bash -cat skills/w3ds/SKILL.md > AGENTS.md -echo -e "\n\n---\n" >> AGENTS.md -for f in skills/w3ds/reference/*.md; do - echo -e "\n## $(basename "$f" .md)\n" >> AGENTS.md - cat "$f" >> AGENTS.md -done +curl -fsSL https://docs.w3ds.metastate.foundation/skill/w3ds-full.txt > AGENTS.md ``` **Windows (PowerShell):** ```powershell -Get-Content skills/w3ds/SKILL.md | Set-Content AGENTS.md -Add-Content AGENTS.md "`n`n---`n" -Get-ChildItem skills/w3ds/reference/*.md | ForEach-Object { - Add-Content AGENTS.md "`n## $($_.BaseName)`n" - Get-Content $_.FullName | Add-Content AGENTS.md -} +Invoke-WebRequest https://docs.w3ds.metastate.foundation/skill/w3ds-full.txt -OutFile AGENTS.md ``` -Or, if `AGENTS.md` already exists, append the skill as a section. +If `AGENTS.md` already exists, append instead of overwriting: **macOS / Linux (bash):** ```bash -echo -e "\n\n# W3DS reference\n" >> AGENTS.md -cat skills/w3ds/SKILL.md skills/w3ds/reference/*.md >> AGENTS.md +printf '\n\n# W3DS reference\n\n' >> AGENTS.md +curl -fsSL https://docs.w3ds.metastate.foundation/skill/w3ds-full.txt >> AGENTS.md ``` **Windows (PowerShell):** ```powershell Add-Content AGENTS.md "`n`n# W3DS reference`n" -Get-Content skills/w3ds/SKILL.md, skills/w3ds/reference/*.md | Add-Content AGENTS.md +(Invoke-WebRequest https://docs.w3ds.metastate.foundation/skill/w3ds-full.txt).Content | + Add-Content AGENTS.md +``` + +Working from a metastate clone instead? Concatenate the local files: + +```bash +cat skills/w3ds/SKILL.md skills/w3ds/reference/*.md > AGENTS.md ``` ### User-scoped -Put the same concatenated content in `~/.codex/AGENTS.md` if you want it available in every project you touch. +Put the same content in `~/.codex/AGENTS.md` if you want it available in every project you touch. ## Cursor (manual) @@ -264,17 +298,20 @@ Copilot reads `.github/copilot-instructions.md` for repo-level guidance. ```bash mkdir -p .github -cat skills/w3ds/SKILL.md skills/w3ds/reference/*.md > .github/copilot-instructions.md +curl -fsSL https://docs.w3ds.metastate.foundation/skill/w3ds-full.txt \ + > .github/copilot-instructions.md ``` **Windows (PowerShell):** ```powershell New-Item -ItemType Directory -Force -Path .github | Out-Null -Get-Content skills/w3ds/SKILL.md, skills/w3ds/reference/*.md | - Set-Content .github/copilot-instructions.md +Invoke-WebRequest https://docs.w3ds.metastate.foundation/skill/w3ds-full.txt ` + -OutFile .github/copilot-instructions.md ``` +Copilot has no fetch tool of its own, so this copy is all it will ever see. Re-run the command when the docs change, and expect the skill to flag identifiers it could not verify rather than resolving them itself. + Commit the file. Copilot picks it up automatically for repositories that have it enabled in settings (Copilot → Chat → *Instructions*). ## Windsurf (manual) @@ -286,13 +323,14 @@ Windsurf reads `.windsurfrules` at the repo root. **macOS / Linux (bash):** ```bash -cat skills/w3ds/SKILL.md skills/w3ds/reference/*.md > .windsurfrules +curl -fsSL https://docs.w3ds.metastate.foundation/skill/w3ds-full.txt > .windsurfrules ``` **Windows (PowerShell):** ```powershell -Get-Content skills/w3ds/SKILL.md, skills/w3ds/reference/*.md | Set-Content .windsurfrules +Invoke-WebRequest https://docs.w3ds.metastate.foundation/skill/w3ds-full.txt ` + -OutFile .windsurfrules ``` For user-level rules, put the same content in: @@ -319,18 +357,19 @@ aider --read CONVENTIONS.md Cline, Roo, Continue.dev, Gemini, Zed, Goose, Kilo, and dozens more are all supported by `npx skills` — try `-a ` from the [main install section](#install-with-npx-skills-all-tools) first. If your agent isn't supported yet or you want to bypass the CLI, use this universal pattern: -1. Concatenate the skill into one markdown file. +1. Download the skill as one markdown file. **macOS / Linux (bash):** ```bash - cat skills/w3ds/SKILL.md skills/w3ds/reference/*.md > w3ds-context.md + curl -fsSL https://docs.w3ds.metastate.foundation/skill/w3ds-full.txt > w3ds-context.md ``` **Windows (PowerShell):** ```powershell - Get-Content skills/w3ds/SKILL.md, skills/w3ds/reference/*.md | Set-Content w3ds-context.md + Invoke-WebRequest https://docs.w3ds.metastate.foundation/skill/w3ds-full.txt ` + -OutFile w3ds-context.md ``` 2. Add `w3ds-context.md` to whatever the agent uses for repo-level context: @@ -343,9 +382,9 @@ Cline, Roo, Continue.dev, Gemini, Zed, Goose, Kilo, and dozens more are all supp If your tool isn't listed above, the pattern is always the same: -1. Concatenate `skills/w3ds/SKILL.md` and `skills/w3ds/reference/*.md` into whatever file the tool reads for repo instructions. +1. Put `https://docs.w3ds.metastate.foundation/skill/w3ds-full.txt` into whatever file the tool reads for repo instructions. 2. If the tool supports rule-file frontmatter (Cursor, some others), keep it descriptive so the tool knows when to activate the rule. -3. If the tool has no rule system at all, point it at the skill in your prompt: *"Use `skills/w3ds/` in this repo as authoritative W3DS reference before answering."* +3. If the tool has no rule system at all, point it at the URL in your prompt: *"Read https://docs.w3ds.metastate.foundation/skill/SKILL.md and treat https://docs.w3ds.metastate.foundation as the authoritative source before answering."* ## Updating @@ -353,7 +392,7 @@ The skill mirrors the docs. When docs change, pull the latest metastate `main` a - **`npx skills` install (any agent):** `npx skills update` — updates every installed skill across every agent. - **Symlink install (Claude Code):** nothing — edits take effect immediately. -- **Manual copy install (Cursor, Copilot, Windsurf, Codex, Aider):** re-run the concatenation command from the relevant section above. +- **Manual copy install (Cursor, Copilot, Windsurf, Codex, Aider):** re-run the download command from the relevant section above. `/skill/w3ds-full.txt` is rebuilt on every docs deploy, so a re-fetch is always current. If you're building on a fork and shipping the manual copy, add a repo hook or pre-commit step that re-runs the concatenation so the copy in your project stays fresh. @@ -361,9 +400,11 @@ If you're building on a fork and shipping the manual copy, add a repo hook or pr Gaps or wrong answers? PRs welcome. The skill lives at `skills/w3ds/` in this repo. Rules of thumb: -- Ground every claim in a `docs/docs/...` path. +- Ground every claim in a `https://docs.w3ds.metastate.foundation/docs/...` URL. The skill is installed outside this repo far more often than inside it, so a repo-relative path is a dead end for most readers. +- **No ontology UUIDs in the skill.** They go stale, and an agent will copy one rather than resolve it. Teach the lookup instead. - Keep the main `SKILL.md` scannable (under ~200 lines); push detail into `reference/*.md`. -- Don't invent APIs. If the docs don't say it, don't put it in the skill. +- Don't invent APIs. If the docs don't say it, don't put it in the skill — add it to the docs first. +- If a change alters what the agent *does* rather than what it knows, say so in [What the skill enforces](#what-the-skill-enforces). ## Reference diff --git a/skills/README.md b/skills/README.md index ef3fe263d..7618282e8 100644 --- a/skills/README.md +++ b/skills/README.md @@ -2,11 +2,27 @@ Installable AI-agent skills for the MetaState / W3DS ecosystem. Powered by the [skills.sh](https://skills.sh) CLI — works with Claude Code, Codex, Cursor, GitHub Copilot, Windsurf, OpenCode, Cline, Gemini, and 60+ other coding agents. +**[docs.w3ds.metastate.foundation](https://docs.w3ds.metastate.foundation) is authoritative.** These skills are a condensed index of it and can lag behind it — where a skill and the docs disagree, the docs win. Every citation in the skill is a live URL for that reason, and the skill contains no ontology UUIDs: they are resolved from the Ontology service at the time of use. + ## Available skills | Skill | Purpose | |-------|---------| -| [w3ds](./w3ds) | Web 3 Data Spaces — build post-platforms, call the eVault GraphQL API, wire the Web3 Adapter, implement `w3ds://auth` / `w3ds://sign`, debug local dev. | +| [w3ds](./w3ds) | Web 3 Data Spaces — build post-platforms, call the eVault GraphQL API, wire the Web3 Adapter, implement `w3ds://auth` / `w3ds://sign`, debug local dev. Enforces eVault-first design: the eVault is the source of truth, the platform DB is a projection. | + +## Use it without installing anything + +Every skill file is published on the docs site, rebuilt on each deploy: + +| URL | What | +|-----|------| +| [`/skill/SKILL.md`](https://docs.w3ds.metastate.foundation/skill/SKILL.md) | The skill router | +| [`/skill/reference/`](https://docs.w3ds.metastate.foundation/skill/reference/platform.md) | Reference files, e.g. `platform.md` | +| [`/skill/w3ds-full.txt`](https://docs.w3ds.metastate.foundation/skill/w3ds-full.txt) | The whole skill in one file | +| [`/llms.txt`](https://docs.w3ds.metastate.foundation/llms.txt) | Index of every docs page, with URLs | +| [`/llms-full.txt`](https://docs.w3ds.metastate.foundation/llms-full.txt) | The whole docs corpus in one file | + +Any agent that can fetch a URL can self-serve. Installing is still better where the agent supports skills — it then loads on the right questions rather than when someone remembers to paste a link. ## Install @@ -46,7 +62,7 @@ npx skills use MetaState-Prototype-Project/prototype@w3ds | claude npx skills use MetaState-Prototype-Project/prototype@w3ds --agent cursor ``` -Full per-tool install guide (manual paths for agents not yet covered by the CLI, or if you'd rather bypass it) lives at [docs/Post Platform Guide/AI Agent Skill](../docs/docs/Post%20Platform%20Guide/ai-agent-skill.md). +Full per-tool install guide (manual paths for agents not yet covered by the CLI, or if you'd rather bypass it) lives at [AI Agent Skill](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/ai-agent-skill). ## Local development @@ -73,3 +89,8 @@ Edits to files under `skills/w3ds/` take effect on the next skill invocation — Each skill is a directory with a top-level `SKILL.md` and optional `reference/` files. The `SKILL.md` frontmatter needs at minimum a `name` and a `description`; the description is what the agent uses to decide when to trigger the skill, so list the concrete surfaces it covers (concepts, APIs, protocol names, common questions). Keep the main `SKILL.md` scannable (~150 lines) and push deep content into `reference/*.md` files that get loaded on demand. + +Two rules specific to these skills: + +- **Cite live URLs, never repo paths.** A skill is installed outside this repo far more often than inside it, so `docs/docs/...` is a dead end for most readers. +- **No ontology UUIDs.** They go stale, and an agent will copy one rather than resolve it. Teach the `GET /schemas` lookup instead. diff --git a/skills/w3ds/SKILL.md b/skills/w3ds/SKILL.md index ce3d5ee5d..0563a9ca8 100644 --- a/skills/w3ds/SKILL.md +++ b/skills/w3ds/SKILL.md @@ -1,6 +1,6 @@ --- name: w3ds -description: "Use when the user is building on Web 3 Data Spaces (W3DS) or the MetaState prototype — building a post-platform, integrating an eVault, calling the eVault GraphQL API (createMetaEnvelope, updateMetaEnvelope, removeMetaEnvelope, bulkCreateMetaEnvelopes, uploadFile, bindingDocument*), wiring the Web3 Adapter, writing a webhook controller for /api/webhook, authoring mapping.json files, using the wallet-sdk, implementing the w3ds://auth or w3ds://sign flow, resolving W3IDs / eNames via the Registry, working with the Ontology service, dealing with Binding Documents, dereferencing w3ds://file URIs, provisioning an eVault, syncing public keys, or debugging local dev (Registry, Provisioner, eVault-core, Dev Sandbox, pnpm dev:core). Also use when the user asks what an eVault, W3ID, eName, MetaEnvelope, Envelope, Ontology, Web3 Adapter, Awareness Protocol, or Awareness-as-a-Service is." +description: "Use when the user is building on Web 3 Data Spaces (W3DS) or the MetaState prototype — building a post-platform, integrating an eVault, calling the eVault GraphQL API (createMetaEnvelope, updateMetaEnvelope, removeMetaEnvelope, bulkCreateMetaEnvelopes, uploadFile, bindingDocument*), wiring the Web3 Adapter, writing a webhook controller for /api/webhook, authoring mapping.json files, using the wallet-sdk, implementing the w3ds://auth or w3ds://sign flow, resolving W3IDs / eNames via the Registry, working with the Ontology service, proposing a new ontology, dealing with Binding Documents, dereferencing w3ds://file URIs, provisioning an eVault, syncing public keys, or debugging local dev (Registry, Provisioner, eVault-core, Dev Sandbox, pnpm dev:core). Also use for any design decision about where data lives on a W3DS platform — adding a table, entity, model or field, deciding what to cache locally, or asking whether something belongs in the database or the eVault. Also use when the user asks what an eVault, W3ID, eName, MetaEnvelope, Envelope, Ontology, Web3 Adapter, Awareness Protocol, or Awareness-as-a-Service is." license: Apache 2.0 --- @@ -8,19 +8,84 @@ license: Apache 2.0 W3DS lets users own their data in a personal **eVault** while platforms act as interchangeable frontends. Data written on one platform automatically syncs to every other registered platform via the **Awareness Protocol**. This skill is for developers building on W3DS: integrating platforms, calling the eVault GraphQL API, wiring the Web3 Adapter, and debugging local dev. +## Authority + +**`https://docs.w3ds.metastate.foundation` is the authoritative source.** This skill is a condensed index of it and can lag behind it. **Where this skill and the docs disagree, the docs win** — say so and follow the docs. + +Fetch the docs whenever you are uncertain, and whenever the answer would otherwise come from memory. Every citation in this skill is a live URL, so there is nothing to resolve first. + +| Need | Fetch | +|---|---| +| Machine-readable index of every page | `https://docs.w3ds.metastate.foundation/llms.txt` | +| Whole documentation corpus in one file | `https://docs.w3ds.metastate.foundation/llms-full.txt` | +| Latest version of this skill | `https://docs.w3ds.metastate.foundation/skill/SKILL.md` | +| Every ontology and its `schemaId` | `https://ontology.w3ds.metastate.foundation/schemas` | +| Domains, and the schemas under one | `https://ontology.w3ds.metastate.foundation/domains` | + +Working inside the `MetaState-Prototype-Project/prototype` checkout? `docs/docs/**` mirrors the site — a convenience, not a second authority. + +## Non-negotiables + +Read [Data Ownership Rules](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/Data-Ownership-Rules) before designing anything. In short: + +1. **The eVault is the source of truth. Anything a platform stores is a projection of it.** The docs call platforms "caches and aggregators" — that is permission to keep a fast local copy of data that is authoritative elsewhere, not permission to own it. +2. **Every persisted entity needs three things:** an ontology (`schemaId` resolved, never invented), a resolvable owner (`ownerEnamePath` that resolves for *every* row, to the data subject — not to the platform), and a named write path to that owner's eVault. +3. **Resolve, never recall.** Ontology IDs, endpoints, GraphQL field names, ACL verbs and eNames are looked up, not remembered. This skill deliberately contains no ontology UUIDs. + +A local database is not a violation. A local database that is the only place some user data exists is. + +## Pre-flight — before writing any W3DS code + +Answer all four. If you cannot, that is the finding — report it instead of writing code around it. + +1. **Which ontology?** `GET https://ontology.w3ds.metastate.foundation/schemas`, match on `title`, then confirm the field names with `GET /schemas/:id`. Narrow by subject area first with `GET /domains/:id/schemas`. No match → [Proposing a new ontology](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Ontology#proposing-a-new-ontology). **Never invent a `schemaId`** — it fails silently: the write succeeds, the packet fans out, and every receiving platform drops it for having no matching mapping. +2. **Whose eVault owns it?** Name the `ownerEnamePath` and check it resolves for every row, not the happy path. The owner is the person or group the data is *about*. If it resolves to nothing, `handleChange` returns silently and the entity never syncs. +3. **Truth or projection?** Apply the reconstructability test: *if this database were dropped and rebuilt by replaying the relevant eVaults, what would be lost?* "Nothing that matters" — projection, correct. Anything a user would miss — that data has no home but yours. +4. **What writes it to the eVault?** Point at the call site: the `handleChange` after the local write, or the direct eVault write for a stateless app. "Sync comes later" means the platform owns the data today. + +## Stop rules + +Stop, state the problem in the user's own terms, propose the eVault-first alternative, and **ask before writing code** when: + +- **The design makes the local database authoritative for user data.** A table with no mapping; a field with no ontology counterpart; an entity written locally with no path to any eVault; reads that prefer local rows over eVault-derived state while the eVault is reachable. +- **A persisted entity type has no ontology** and none can be resolved from the Ontology service. + +The second is a path, not a wall. Ontologies are ordinary JSON files that anyone can propose: draft the draft-07 schema (fresh random UUIDv4 `schemaId`, a `domain` from `GET /domains`, a `description` per property, `additionalProperties: false`), offer to open the PR against `services/ontology/schemas/`, and say clearly that until it merges the type does not exist. Full procedure in [reference/w3ds-native.md](reference/w3ds-native.md). + +These two stops only. Everything else — a missing owner path, an unfamiliar directive, a service that is down — is a normal problem to solve and report, not a reason to halt. + +## When you cannot verify + +No fetch tool, offline, or the fetch failed? **Proceed, but never present an unverified identifier as confirmed.** + +- Name every unverified item in your response, with the exact URL that would settle it. +- Mark it at the call site: `// TODO(w3ds): unverified — confirm against https://ontology.w3ds.metastate.foundation/schemas`. +- Never fill the gap with a plausible-looking UUID, endpoint or field name. A wrong identifier is worse than an obvious placeholder, because it fails silently. + +## Definition of done + +Before reporting a W3DS task complete, check every line: + +- [ ] Every eVault GraphQL and HTTP call sends `X-ENAME: @`. +- [ ] Every eVault URL came from the Registry at call time — none hardcoded. +- [ ] Every `schemaId` was resolved from the Ontology service in this session, not recalled. +- [ ] Every new entity type has an ontology, a resolving `ownerEnamePath`, and a write path to the owner's eVault. +- [ ] `handleChange` is called after every write to a mapped table — including writes from migrations, seeds, admin paths and background jobs. +- [ ] The webhook controller is idempotent on the global `id`, and returns 200 for ontologies the platform does not consume. +- [ ] Nothing was invented: no UUID, endpoint path, GraphQL field, mapping directive or ACL verb that was not verified — or, if unverifiable, each is flagged in the response and marked in code. +- [ ] The reconstructability test was applied to anything newly persisted, and the answer stated. + ## Ecosystem map The "digital self" is a triad: **eName + eID certificate + eVault**. Users hold keys in the **eID Wallet**. The **Provisioner** creates their eVault. The **Registry** resolves W3IDs to eVault URLs and hosts the platform directory. The **Ontology** service publishes JSON Schemas that platforms map their local schemas to. A **Web3 Adapter** on each platform bridges the local DB to the owner's eVault. When data changes, eVault fires the **Awareness Protocol** to notify every other registered platform. -## Components - | Component | One line | Load this reference | |---|---|---| | **eVault** | GraphQL data store per W3ID, Neo4j-backed, delivers webhooks on writes | [reference/evault.md](reference/evault.md) | | **W3ID / eName** | UUID-based persistent identifier; eName = W3ID registered in Registry | [reference/identity.md](reference/identity.md) | | **Binding Document** | Signed MetaEnvelope tying a user to an eName (id_document, photograph, social_connection, self) | [reference/identity.md](reference/identity.md) | | **Registry** | W3ID resolution, `/entropy` for provisioning, JWKS, platform list, key-binding certs (temporary) | [reference/registry.md](reference/registry.md) | -| **Ontology** | JSON Schema draft-07 registry served at `/schemas` and `/schemas/:id` | [reference/registry.md](reference/registry.md) | +| **Ontology** | JSON Schema draft-07 registry served at `/schemas`, `/schemas/:id`, `/domains` | [reference/registry.md](reference/registry.md) | | **Provisioner** | Creates new eVaults; exposes `POST /provision` | [reference/wallet.md](reference/wallet.md) | | **eID Wallet** | Mobile app (Tauri/SvelteKit); holds ECDSA P-256 keys in Secure Enclave / HSM | [reference/wallet.md](reference/wallet.md) | | **wallet-sdk** | TypeScript SDK: `provision`, `authenticate`, `syncPublicKeyToEvault`; crypto-agnostic via `CryptoAdapter` | [reference/wallet.md](reference/wallet.md) | @@ -35,57 +100,59 @@ The "digital self" is a triad: **eName + eID certificate + eVault**. Users hold | Service | URL | |---|---| +| Docs (authoritative) | `https://docs.w3ds.metastate.foundation` | | Provisioner | `https://provisioner.w3ds.metastate.foundation` | | Registry | `https://registry.w3ds.metastate.foundation` | | Ontology | `https://ontology.w3ds.metastate.foundation` | +| GitW3 (W3DS-aware Git forge) | `https://git.w3ds.metastate.foundation` | -Source: `docs/docs/W3DS Basics/Links.md`. +Source: [Links](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/Links). ## Routing rules -When answering a user's question, load the reference file(s) below **before** writing any code or configuration. Do not fabricate ontology IDs, GraphQL field names, mapping directives, or endpoint paths from memory — grep `docs/docs/` if a reference file doesn't answer the question. +Load the reference file(s) below **before** writing any code or configuration. | User question mentions... | Load | |---|---| +| "should this live in the database", "add a table / entity / model / field", "how do I model X", caching, local copies, data ownership, "is this W3DS-native", a design or architecture review | [reference/w3ds-native.md](reference/w3ds-native.md) | | webhook controller, mapping.json, `handleChange`, `fromGlobal`, `toGlobal`, Web3 Adapter, `ownerEnamePath`, `__date`, `__calc`, `__file`, "how do I build a platform" | [reference/platform.md](reference/platform.md) | | GraphQL, `createMetaEnvelope`, `updateMetaEnvelope`, `removeMetaEnvelope`, `bulkCreateMetaEnvelopes`, `uploadFile`, `metaEnvelope(id)`, `metaEnvelopes`, ACL, `X-ENAME`, `/whois`, `/logs`, MetaEnvelope, Envelope, Neo4j model | [reference/evault.md](reference/evault.md) | | W3ID, eName, `@` format, X-ENAME header, Binding Document, id_document, photograph, social_connection, self, key rotation, friend-based recovery | [reference/identity.md](reference/identity.md) | -| Registry, `/resolve`, `/entropy`, `/list`, JWKS, key binding certificate, ontology ID for User / Post / Group / Ledger / Currency / Account / Binding / File, `/schemas` | [reference/registry.md](reference/registry.md) | +| Registry, `/resolve`, `/entropy`, `/list`, JWKS, key binding certificate, finding an ontology ID, proposing a new ontology, `/schemas`, `/domains` | [reference/registry.md](reference/registry.md) | | w3ds://auth, w3ds://sign, Awareness Protocol packet, signature verification, ECDSA P-256, multibase, base58btc, base64 signature format, `verifySignature`, AaaS, w3ds://file URI, dereferencing files | [reference/protocols.md](reference/protocols.md) | | eID Wallet, wallet-sdk, `provision`, `authenticate`, `syncPublicKeyToEvault`, `CryptoAdapter`, hardware vs software keys, `PATCH /public-key`, key delegation across devices | [reference/wallet.md](reference/wallet.md) | | `pnpm dev:core`, Dev Sandbox, ports (4321 / 3001 / 4000 / 8080), `REGISTRY_ENTROPY_KEY_JWK`, `pnpm generate-entropy-jwk`, "webhook not firing", "signature verification fails", "duplicate entities" | [reference/dev-setup.md](reference/dev-setup.md) | -If the question spans multiple topics (common for platform builds), load the two or three most relevant references in one turn rather than piecemeal. +If the question spans multiple topics (common for platform builds), load the two or three most relevant references in one turn rather than piecemeal. A build task almost always needs `w3ds-native.md` plus `platform.md`. ## Do not guess -The docs are ground truth. Any of these values, if guessed, is almost certainly wrong: +Any of these values, if guessed, is almost certainly wrong: -- **Ontology UUIDs** — memorized table lives in [reference/registry.md](reference/registry.md). `w3ds-file-v1` is a **string literal**, not a UUID. -- **GraphQL field / mutation names** — `createMetaEnvelope` is the idiomatic name; `storeMetaEnvelope` is a legacy alias still used internally by the Web3 Adapter's `EVaultClient`. Full signatures live in [reference/evault.md](reference/evault.md). +- **Ontology `schemaId`s** — resolve from `https://ontology.w3ds.metastate.foundation/schemas`. This skill contains none by design. `w3ds-file-v1` is the one exception: a **protocol string literal**, not a registry lookup and not a UUID. +- **GraphQL field / mutation names** — `createMetaEnvelope` is the idiomatic name; `storeMetaEnvelope` is a legacy alias still used internally by the Web3 Adapter's `EVaultClient`. Full signatures in [reference/evault.md](reference/evault.md). - **Mapping directive syntax** — `__date(...)`, `__calc(...)`, `__file(...)`, `tableName(path),globalAlias`, and array `users(participants[].id),participantIds` — verbatim examples in [reference/platform.md](reference/platform.md). - **Signature encoding** — software keys emit base64 raw 64-byte (r || s); hardware keys emit multibase base58btc (`z...`). See [reference/protocols.md](reference/protocols.md). -- **Endpoint paths and headers** — every eVault request needs `X-ENAME: @`. The `/provision` endpoint lives on the Provisioner, not eVault-core (though in local dev they run in the same eVault-core process on port 3001). +- **Endpoint paths and headers** — every eVault request needs `X-ENAME: @`. `/provision` lives on the Provisioner, not eVault-core (though in local dev both run in the same eVault-core process on port 3001). -If uncertain, `grep -r docs/docs/` before writing the answer. +Uncertain? Fetch the relevant page from `https://docs.w3ds.metastate.foundation` — or, if you cannot, follow [When you cannot verify](#when-you-cannot-verify). ## Terminology anchors -Common confusion points — internalize these once: - - **MetaEnvelope vs Envelope**: MetaEnvelope is the top-level entity (one post, one user). Envelope is a single field of that entity, stored as its own Neo4j node linked via `LINKS_TO`. - **W3ID vs eName**: All eNames are W3IDs. Only W3IDs registered in the Registry are eNames (resolvable). Both use the `@` format when global. -- **Ontology vs schema**: "Ontology" in this ecosystem refers to a specific JSON Schema published by the Ontology service and referenced by its schemaId (a W3ID). Do not confuse with generic "ontology" from semantic web. -- **Platform vs post-platform**: A platform participates in W3DS via a Web3 Adapter and a `/api/webhook` endpoint. A post-platform is a platform that operates in "dataless" mode — it doesn't own the data, users' eVaults do. -- **`w3ds-file-v1` vs `File` ontology**: `w3ds-file-v1` is the low-level storage envelope created by `uploadFile` for blob dereferencing. The `File` ontology (`a1b2c3d4-e5f6-7890-abcd-ef1234567890`) is a higher-level platform record for file-manager / esigner style apps. They are not interchangeable — different field names, different layer. Detail in [reference/protocols.md](reference/protocols.md). -- **Awareness Protocol vs AaaS**: Awareness Protocol is the prototype-level fire-and-forget fanout from eVault-core. Awareness-as-a-Service (AaaS) is the production-grade replacement with subscriptions, persistence, retries, and a dead-letter queue. -- **`storeMetaEnvelope` / `updateMetaEnvelopeById`**: Legacy GraphQL mutation names, still used internally by the Web3 Adapter's `EVaultClient`. External integrations should use `createMetaEnvelope` / `updateMetaEnvelope` / `removeMetaEnvelope` instead. +- **Ontology vs schema**: "Ontology" here means a specific JSON Schema published by the Ontology service and referenced by its `schemaId` (a W3ID). Not the semantic-web sense of the word. +- **Platform vs post-platform**: A platform participates in W3DS via a Web3 Adapter and a `/api/webhook` endpoint. A post-platform operates in "dataless" mode — it doesn't own the data, users' eVaults do. +- **`w3ds-file-v1` vs `File` ontology**: `w3ds-file-v1` is the low-level storage envelope created by `uploadFile` for blob dereferencing. The `File` ontology is a higher-level platform record for file-manager / esigner style apps. Not interchangeable — different field names, different layer. Detail in [reference/protocols.md](reference/protocols.md). +- **Awareness Protocol vs AaaS**: Awareness Protocol is the prototype-level fire-and-forget fanout from eVault-core. AaaS is the production-grade replacement with subscriptions, persistence, retries, and a dead-letter queue. +- **`storeMetaEnvelope` / `updateMetaEnvelopeById`**: Legacy GraphQL mutation names, still used internally by the Web3 Adapter's `EVaultClient`. External integrations should use `createMetaEnvelope` / `updateMetaEnvelope` / `removeMetaEnvelope`. ## Working style -- Always resolve the eVault URL for a user via the Registry before hitting `/graphql` or `/whois`. Do not hardcode eVault URLs. +- Always resolve the eVault URL for a user via the Registry before hitting `/graphql` or `/whois`. Never hardcode eVault URLs; cache the resolution, revalidate it, and evict on a failed `HEAD /whois`. - Every GraphQL and HTTP call to eVault needs `X-ENAME`. Missing this header is the most common cause of 400s. - Two ACL models coexist. The `_acl` block gives per-verb grants (READ/CREATE/UPDATE/DELETE bitmask), denials, and ontology conditions, and is authoritative where present. The legacy `acl` string array is all-or-nothing except `["*"]` and still applies to records with no `_acl`. Do not describe ACLs as all-or-nothing without that distinction — see [reference/evault.md](reference/evault.md). -- Webhook delivery is fire-and-forget and prototype-level: no retries, no ordering, no at-least-once. Design your platform's webhook controller to be **idempotent** on global `id`. +- Webhook delivery is fire-and-forget and prototype-level: no retries, no ordering, no at-least-once. Make the webhook controller **idempotent** on global `id`. - After `storeMetaEnvelope` there is a 3-second delay before webhook fanout to prevent ping-pong. `updateMetaEnvelopeById` fanout is immediate. -- If the user is running things locally, refer them to [reference/dev-setup.md](reference/dev-setup.md) before troubleshooting — most sync bugs come from a service that isn't running or a missing env var. +- Do not mirror what you can already observe. If a record reaches you through the Awareness Protocol, subscribe to it rather than writing a second envelope to make it visible. +- If the user is running things locally, check [reference/dev-setup.md](reference/dev-setup.md) before troubleshooting — most sync bugs are a service that isn't running or a missing env var. diff --git a/skills/w3ds/reference/dev-setup.md b/skills/w3ds/reference/dev-setup.md index d63b795ea..7e703e36f 100644 --- a/skills/w3ds/reference/dev-setup.md +++ b/skills/w3ds/reference/dev-setup.md @@ -1,6 +1,6 @@ # Local dev + debugging -One command spins up the full W3DS core stack. Most sync bugs come from a service that isn't running or a missing env var — always verify the stack is healthy before hunting deeper. Source: `docs/docs/Post Platform Guide/local-dev-quick-start.md`, `dev-sandbox.md`. +One command spins up the full W3DS core stack. Most sync bugs come from a service that isn't running or a missing env var — always verify the stack is healthy before hunting deeper. Source: [Local Dev Quick Start](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/local-dev-quick-start) and [Using the Dev Sandbox](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/dev-sandbox). ## Prerequisites @@ -84,7 +84,7 @@ Open **http://localhost:8080** for the Dev Sandbox. ## Dev Sandbox — the wallet substitute -Source: `docs/docs/Post Platform Guide/dev-sandbox.md`. +Source: [Using the Dev Sandbox](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/dev-sandbox). The Dev Sandbox is a minimal browser app that uses `wallet-sdk` with a Web Crypto adapter. It lets you: @@ -222,6 +222,6 @@ pnpm docker:core:down ## References in the docs -- Local dev quick start: `docs/docs/Post Platform Guide/local-dev-quick-start.md` -- Dev Sandbox: `docs/docs/Post Platform Guide/dev-sandbox.md` +- Local dev quick start: [Local Dev Quick Start](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/local-dev-quick-start) +- Dev Sandbox: [Using the Dev Sandbox](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/dev-sandbox) - Full Docker setup + platforms: repo root `README.md` diff --git a/skills/w3ds/reference/evault.md b/skills/w3ds/reference/evault.md index fc3d043d4..5f86ef654 100644 --- a/skills/w3ds/reference/evault.md +++ b/skills/w3ds/reference/evault.md @@ -1,6 +1,6 @@ # eVault — data store + GraphQL -The eVault is the personal data store for a single W3ID. One eVault per tenant, Neo4j-backed, GraphQL at `/graphql`, HTTP endpoints for identity/log/file resolution. Source: `docs/docs/Infrastructure/eVault.md`. +The eVault is the personal data store for a single W3ID. One eVault per tenant, Neo4j-backed, GraphQL at `/graphql`, HTTP endpoints for identity/log/file resolution. Source: [eVault](https://docs.w3ds.metastate.foundation/docs/Infrastructure/eVault). ## Data model @@ -21,7 +21,7 @@ Missing this header returns 400 or "access denied" — it is the #1 integration ## GraphQL — idiomatic API -All shown below verified against `docs/docs/Infrastructure/eVault.md`. Endpoint: `POST {evaultUrl}/graphql`. +All shown below verified against [eVault](https://docs.w3ds.metastate.foundation/docs/Infrastructure/eVault). Endpoint: `POST {evaultUrl}/graphql`. ### Query one @@ -44,7 +44,7 @@ query { query { metaEnvelopes( filter: { - ontologyId: "550e8400-e29b-41d4-a716-446655440001" + ontologyId: "" search: { term: "hello", caseSensitive: false, mode: CONTAINS } } first: 10 @@ -64,7 +64,7 @@ Filter fields: `ontologyId`, `search.term`, `search.caseSensitive`, `search.fiel ```graphql mutation { createMetaEnvelope(input: { - ontology: "550e8400-e29b-41d4-a716-446655440001" + ontology: "" payload: { content: "Hello, world!" mediaUrls: [] @@ -88,7 +88,7 @@ mutation { updateMetaEnvelope( id: "global-id-123" input: { - ontology: "550e8400-e29b-41d4-a716-446655440001" + ontology: "" payload: { content: "Updated content", mediaUrls: [] } acl: ["*"] } @@ -195,7 +195,7 @@ mutation { } ``` -Binding documents are stored as MetaEnvelopes with ontology `b1d0a8c3-4e5f-6789-0abc-def012345678`. The MetaEnvelope ID is the binding document ID. See [identity.md](identity.md) for the type-specific data shapes. +Binding documents are stored as MetaEnvelopes with the `Binding Document` ontology (resolve its `schemaId` — see [registry.md § Resolving an ontology](registry.md#resolving-an-ontology)). The MetaEnvelope ID is the binding document ID. See [identity.md](identity.md) for the type-specific data shapes. ## GraphQL — legacy names (still valid) @@ -252,7 +252,7 @@ Response: "operation": "create", // create | update | delete | update_envelope_value "platform": "https://platform.example.com", "timestamp": "2025-02-04T12:00:00.000Z", - "ontology": "550e8400-e29b-41d4-a716-446655440001" + "ontology": "" } ], "nextCursor": "2025-02-04T12:00:00.000Z|log-entry-id", @@ -318,7 +318,7 @@ A valid platform Bearer token satisfies the legacy path but does **not** bypass Undeterminable membership is not "not a member": a grant needs proof and is withheld, a denial stands until non-membership is shown. With no resolver configured, groups match nobody at all. -Not yet wired: no condition evaluator is connected, so any `require` group containing conditions fails closed. Write policies using `grants`, `denials.enames`, group enames, and empty-group `require`. Full model: `docs/docs/W3DS Protocol/Access-Control.md`. +Not yet wired: no condition evaluator is connected, so any `require` group containing conditions fails closed. Write policies using `grants`, `denials.enames`, group enames, and empty-group `require`. Full model: [Access Control](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Access-Control). Special cases: @@ -342,7 +342,7 @@ Payload: { "id": "a1b2c3d4-...", "w3id": "@user-a.w3id", - "schemaId": "550e8400-e29b-41d4-a716-446655440001", + "schemaId": "", "data": { "content": "Hello, world!", "mediaUrls": [], @@ -369,7 +369,7 @@ The Provisioner supports multiple W3IDs sharing infrastructure, but each eVault ## References in the docs -- Full spec: `docs/docs/Infrastructure/eVault.md` -- Data model + ontology field semantics: `docs/docs/Infrastructure/Ontology.md` -- Webhook packet + delivery mechanics: `docs/docs/W3DS Protocol/Awareness-Protocol.md` -- Key binding certificate detail: `docs/docs/Infrastructure/eVault-Key-Delegation.md` +- Full spec: [eVault](https://docs.w3ds.metastate.foundation/docs/Infrastructure/eVault) +- Data model + ontology field semantics: [Ontology](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Ontology) +- Webhook packet + delivery mechanics: [Awareness Protocol](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Awareness-Protocol) +- Key binding certificate detail: [eVault Key Delegation](https://docs.w3ds.metastate.foundation/docs/Infrastructure/eVault-Key-Delegation) diff --git a/skills/w3ds/reference/identity.md b/skills/w3ds/reference/identity.md index cc397e883..80c67cd15 100644 --- a/skills/w3ds/reference/identity.md +++ b/skills/w3ds/reference/identity.md @@ -1,6 +1,6 @@ # Identity — W3ID, eName, Binding Documents -W3IDs identify every user, group, eVault, and MetaEnvelope in the ecosystem. An eName is a W3ID that has been registered in the Registry and is therefore resolvable to a service URL. Source: `docs/docs/W3DS Basics/W3ID.md`, `eName.md`, `Binding-Documents.md`. +W3IDs identify every user, group, eVault, and MetaEnvelope in the ecosystem. An eName is a W3ID that has been registered in the Registry and is therefore resolvable to a service URL. Source: [W3ID](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/W3ID), [eName](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/eName), [Binding Documents](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/Binding-Documents). ## W3ID @@ -8,8 +8,8 @@ UUID-based (RFC 4122), persistent, globally unique. Two forms: | Form | Format | Example | Use | |---|---|---|---| -| **Global** (eName) | `@` | `@e4d909c2-5d2f-4a7d-9473-b34b6c0f1a5a` | Cross-platform identity, ACLs, X-ENAME header | -| **Local** | plain `` | `f2a6743e-8d5b-43bc-a9f0-1c7a3b9e90d7` | Object identifier within one eVault | +| **Global** (eName) | `@` | `@e4d909c2-…-b34b6c0f1a5a` (elided; substitute a real eName) | Cross-platform identity, ACLs, X-ENAME header | +| **Local** | plain `` | `f2a6743e-…-1c7a3b9e90d7` | Object identifier within one eVault | Namespace has range 2^122 (from UUID); collision probability is negligible. Global IDs are case-insensitive. @@ -37,7 +37,7 @@ If you're building a platform, users and groups you interact with will always ha Required on every eVault GraphQL / HTTP request: ```http -X-ENAME: @e4d909c2-5d2f-4a7d-9473-b34b6c0f1a5a +X-ENAME: @ ``` Determines: which eVault to route the request to, ACL enforcement, log ownership. Missing header = 400. @@ -54,7 +54,7 @@ Determines: which eVault to route the request to, ACL enforcement, log ownership ## Binding Documents -A Binding Document is a special MetaEnvelope (ontology `b1d0a8c3-4e5f-6789-0abc-def012345678`) that ties a subject eName to a real-world credential or claim. Every binding document has: +A Binding Document is a special MetaEnvelope (ontology: the `Binding Document` `schemaId`, resolved from the Ontology service — see [registry.md § Resolving an ontology](registry.md#resolving-an-ontology)) that ties a subject eName to a real-world credential or claim. Every binding document has: - `subject` — the eName being bound (with `@` prefix) - `type` — one of `id_document | photograph | social_connection | self` @@ -120,9 +120,9 @@ The W3ID system supports binding an identity to a passport or other physical doc ## References in the docs -- W3ID spec: `docs/docs/W3DS Basics/W3ID.md` -- eName vs W3ID: `docs/docs/W3DS Basics/eName.md` -- Binding document types + operations: `docs/docs/W3DS Basics/Binding-Documents.md` -- ACL semantics: `docs/docs/Infrastructure/eVault.md` (§ Access Control) -- Granular `_acl` permissions: `docs/docs/W3DS Protocol/Access-Control.md` -- Key binding certificates: `docs/docs/Infrastructure/eVault-Key-Delegation.md` +- W3ID spec: [W3ID](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/W3ID) +- eName vs W3ID: [eName](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/eName) +- Binding document types + operations: [Binding Documents](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/Binding-Documents) +- ACL semantics: [eVault](https://docs.w3ds.metastate.foundation/docs/Infrastructure/eVault) (§ Access Control) +- Granular `_acl` permissions: [Access Control](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Access-Control) +- Key binding certificates: [eVault Key Delegation](https://docs.w3ds.metastate.foundation/docs/Infrastructure/eVault-Key-Delegation) diff --git a/skills/w3ds/reference/platform.md b/skills/w3ds/reference/platform.md index bf8a1a630..6505188d2 100644 --- a/skills/w3ds/reference/platform.md +++ b/skills/w3ds/reference/platform.md @@ -1,17 +1,19 @@ # Building a post-platform -This is the primary developer reference. A platform participating in W3DS needs four things: an auth flow, a webhook endpoint, JSON mapping files, and a Web3 Adapter wired to the local DB. Source: `docs/docs/Post Platform Guide/*.md`, `docs/docs/Infrastructure/Web3-Adapter.md`. +This is the primary developer reference. A platform participating in W3DS needs four things: an auth flow, a webhook endpoint, JSON mapping files, and a Web3 Adapter wired to the local DB. Source: the [Post Platform Guide](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/getting-started) section of the docs, plus [Web3 Adapter](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Web3-Adapter). ## The four required pieces | Piece | What | Reference doc | |---|---|---| -| **Auth endpoints** | `GET /api/auth/offer` + `POST /api/auth`, using `signature-validator` | `docs/docs/Post Platform Guide/getting-started.md` | -| **Webhook endpoint** | `POST /api/webhook` — idempotent, uses `adapter.fromGlobal` + mapping DB | `docs/docs/Post Platform Guide/webhook-controller.md` | -| **Mapping files** | JSON per local table describing the global schema mapping | `docs/docs/Post Platform Guide/mapping-rules.md` | -| **Web3 Adapter** | Instance holding the mapping configs, mapping DB, and eVault client; call `handleChange(...)` after every DB write | `docs/docs/Infrastructure/Web3-Adapter.md` | +| **Auth endpoints** | `GET /api/auth/offer` + `POST /api/auth`, using `signature-validator` | [Getting Started with Platform Development](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/getting-started) | +| **Webhook endpoint** | `POST /api/webhook` — idempotent, uses `adapter.fromGlobal` + mapping DB | [Webhook Controller Guide](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/webhook-controller) | +| **Mapping files** | JSON per local table describing the global schema mapping | [Mapping Rules](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/mapping-rules) | +| **Web3 Adapter** | Instance holding the mapping configs, mapping DB, and eVault client; call `handleChange(...)` after every DB write | [Web3 Adapter](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Web3-Adapter) | -If your app is stateless — writes directly to eVaults and doesn't own a local DB — you can skip the Web3 Adapter entirely. Adapter is only needed when a platform DB has to stay in sync with eVaults. +If your app is stateless — writes directly to eVaults and doesn't own a local DB — you can skip the Web3 Adapter entirely. The adapter exists only to keep a platform DB in sync with eVaults, and for a small application stateless is both less code and more obviously W3DS-native. Suggest it before building a sync layer nobody asked for. + +**Before you build any of this**, settle where the data lives. The eVault is the source of truth; the platform DB is a projection of it. Every entity you persist needs an ontology, an owner eName, and a write path to that owner's eVault — run the pre-flight in [SKILL.md](../SKILL.md#pre-flight--before-writing-any-w3ds-code) and read [w3ds-native.md](w3ds-native.md) if the answer to any of the four is unclear. The mechanics below assume that question is already answered; getting it wrong produces a conventional application with sync bolted on, which is the failure this reference exists to prevent. ## Auth flow @@ -151,7 +153,7 @@ The endpoint receives packets for **every** ontology — you must filter and dro **Idempotency is mandatory.** Same `globalId` may arrive more than once. Never create a second local row for the same global ID; always upsert. -Detail: `docs/docs/Post Platform Guide/webhook-controller.md`. +Detail: [Webhook Controller Guide](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/webhook-controller). ## Mapping directives @@ -173,13 +175,13 @@ Mapping files describe how local table fields ↔ global ontology fields. Same f ``` - `tableName` — local table / entity name. -- `schemaId` — global ontology W3ID (from [registry.md § Canonical ontology W3IDs](registry.md#canonical-ontology-w3ids)). +- `schemaId` — global ontology W3ID (from [registry.md § Resolving an ontology](registry.md#resolving-an-ontology)). - `ownerEnamePath` — how to determine which eVault owns rows in this table. Supports fallbacks with `||`. - `ownedJunctionTables` — for many-to-many relationships; when a junction row changes, the adapter re-syncs the parent. - `readOnly` (optional) — when `true`, `handleChange` skips this table for outbound sync. - `localToUniversalMap` — the field mapping. -### Directives (verbatim, from `docs/docs/Post Platform Guide/mapping-rules.md`) +### Directives (verbatim, from [Mapping Rules](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/mapping-rules)) **Direct field:** @@ -256,7 +258,14 @@ Detail on the URI scheme: [protocols.md § File URIs](protocols.md#file-uris-w3d "ownerEnamePath": "users(createdBy.ename) || ename" // fallback ``` -The adapter uses this to write to the correct owner's eVault. If it resolves to nothing, `handleChange` returns without syncing. +The adapter uses this to write to the correct owner's eVault. If it resolves to nothing, `handleChange` returns without syncing — silently. + +Two rules that decide whether the data is really owned by its subject: + +- **The owner is the data subject** — the person or group the record is *about*, not the platform that received the write. Pointing `ownerEnamePath` at the platform's own eName because it always resolves produces platform-owned data wearing an eVault costume: the user cannot take it, revoke it, or see it from another platform. +- **It must resolve for every row**, not the happy path. Check the nullable relation, the imported row, the record created before the user existed. Each unresolved row is an entity that never leaves the platform. + +Fallbacks with `||` are for alternative paths to the *same* subject, not a way to reach a default owner when the real one is missing. ### Junction tables @@ -275,7 +284,7 @@ User: ```json { "tableName": "users", - "schemaId": "550e8400-e29b-41d4-a716-446655440000", + "schemaId": "", "ownerEnamePath": "ename", "ownedJunctionTables": ["user_followers", "user_following"], "localToUniversalMap": { @@ -295,7 +304,7 @@ Group with relations: ```json { "tableName": "groups", - "schemaId": "550e8400-e29b-41d4-a716-446655440003", + "schemaId": "", "ownerEnamePath": "users(participants[].ename)", "localToUniversalMap": { "name": "name", @@ -311,7 +320,7 @@ Group with relations: ## Web3 Adapter — the sync engine -Source: `docs/docs/Infrastructure/Web3-Adapter.md`. +Source: [Web3 Adapter](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Web3-Adapter). ### Components @@ -369,6 +378,9 @@ Every one of these has bitten someone. Address them in code review: 7. **Non-idempotent webhook controller** → duplicate deliveries create duplicate rows. Always upsert by global ID. 8. **Auth expects the user to exist before webhook** → for a fresh eVault, the User MetaEnvelope needs to have been synced first. Order: provision + create User in eVault → webhook creates local user → then login can succeed. 9. **Missing `X-ENAME` on adapter calls** → 400s on every eVault write. +10. **A local table with no mapping** → the entity exists on this platform and nowhere else. Either it is operational state (sessions, queues, ID mappings, caches — fine) or it is user data the platform has quietly claimed. Resolve an ontology, or [propose one](w3ds-native.md#proposing-a-new-ontology). +11. **`handleChange` missed on some write paths** → the mapping is correct and the entity still doesn't sync, because migrations, seeds, admin endpoints or background jobs write around the adapter. Cover every path with an ORM listener or a transactional outbox, not a call bolted onto one controller. +12. **`ownerEnamePath` pointing at the platform** for records about users → everything syncs, into the wrong eVault. See [`ownerEnamePath` patterns](#ownerenamepath-patterns). ## Known limitations @@ -384,9 +396,11 @@ Design your platform's consistency layer accordingly. ## References in the docs -- Getting started (auth): `docs/docs/Post Platform Guide/getting-started.md` -- Webhook controller: `docs/docs/Post Platform Guide/webhook-controller.md` -- Mapping rules: `docs/docs/Post Platform Guide/mapping-rules.md` -- eCurrency example: `docs/docs/Post Platform Guide/ecurrency-accounts-and-ledger.md` -- Web3 Adapter architecture: `docs/docs/Infrastructure/Web3-Adapter.md` -- Awareness Protocol packet + timing: `docs/docs/W3DS Protocol/Awareness-Protocol.md` +- Getting started (auth): [Getting Started with Platform Development](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/getting-started) +- Webhook controller: [Webhook Controller Guide](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/webhook-controller) +- Mapping rules: [Mapping Rules](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/mapping-rules) +- eCurrency example: [eCurrency: Accounts and Ledger MetaEnvelopes](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/ecurrency-accounts-and-ledger) +- Web3 Adapter architecture: [Web3 Adapter](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Web3-Adapter) +- Awareness Protocol packet + timing: [Awareness Protocol](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Awareness-Protocol) +- Where data lives: [Data Ownership Rules](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/Data-Ownership-Rules) +- Access control on synced records: [Implementing Access Control](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/access-control) diff --git a/skills/w3ds/reference/protocols.md b/skills/w3ds/reference/protocols.md index dc1d68bba..dad5ba270 100644 --- a/skills/w3ds/reference/protocols.md +++ b/skills/w3ds/reference/protocols.md @@ -1,6 +1,6 @@ # W3DS protocols -Four wire-level protocols to know: `w3ds://auth`, `w3ds://sign`, the Awareness Protocol (webhooks), and `w3ds://file` URIs. Plus signature verification, which is shared by auth and sign. Source: `docs/docs/W3DS Protocol/*.md`. +Four wire-level protocols to know: `w3ds://auth`, `w3ds://sign`, the Awareness Protocol (webhooks), and `w3ds://file` URIs. Plus signature verification, which is shared by auth and sign. Source: the [W3DS Protocol](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Authentication) section of the docs. ## w3ds://auth (authentication) @@ -38,7 +38,7 @@ Request body: ```json { "w3id": "@user-a.w3id", - "session": "550e8400-e29b-41d4-a716-446655440000", + "session": "", "signature": "xK3vJZQ2...==", "appVersion": "0.4.0" } @@ -74,7 +74,7 @@ Temporary field (will be sunset). Present because some early wallets signed diff - Expire in ≤ 5 minutes. - Return generic errors; never leak "user not found" vs "signature invalid". -Detail: `docs/docs/W3DS Protocol/Authentication.md`. +Detail: [Authentication](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Authentication). ## w3ds://sign (arbitrary signatures) @@ -107,11 +107,11 @@ w3ds://sign?session={sessionId}&data={base64Data}&redirect_uri={encodedCallback} 5. Platform validates, verifies signature with `verifySignature(...)` using `message` as the `payload`, then processes the action and marks the session `completed` (or `security_violation`). -Detail: `docs/docs/W3DS Protocol/Signing.md`. +Detail: [Signing](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Signing). ## Awareness Protocol (webhooks) -Prototype-level fanout from eVault-core to every registered platform after a write. Fire-and-forget. Source: `docs/docs/W3DS Protocol/Awareness-Protocol.md`. +Prototype-level fanout from eVault-core to every registered platform after a write. Fire-and-forget. Source: [Awareness Protocol](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Awareness-Protocol). ### When it fires @@ -133,7 +133,7 @@ Prototype-level fanout from eVault-core to every registered platform after a wri { "id": "a1b2c3d4-...", "w3id": "@e4d909c2-...", - "schemaId": "550e8400-e29b-41d4-a716-446655440001", + "schemaId": "", "data": { "content": "Hello, world!", "mediaUrls": [], @@ -167,7 +167,7 @@ For production, use Awareness-as-a-Service. ### Awareness-as-a-Service (AaaS) -Production-grade replacement layer. Source: `docs/docs/Services/Awareness-as-a-Service.md`. Key differences vs raw Awareness Protocol: +Production-grade replacement layer. Source: [Awareness as a Service (AaaS)](https://docs.w3ds.metastate.foundation/docs/Services/Awareness-as-a-Service). Key differences vs raw Awareness Protocol: - `POST /ingest` accepts packets from eVault-core. - `GET /api/packets` — poll query with filters (ontology, eVault, time). @@ -180,7 +180,7 @@ Undifferentiated fanout → targeted delivery; no history → queryable; ungover ## Signature formats -Source: `docs/docs/W3DS Protocol/Signature-Formats.md`. +Source: [Signature Formats](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Signature-Formats). ### The algorithm @@ -242,7 +242,7 @@ import { verifySignature } from "signature-validator"; const result = await verifySignature({ eName: "@user.w3id", signature: "z3K7vJZQ2F3k5L8mN9pQrS7tUvW1xY3zA5bC7dE9fG1hIjKlMnOpQrStUvWxYz", - payload: "550e8400-e29b-41d4-a716-446655440000", + payload: "", registryBaseUrl: "https://registry.w3ds.metastate.foundation", }); @@ -251,7 +251,7 @@ const result = await verifySignature({ ## File URIs (`w3ds://file`) -Standard URI scheme for referencing blobs. Source: `docs/docs/W3DS Protocol/File-URIs.md`. +Standard URI scheme for referencing blobs. Source: [File URIs](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/File-URIs). ### Format @@ -311,7 +311,7 @@ Two distinct schemas. Conflating them is a common source of bugs. | | `w3ds-file-v1` | `File` ontology | |---|---|---| -| Identifier | `w3ds-file-v1` (string literal) | `a1b2c3d4-e5f6-7890-abcd-ef1234567890` (UUID) | +| Identifier | `w3ds-file-v1` — a **protocol string literal**, never looked up | A UUID `schemaId`, resolved from `GET /schemas` by title `File` | | Created by | `uploadFile` mutation | Platform apps (file-manager, esigner) via Web3 Adapter mapping | | Layer | Storage / transport — describes a blob | Application domain — a file record in a platform DB | | Payload keys | `filename`, `contentType`, `size`, `blobKey`, `publicUrl`, `uploadedAt` | `id`, `name`, `displayName`, `description`, `mimeType`, `size`, `md5Hash`, `data`, `url`, `ownerId`, `folderId`, `createdAt`, `updatedAt` | @@ -326,9 +326,9 @@ The Web3 Adapter's `__file(...)` mapping directive automatically calls `uploadFi ## References in the docs -- Authentication: `docs/docs/W3DS Protocol/Authentication.md` -- Signing: `docs/docs/W3DS Protocol/Signing.md` -- Signature formats: `docs/docs/W3DS Protocol/Signature-Formats.md` -- Awareness Protocol: `docs/docs/W3DS Protocol/Awareness-Protocol.md` -- File URIs: `docs/docs/W3DS Protocol/File-URIs.md` -- Awareness-as-a-Service: `docs/docs/Services/Awareness-as-a-Service.md` +- Authentication: [Authentication](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Authentication) +- Signing: [Signing](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Signing) +- Signature formats: [Signature Formats](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Signature-Formats) +- Awareness Protocol: [Awareness Protocol](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Awareness-Protocol) +- File URIs: [File URIs](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/File-URIs) +- Awareness-as-a-Service: [Awareness as a Service (AaaS)](https://docs.w3ds.metastate.foundation/docs/Services/Awareness-as-a-Service) diff --git a/skills/w3ds/reference/registry.md b/skills/w3ds/reference/registry.md index fc52f9629..0ca585ff5 100644 --- a/skills/w3ds/reference/registry.md +++ b/skills/w3ds/reference/registry.md @@ -1,6 +1,6 @@ # Registry + Ontology -The Registry is the discovery layer: it resolves W3IDs to service URLs, publishes JWKS, provides signed entropy for provisioning, and (temporarily) issues key-binding certificates. The Ontology service is the schema registry. Source: `docs/docs/Infrastructure/Registry.md`, `Ontology.md`. +The Registry is the discovery layer: it resolves W3IDs to service URLs, publishes JWKS, provides signed entropy for provisioning, and (temporarily) issues key-binding certificates. The Ontology service is the schema registry. Source: [Registry](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Registry) and [Ontology](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Ontology). ## Registry @@ -70,30 +70,35 @@ Flow: eVault stores a user's public key at provisioning time and internally requ Production URL: `https://ontology.w3ds.metastate.foundation`. +**This service is the only correct source of a `schemaId`.** This skill deliberately contains no ontology UUIDs: they are not derivable, not sequential, and not stable enough to recall. A wrong `schemaId` fails silently — the write succeeds and every receiving platform drops the packet for having no matching mapping. + ### GET /schemas -Returns a list of every registered schema: +Every registered schema: ```json [ - { "id": "550e8400-e29b-41d4-a716-446655440000", "title": "User" }, - { "id": "550e8400-e29b-41d4-a716-446655440001", "title": "SocialMediaPost" } + { "id": "", "title": "User", "domain": "identity" }, + { "id": "", "title": "SocialMediaPost", "domain": "social" } ] ``` -### GET /schemas/:id +- `id` — the `schemaId` to put in `mapping.json` and in eVault calls. +- `title` — what to match on. Singular, PascalCase. +- `domain` — the domain the schema belongs to, or `null`. -Returns the full JSON Schema (draft-07) for a schema W3ID. 404 if not found. +### GET /schemas/:id -Every schema must include: `schemaId` (W3ID), `title`, `type` (usually `"object"`), `properties`, `required`, `additionalProperties: false` (usually). +The full JSON Schema (draft-07) for a schema W3ID. 404 if not found. Read this before writing a mapping — it is where the real property names live. -Example: +Every schema includes: `schemaId` (W3ID), `title`, `domain`, `type` (usually `"object"`), `properties`, `required`, `additionalProperties: false` (usually). ```json { "$schema": "http://json-schema.org/draft-07/schema#", - "schemaId": "550e8400-e29b-41d4-a716-446655440001", + "schemaId": "", "title": "SocialMediaPost", + "domain": "social", "type": "object", "properties": { "id": { "type": "string", "format": "uri", "description": "W3ID" }, @@ -106,7 +111,17 @@ Example: } ``` -In eVault, a `SocialMediaPost` MetaEnvelope has `ontology: "550e8400-e29b-41d4-a716-446655440001"`; its Envelopes have `fieldKey` values matching the schema's property names (`content`, `authorId`, `createdAt`, ...). +In eVault, a `SocialMediaPost` MetaEnvelope has `ontology: ""`; its Envelopes have `fieldKey` values matching the schema's property names (`content`, `authorId`, `createdAt`, ...). + +### GET /domains + +The domain list every schema is tagged with — the same list a platform is granted access to, one domain at a time. Returns `{ schemaId, domains: [{ id, label, description }] }`. Read from the `Domain` schema's own enum, so it is versioned like any other type. + +### GET /domains/:id/schemas + +Every schema under one domain: `{ domain, schemas: [{ id, title }] }`. 404 if the domain does not exist. + +Use this when you know the subject area but not the type name — "what does W3DS already have for finance?" — before concluding nothing fits. ### Human viewer @@ -115,31 +130,29 @@ In eVault, a `SocialMediaPost` MetaEnvelope has `ontology: "550e8400-e29b-41d4-a Use `/schemas` and `/schemas/:id` for programmatic access. -## Canonical ontology W3IDs +## Resolving an ontology + +The procedure, every time. Do not skip to memory. + +1. `GET https://ontology.w3ds.metastate.foundation/schemas` and match on `title`. +2. Narrow first with `GET /domains/:id/schemas` when you know the subject area but not the name. +3. `GET /schemas/:id` on the match — confirm the property names before writing `localToUniversalMap`, because the mapping's global side must use them exactly. +4. No match? Read the near misses in full. Extending one by PR beats creating a parallel type. +5. Still nothing? [Propose a new ontology](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Ontology#proposing-a-new-ontology) — full procedure in [w3ds-native.md](w3ds-native.md#proposing-a-new-ontology). Until that PR merges the type does not exist; do not ship a mapping against an unmerged `schemaId`. -Memorize this table. These IDs appear in mapping.json files, in Awareness Protocol packets (`schemaId`), and in every eVault call — guessing them is guaranteed to be wrong. +Cannot reach the service? Use an obvious placeholder plus `// TODO(w3ds): unverified — confirm against https://ontology.w3ds.metastate.foundation/schemas`, and say so in your response. Never substitute a plausible-looking UUID. -| Ontology | ID | -|---|---| -| **User** | `550e8400-e29b-41d4-a716-446655440000` | -| **SocialMediaPost** | `550e8400-e29b-41d4-a716-446655440001` | -| **Group** | `550e8400-e29b-41d4-a716-446655440003` | -| **Ledger** (eCurrency) | `550e8400-e29b-41d4-a716-446655440006` | -| **Currency** (eCurrency) | `550e8400-e29b-41d4-a716-446655440008` | -| **Account** (eCurrency) | `6fda64db-fd14-4fa2-bd38-77d2e5e6136d` | -| **Binding Document** | `b1d0a8c3-4e5f-6789-0abc-def012345678` | -| **File** (application layer) | `a1b2c3d4-e5f6-7890-abcd-ef1234567890` | -| **`w3ds-file-v1`** (storage layer) | `w3ds-file-v1` — **string literal, not a UUID** | +### The one identifier that is not a lookup -Never confuse the `File` ontology and `w3ds-file-v1`. Different layers, different field names. Detail in [protocols.md § File URIs](protocols.md#file-uris-w3dsfile). +`w3ds-file-v1` is a **protocol string literal**, not a registry entry and not a UUID. It is the low-level storage envelope created by `uploadFile` for blob dereferencing. -If a user asks about an ontology not on this list, call `GET https://ontology.w3ds.metastate.foundation/schemas` to enumerate — do not guess. +Never confuse it with the `File` ontology, which is a higher-level platform record (file-manager / esigner style apps) with its own resolvable `schemaId`. Different layers, different field names. Detail in [protocols.md § File URIs](protocols.md#file-uris-w3dsfile). ## Provisioner (adjacent, not part of Registry) Production URL: `https://provisioner.w3ds.metastate.foundation`. Local dev port: **3001** (co-hosted by eVault-core). -`POST /provision` creates a new eVault. Detail in [wallet.md](wallet.md#provisioning). Body: +`POST /provision` creates a new eVault. Detail in [wallet.md](wallet.md#provisioning--onboarding-a-new-evault). Body: ```json { @@ -154,7 +167,9 @@ Response: `{ w3id, uri }`. ## References in the docs -- Registry endpoints + JWKS: `docs/docs/Infrastructure/Registry.md` -- Ontology API + schema format: `docs/docs/Infrastructure/Ontology.md` -- Provisioning flow: `docs/docs/Infrastructure/eID-Wallet.md`, `docs/docs/Infrastructure/wallet-sdk.md` -- Production URLs: `docs/docs/W3DS Basics/Links.md` +- Registry endpoints + JWKS: [Registry](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Registry) +- Ontology API + schema format: [Ontology](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Ontology) +- Provisioning flow: [eID Wallet](https://docs.w3ds.metastate.foundation/docs/Infrastructure/eID-Wallet), [wallet-sdk](https://docs.w3ds.metastate.foundation/docs/Infrastructure/wallet-sdk) +- Production URLs: [Links](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/Links) +- Where data lives: [Data Ownership Rules](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/Data-Ownership-Rules) +- Proposing a new ontology: [Ontology](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Ontology#proposing-a-new-ontology) diff --git a/skills/w3ds/reference/w3ds-native.md b/skills/w3ds/reference/w3ds-native.md new file mode 100644 index 000000000..7bc6d3db0 --- /dev/null +++ b/skills/w3ds/reference/w3ds-native.md @@ -0,0 +1,179 @@ +# Building W3DS-native + +Load this when the task is a design decision rather than an API call: adding a table, entity, model or field; deciding what to cache; reviewing whether an application is actually W3DS-native or just W3DS-flavoured. + +Authoritative source: [Data Ownership Rules](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/Data-Ownership-Rules). Supporting: [Getting Started](https://docs.w3ds.metastate.foundation/docs/Getting%20Started/getting-started), [W3DS Basics](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/getting-started), [Web3 Adapter](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Web3-Adapter). + +## The claim, and what it does not mean + +> **The eVault is the source of truth. Anything a platform stores is a projection of it.** + +The docs also say platforms "act as frontends that display and interact with this data, while also serving as caches and aggregators for improved performance and user experience." Those are not in tension. A cache is a fast copy of something authoritative elsewhere. The permission is to keep the copy; it was never permission to be the original. + +So: **a local database is fine.** Pictique, Blabsy and eCurrency all run one. What is not fine is a local database that is the only place some user data exists. + +## The reconstructability test + +The one question that settles almost every case: + +> If this database were dropped and rebuilt by replaying the relevant eVaults, what would be lost? + +Apply it **per entity type**, not per application. Platforms are usually correct about their main entity and wrong about the one table someone added in a hurry. + +### Worked examples + +**A user's post — projection, correct.** +The post is mapped, has an `ownerEnamePath` resolving to the author, and `handleChange` runs on write. Drop the database, replay the authors' eVaults, and every post comes back. The local row exists so the feed renders in 20ms instead of N eVault round trips. Textbook cache. + +**A draft the user never published — violation.** +It is in `posts` with `status = 'draft'`, and the mapping only syncs on publish. Drop the database and the user's unfinished work is gone — there is nowhere to replay it from. This is user data the platform has taken ownership of by accident. + +*The fix is not "sync drafts to the eVault" reflexively.* Ask which is true: (a) it is user data → give it an ontology and an owner, and sync it; or (b) it is genuinely ephemeral UI state the user does not expect to survive → keep it client-side and say so. What you may not do is persist it server-side, indefinitely, in the platform's database alone. + +**A session token — operational, correct.** +Nothing to reconstruct; it is meaningless outside this platform and expires. Never belonged in an eVault. + +**A `(localId, globalId)` mapping row — operational, correct and required.** +It is bookkeeping *about* the sync, not user data. Without it the same logical entity gets duplicated or never linked, which is [the classic integration bug](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Web3-Adapter). + +**A denormalised follower count — projection, correct.** +Derived from records that are themselves eVault-sourced. Recomputable, so nothing is lost. + +## What every persisted entity needs + +1. **An ontology** — a `schemaId` resolved from the Ontology service. Never invented. +2. **A resolvable owner** — an `ownerEnamePath` that resolves for *every* row. The owner is the data subject: the person or group the data is *about*. +3. **A named write path** — the `handleChange` call site, or the direct eVault write. + +Missing any one means the entity is platform-owned. + +## Legitimate local-only state + +- Sessions, auth nonces, the short-lived session IDs from the `w3ds://auth` flow. +- Job queues, retry state, outbox rows, dead letters. +- Rate limits, feature flags, request logs. +- The `(localId, globalId)` mapping table. +- Cached Registry resolutions and the platform's own `w3id` / `uri`, which [Platform eVault registration](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/platform-evault-registration) explicitly tells platforms to persist and reuse on boot. +- Derived indexes, search indexes, aggregates and read models built *from* eVault-sourced records. + +Common thread: none of it is data about a user that a user would expect to take with them. + +## Anti-patterns + +### 1. A local table with no mapping + +**Wrong** — a `reactions` table, no `mapping.json`, no `schemaId`. Likes exist on this platform and nowhere else. + +**Right** — resolve an ontology for it (`GET /schemas`, narrow with `GET /domains/:id/schemas`); if none fits, propose one (below) and say the type does not exist until the PR merges. + +**Why** — reactions are the user's data. Without an ontology they cannot leave, and the platform has silently claimed them. + +### 2. `handleChange` never called + +**Wrong** — the entity is mapped, but the write happens in a migration, an admin endpoint or a background job that skips the adapter. + +**Right** — name the write hook and cover *every* path: an ORM event listener (afterInsert / afterUpdate / afterRemove) or a transactional outbox, not a call bolted onto one controller. The adapter does **not** poll; if nobody calls it, nothing syncs. + +**Why** — partial coverage is worse than none. Rows written through the uncovered path are invisible to the ecosystem while looking synced. + +### 3. `ownerEnamePath` pointing at the platform + +**Wrong** — `"ownerEnamePath": "ename"` resolving to the platform's own eName for records about users, because it always resolves and makes the errors go away. + +**Right** — resolve to the data subject: `users(createdBy.ename)`, `users(participants[].ename)`, with `||` fallbacks only between paths that all name the subject. + +**Why** — the data lands in the platform's eVault. Ownership, ACLs and portability all follow the owner eName, so this is platform-owned data wearing an eVault costume. The user cannot take it, revoke it, or see it from another platform. + +### 4. Reading local in preference to eVault-derived state + +**Wrong** — the local row is served as canonical, and inbound webhook updates are dropped or merged as "conflicts" against it. + +**Right** — inbound wins on the entity's own fields; the local row is a projection. Keep local-only columns strictly to operational state. + +**Why** — the platform becomes authoritative in practice regardless of what the architecture diagram says, and a user's change from another platform silently disappears. + +### 5. Inventing a `schemaId` to unblock + +**Wrong** — a plausible UUID in `mapping.json` so the work can proceed. + +**Right** — resolve it, or propose the schema. If you genuinely cannot reach the Ontology service, use an obvious placeholder plus `// TODO(w3ds): unverified` and say so in your response. + +**Why** — this fails silently and expensively. The MetaEnvelope writes, the packet fans out, and every receiving platform finds no mapping and drops it. Everything looks healthy on the writing platform. + +### 6. Mirroring an uploaded blob as a second `File` record + +**Wrong** — after `uploadFile`, writing a second envelope under the `File` ontology so the upload is observable. + +**Right** — consume the awareness packet. [File URIs](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/File-URIs) states it directly: there is no need to mirror the upload just to make it observable. + +**Why** — two records for one blob, drifting apart, and `w3ds-file-v1` and the `File` ontology are different layers with different field names. + +### 7. Caching a resolved eVault URL forever + +**Wrong** — resolve once, store the URL, use it indefinitely (or hardcode it). + +**Right** — cache the resolution, revalidate, evict on a failed `HEAD /whois` and re-resolve. This is what `EVaultClient` does. + +**Why** — the eName is permanent, the URL is not. eVaults migrate; the Registry is how you find out. + +### 8. Treating eventual consistency as immediate + +**Wrong** — write locally, then immediately read back the eVault-derived version and assume it is there. + +**Right** — design for last-write-wins, no ordering, no at-least-once delivery, a delay after create before fanout (immediate on update), and the requesting platform excluded from its own fanout. Idempotent on the global `id`, tolerant of a record that has not arrived. + +**Why** — the Awareness Protocol is prototype-level and fire-and-forget. Anything user-visible that assumes otherwise breaks intermittently and unreproducibly. + +## Proposing a new ontology + +The escape hatch when nothing fits. Ontologies are ordinary JSON files anyone can propose, so "no ontology exists" is never a reason to invent one or to give up. + +**First, be sure.** `GET https://ontology.w3ds.metastate.foundation/schemas` and search titles; `GET /domains/:id/schemas` for the subject area; read near misses in full with `GET /schemas/:id`. **Extending a near match by PR beats creating a parallel type** — two schemas meaning the same thing split the ecosystem, and platforms mapping one will not see data from platforms mapping the other. + +**Then write it.** A file at `services/ontology/schemas/.json` in `MetaState-Prototype-Project/prototype`. The service loads the directory into an in-memory index at boot — no database, no registration call. + +```json +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "schemaId": "", + "title": "Reaction", + "domain": "social", + "type": "object", + "properties": { + "id": { "type": "string", "format": "uuid", "description": "The unique identifier for the reaction" }, + "authorId": { "type": "string", "format": "uri", "description": "W3ID of the reacting user" }, + "targetId": { "type": "string", "format": "uri", "description": "W3ID of the record reacted to" }, + "kind": { "type": "string", "description": "The reaction type, e.g. like" }, + "createdAt": { "type": "string", "format": "date-time", "description": "When the reaction was created" } + }, + "required": ["id", "authorId", "targetId", "createdAt"], + "additionalProperties": false +} +``` + +- **`schemaId`** — freshly generated random UUIDv4 (`uuidgen`, `crypto.randomUUID()`). Never derived from an existing ID, never continuing a numeric sequence you notice in the directory. +- **`title`** — the type name others will search for. Singular, PascalCase. +- **`domain`** — one value from `GET /domains`. Platforms are granted access domain by domain, so this decides who can consume the type. +- **`properties`** — every field with a `description`. Each property name becomes an Envelope's `ontology` value, so **name fields for cross-platform meaning, not after local columns**. +- **`additionalProperties`** — `false` unless there is a specific reason. + +**Then open the PR**, saying what the type is for and which platform will write it, and answering up front why no existing schema could carry it. + +**Until it merges, the type does not exist.** Do not ship a `mapping.json` referencing an unmerged `schemaId` — it will look fine locally and drop everywhere else. + +Full version: [Proposing a new ontology](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Ontology#proposing-a-new-ontology). + +## Stateless is simplest + +An application that writes directly to eVaults and keeps no local database needs no Web3 Adapter at all — the adapter exists to keep a database in sync, and there is nothing to sync. For a small application this is both the least code and the most obviously W3DS-native option. Suggest it before building a sync layer nobody asked for. + +## Review checklist + +For an existing platform, in order: + +1. List every persisted table or collection. +2. For each, find the `mapping.json`. No mapping → is it operational state, or claimed user data? +3. For each mapping, check `ownerEnamePath` resolves to the data subject for every row, not just the common case. +4. Find every write path per mapped table — migrations, seeds, admin endpoints, background jobs included — and confirm each reaches `handleChange`. +5. Check the webhook controller is idempotent on global `id` and 200s ontologies it does not consume. +6. Run the reconstructability test over the whole set and state what would be lost. diff --git a/skills/w3ds/reference/wallet.md b/skills/w3ds/reference/wallet.md index a8fff8b45..8bbbc9d6f 100644 --- a/skills/w3ds/reference/wallet.md +++ b/skills/w3ds/reference/wallet.md @@ -1,6 +1,6 @@ # eID Wallet + wallet-sdk + key delegation -Everything about identity provisioning, signature creation, and key management across devices. Source: `docs/docs/Infrastructure/eID-Wallet.md`, `wallet-sdk.md`, `eVault-Key-Delegation.md`. +Everything about identity provisioning, signature creation, and key management across devices. Source: [eID Wallet](https://docs.w3ds.metastate.foundation/docs/Infrastructure/eID-Wallet), [wallet-sdk](https://docs.w3ds.metastate.foundation/docs/Infrastructure/wallet-sdk), [eVault Key Delegation](https://docs.w3ds.metastate.foundation/docs/Infrastructure/eVault-Key-Delegation). ## eID Wallet — what it is @@ -198,7 +198,7 @@ The W3ID does not change during rotation. ### Friend-based recovery -A trust list (2–3 friends or notaries) can vouch for identity and approve key changes. The user defines the list while they still hold their keys. Not yet implemented — described in `docs/docs/W3DS Basics/W3ID.md`. +A trust list (2–3 friends or notaries) can vouch for identity and approve key changes. The user defines the list while they still hold their keys. Not yet implemented — described in [W3ID](https://docs.w3ds.metastate.foundation/docs/W3DS%20Basics/W3ID). ## eVault endpoints used by the wallet @@ -278,8 +278,8 @@ Never commit. Never reuse across environments. Never use desktop keys in product ## References in the docs -- eID Wallet architecture: `docs/docs/Infrastructure/eID-Wallet.md` -- wallet-sdk API: `docs/docs/Infrastructure/wallet-sdk.md` -- Key delegation + `PATCH /public-key`: `docs/docs/Infrastructure/eVault-Key-Delegation.md` -- Desktop signing detail: `docs/docs/W3DS Protocol/Signature-Formats.md` -- Dev Sandbox (in-browser wallet substitute): `docs/docs/Post Platform Guide/dev-sandbox.md` +- eID Wallet architecture: [eID Wallet](https://docs.w3ds.metastate.foundation/docs/Infrastructure/eID-Wallet) +- wallet-sdk API: [wallet-sdk](https://docs.w3ds.metastate.foundation/docs/Infrastructure/wallet-sdk) +- Key delegation + `PATCH /public-key`: [eVault Key Delegation](https://docs.w3ds.metastate.foundation/docs/Infrastructure/eVault-Key-Delegation) +- Desktop signing detail: [Signature Formats](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Signature-Formats) +- Dev Sandbox (in-browser wallet substitute): [Using the Dev Sandbox](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/dev-sandbox) From 54ca32a6ed4170fe4c350d9212225afbd60bdc6d Mon Sep 17 00:00:00 2001 From: coodos Date: Mon, 31 Aug 2026 21:46:16 +0800 Subject: [PATCH 4/4] skill: add GitW3 reference and require the platform to live in a GitW3 repository --- .../Post Platform Guide/ai-agent-skill.md | 2 + skills/README.md | 2 +- skills/w3ds/SKILL.md | 11 +- skills/w3ds/reference/gitw3.md | 134 ++++++++++++++++++ skills/w3ds/reference/platform.md | 2 + 5 files changed, 148 insertions(+), 3 deletions(-) create mode 100644 skills/w3ds/reference/gitw3.md diff --git a/docs/docs/Post Platform Guide/ai-agent-skill.md b/docs/docs/Post Platform Guide/ai-agent-skill.md index 580ad38f8..f53886dc1 100644 --- a/docs/docs/Post Platform Guide/ai-agent-skill.md +++ b/docs/docs/Post Platform Guide/ai-agent-skill.md @@ -48,6 +48,7 @@ The skill is not only a reference. It changes how an agent behaves on W3DS work: - **The eVault is the source of truth.** The platform database is a projection of it. The skill applies the reconstructability test — *if this database were dropped and rebuilt by replaying the relevant eVaults, what would be lost?* — before agreeing to persist anything new. See [Data Ownership Rules](/docs/W3DS%20Basics/Data-Ownership-Rules). - **Resolve, never recall.** Ontology IDs, endpoints, GraphQL field names and ACL verbs are looked up at the time of use. The skill deliberately contains no ontology UUIDs, so there is nothing stale to copy. Where it cannot verify something — no fetch tool, or the service is unreachable — it says so and marks the spot in code rather than substituting a plausible value. - **Two hard stops.** The agent stops and asks, rather than writing code, when a design would make the local database authoritative for user data, or when a persisted entity type has no ontology. The second is a path rather than a wall: ontologies are ordinary JSON files, and the agent will draft the schema and offer to open the PR. See [Proposing a new ontology](/docs/Infrastructure/Ontology#proposing-a-new-ontology). +- **A platform belongs in a GitW3 repository.** The same instinct one layer up: the repository is the source of truth for the platform metadata W3DS publishes. The skill raises this early rather than after the application is wired to another forge, knows that a plain repository import is not the guided port flow, and refuses to hand-edit managed `.w3ds/platform.json` fields, fabricate a proof, or commit `w3ds-deployment-key.json`. See [GitW3](/docs/GitW3/overview). - **A definition of done.** `X-ENAME` on every call, `handleChange` on every write path, an idempotent webhook controller, no invented identifiers. If you want an agent that produces a conventional application with sync bolted on, do not install this skill. That is the outcome it exists to prevent. @@ -62,6 +63,7 @@ If you want an agent that produces a conventional application with sync bolted o - `reference/protocols.md` — `w3ds://auth`, `w3ds://sign`, Awareness Protocol, signature formats, `w3ds://file` - `reference/platform.md` — building a post-platform (auth, webhook, mapping directives, Web3 Adapter) - `reference/wallet.md` — eID Wallet, wallet-sdk, key delegation +- `reference/gitw3.md` — GitW3: the platform manifest, platform / version / deployment eNames, PPA, porting an existing app - `reference/dev-setup.md` — `pnpm dev:core` + debugging playbook Everything in it cites this site by URL, so an agent that gets stuck has somewhere authoritative to go. Where the skill and these docs disagree, the docs win. diff --git a/skills/README.md b/skills/README.md index 7618282e8..0189ff996 100644 --- a/skills/README.md +++ b/skills/README.md @@ -8,7 +8,7 @@ Installable AI-agent skills for the MetaState / W3DS ecosystem. Powered by the [ | Skill | Purpose | |-------|---------| -| [w3ds](./w3ds) | Web 3 Data Spaces — build post-platforms, call the eVault GraphQL API, wire the Web3 Adapter, implement `w3ds://auth` / `w3ds://sign`, debug local dev. Enforces eVault-first design: the eVault is the source of truth, the platform DB is a projection. | +| [w3ds](./w3ds) | Web 3 Data Spaces — build post-platforms, call the eVault GraphQL API, wire the Web3 Adapter, implement `w3ds://auth` / `w3ds://sign`, host the platform on GitW3, debug local dev. Enforces eVault-first design: the eVault is the source of truth, the platform DB is a projection, and the platform's own identity lives in its GitW3 repository. | ## Use it without installing anything diff --git a/skills/w3ds/SKILL.md b/skills/w3ds/SKILL.md index 0563a9ca8..baff1f5e5 100644 --- a/skills/w3ds/SKILL.md +++ b/skills/w3ds/SKILL.md @@ -1,6 +1,6 @@ --- name: w3ds -description: "Use when the user is building on Web 3 Data Spaces (W3DS) or the MetaState prototype — building a post-platform, integrating an eVault, calling the eVault GraphQL API (createMetaEnvelope, updateMetaEnvelope, removeMetaEnvelope, bulkCreateMetaEnvelopes, uploadFile, bindingDocument*), wiring the Web3 Adapter, writing a webhook controller for /api/webhook, authoring mapping.json files, using the wallet-sdk, implementing the w3ds://auth or w3ds://sign flow, resolving W3IDs / eNames via the Registry, working with the Ontology service, proposing a new ontology, dealing with Binding Documents, dereferencing w3ds://file URIs, provisioning an eVault, syncing public keys, or debugging local dev (Registry, Provisioner, eVault-core, Dev Sandbox, pnpm dev:core). Also use for any design decision about where data lives on a W3DS platform — adding a table, entity, model or field, deciding what to cache locally, or asking whether something belongs in the database or the eVault. Also use when the user asks what an eVault, W3ID, eName, MetaEnvelope, Envelope, Ontology, Web3 Adapter, Awareness Protocol, or Awareness-as-a-Service is." +description: "Use when the user is building on Web 3 Data Spaces (W3DS) or the MetaState prototype — building a post-platform, integrating an eVault, calling the eVault GraphQL API (createMetaEnvelope, updateMetaEnvelope, removeMetaEnvelope, bulkCreateMetaEnvelopes, uploadFile, bindingDocument*), wiring the Web3 Adapter, writing a webhook controller for /api/webhook, authoring mapping.json files, using the wallet-sdk, implementing the w3ds://auth or w3ds://sign flow, resolving W3IDs / eNames via the Registry, working with the Ontology service, proposing a new ontology, dealing with Binding Documents, dereferencing w3ds://file URIs, provisioning an eVault, syncing public keys, hosting a platform on GitW3 (the W3DS-aware Git forge — `.w3ds/platform.json`, platform eName, version eName, PPA certification, deployment records, porting an existing app, `w3ds-deployment-key.json`), or debugging local dev (Registry, Provisioner, eVault-core, Dev Sandbox, pnpm dev:core). Also use for any design decision about where data lives on a W3DS platform — adding a table, entity, model or field, deciding what to cache locally, or asking whether something belongs in the database or the eVault. Also use when the user asks what an eVault, W3ID, eName, MetaEnvelope, Envelope, Ontology, Web3 Adapter, Awareness Protocol, or Awareness-as-a-Service is." license: Apache 2.0 --- @@ -30,7 +30,8 @@ Read [Data Ownership Rules](https://docs.w3ds.metastate.foundation/docs/W3DS%20B 1. **The eVault is the source of truth. Anything a platform stores is a projection of it.** The docs call platforms "caches and aggregators" — that is permission to keep a fast local copy of data that is authoritative elsewhere, not permission to own it. 2. **Every persisted entity needs three things:** an ontology (`schemaId` resolved, never invented), a resolvable owner (`ownerEnamePath` that resolves for *every* row, to the data subject — not to the platform), and a named write path to that owner's eVault. -3. **Resolve, never recall.** Ontology IDs, endpoints, GraphQL field names, ACL verbs and eNames are looked up, not remembered. This skill deliberately contains no ontology UUIDs. +3. **A platform lives in a GitW3 repository.** The same instinct one layer up: the repository is the source of truth for the platform metadata W3DS publishes, held in `.w3ds/platform.json` beside the code. Ordinary Git hosting carries the code but not the platform eName, published profile, per-version identities, PPA certificates or deployment records. See [reference/gitw3.md](reference/gitw3.md). +4. **Resolve, never recall.** Ontology IDs, endpoints, GraphQL field names, ACL verbs and eNames are looked up, not remembered. This skill deliberately contains no ontology UUIDs. A local database is not a violation. A local database that is the only place some user data exists is. @@ -74,6 +75,7 @@ Before reporting a W3DS task complete, check every line: - [ ] The webhook controller is idempotent on the global `id`, and returns 200 for ontologies the platform does not consume. - [ ] Nothing was invented: no UUID, endpoint path, GraphQL field, mapping directive or ACL verb that was not verified — or, if unverifiable, each is flagged in the response and marked in code. - [ ] The reconstructability test was applied to anything newly persisted, and the answer stated. +- [ ] If the work touched a platform repository: no managed `.w3ds/platform.json` field was hand-edited, no history was rewritten, and no key material was committed. ## Ecosystem map @@ -95,6 +97,7 @@ The "digital self" is a triad: **eName + eID certificate + eVault**. Users hold | **w3ds://sign** | Session-signing for arbitrary payloads (documents, votes, references) | [reference/protocols.md](reference/protocols.md) | | **w3ds://file** | URI scheme for file blobs; format `w3ds://file?id=@/` | [reference/protocols.md](reference/protocols.md) | | **AaaS** | Awareness-as-a-Service — production-grade replacement for eVault's direct webhook fanout | [reference/protocols.md](reference/protocols.md) | +| **GitW3** | W3DS-aware Git forge; `.w3ds/platform.json`, platform / version / deployment eNames, PPA | [reference/gitw3.md](reference/gitw3.md) | ## Production URLs @@ -115,6 +118,7 @@ Load the reference file(s) below **before** writing any code or configuration. | User question mentions... | Load | |---|---| | "should this live in the database", "add a table / entity / model / field", "how do I model X", caching, local copies, data ownership, "is this W3DS-native", a design or architecture review | [reference/w3ds-native.md](reference/w3ds-native.md) | +| GitW3, `.w3ds/platform.json`, platform eName, version eName, deployment eName, PPA certificate, porting an existing app, `git remote` / tags / releases for a platform, `w3ds-deployment-key.json`, "where do I host this" | [reference/gitw3.md](reference/gitw3.md) | | webhook controller, mapping.json, `handleChange`, `fromGlobal`, `toGlobal`, Web3 Adapter, `ownerEnamePath`, `__date`, `__calc`, `__file`, "how do I build a platform" | [reference/platform.md](reference/platform.md) | | GraphQL, `createMetaEnvelope`, `updateMetaEnvelope`, `removeMetaEnvelope`, `bulkCreateMetaEnvelopes`, `uploadFile`, `metaEnvelope(id)`, `metaEnvelopes`, ACL, `X-ENAME`, `/whois`, `/logs`, MetaEnvelope, Envelope, Neo4j model | [reference/evault.md](reference/evault.md) | | W3ID, eName, `@` format, X-ENAME header, Binding Document, id_document, photograph, social_connection, self, key rotation, friend-based recovery | [reference/identity.md](reference/identity.md) | @@ -134,6 +138,7 @@ Any of these values, if guessed, is almost certainly wrong: - **Mapping directive syntax** — `__date(...)`, `__calc(...)`, `__file(...)`, `tableName(path),globalAlias`, and array `users(participants[].id),participantIds` — verbatim examples in [reference/platform.md](reference/platform.md). - **Signature encoding** — software keys emit base64 raw 64-byte (r || s); hardware keys emit multibase base58btc (`z...`). See [reference/protocols.md](reference/protocols.md). - **Endpoint paths and headers** — every eVault request needs `X-ENAME: @`. `/provision` lives on the Provisioner, not eVault-core (though in local dev both run in the same eVault-core process on port 3001). +- **Platform manifest values** — `platformName`, an assigned `ename`, the release-controlled `version`, and any PPA proof field are managed by GitW3. Never hand-edit, fabricate, or copy them between platforms. See [reference/gitw3.md](reference/gitw3.md). Uncertain? Fetch the relevant page from `https://docs.w3ds.metastate.foundation` — or, if you cannot, follow [When you cannot verify](#when-you-cannot-verify). @@ -155,4 +160,6 @@ Uncertain? Fetch the relevant page from `https://docs.w3ds.metastate.foundation` - Webhook delivery is fire-and-forget and prototype-level: no retries, no ordering, no at-least-once. Make the webhook controller **idempotent** on global `id`. - After `storeMetaEnvelope` there is a 3-second delay before webhook fanout to prevent ping-pong. `updateMetaEnvelopeById` fanout is immediate. - Do not mirror what you can already observe. If a record reaches you through the Awareness Protocol, subscribe to it rather than writing a second envelope to make it visible. +- Building a platform they intend to publish? Say early that it belongs in a GitW3 repository — a plain repository import is not the same as the guided port flow, and retrofitting an identity after the fact is worse than starting there. +- Never commit `w3ds-deployment-key.json`, a platform token, a migration proof or a personal access token. If asked to paste key material anywhere, stop and say why. - If the user is running things locally, check [reference/dev-setup.md](reference/dev-setup.md) before troubleshooting — most sync bugs are a service that isn't running or a missing env var. diff --git a/skills/w3ds/reference/gitw3.md b/skills/w3ds/reference/gitw3.md new file mode 100644 index 000000000..85e5dc128 --- /dev/null +++ b/skills/w3ds/reference/gitw3.md @@ -0,0 +1,134 @@ +# GitW3 — where a platform lives + +Load this when the work touches the platform's identity, repository, releases, certification or deployment records: `.w3ds/platform.json`, a platform eName, `git remote`, tags and releases, PPA, or a deployment key. + +Authoritative source: [GitW3 overview](https://docs.w3ds.metastate.foundation/docs/GitW3/overview) and the rest of the GitW3 section. Production: `https://git.w3ds.metastate.foundation`. + +## The rule + +**A W3DS platform lives in a GitW3 repository.** GitW3 is the W3DS-aware Git forge, and it is built around one principle that mirrors the eVault rule: + +> The repository is the source of truth for the platform metadata that W3DS publishes. + +The metadata lives beside the code in `.w3ds/platform.json`, and changes made from the GitW3 **W3DS** tab become ordinary commits on the default branch. So the same instinct applies at both layers: the user's data belongs in their eVault, and the platform's identity belongs in its repository. Neither is a value you keep in a dashboard, a config service, or your head. + +Ordinary Git hosting still works for the code. What it cannot do is carry the platform's permanent eName, its published profile, its per-version identities, its PPA certificates, or its deployment records — those are what GitW3 adds. If a user is building a platform they intend to publish, GitW3 is where it belongs; say so early rather than after they have wired everything to another forge. + +## What GitW3 manages + +| Item | Meaning | +|---|---| +| Repository | Code, issues, pull requests, tags, releases — the normal forge workflow | +| Platform manifest | `.w3ds/platform.json`, version-controlled W3DS metadata | +| Platform eName | The permanent identity of the platform, stable across releases. Gets the platform eVault | +| Version eName | The identity of one exact released version. A Registry record; does **not** get its own eVault | +| PPA certificate | Approval for one exact platform version | +| Deployment eName | A verifiable record of one running deployment | + +GitW3 **records** deployments; it does not host them. Keep the user's existing hosting provider and pipeline. + +## The lifecycle + +Sign in with W3DS → create or port a repository → manifest committed → permanent platform eName provisioned → publish a stable release → sign and submit the PPA application → certificate granted → register and sign a deployment. + +Identity and profile publication are **asynchronous**. Pushes and repository creation do not block on W3DS services; the **W3DS** tab reports publisher state and retries. Do not write code that assumes an eName exists immediately after creation. + +## Two starting paths — do not conflate them + +- **Make a new platform** — for an application with no W3DS identity. GitW3 creates the repository, a README, `.w3ds/platform.json`, and the first commit, then provisions the permanent platform eName in the background. No reusable platform private key is created or exposed. +- **Port an existing app** — for an application that already exists, especially one with `.w3ds` configuration or a permanent platform eName. It creates an **empty destination first**, then you push, then any eName migration is staged and signed, and the public cutover is activated explicitly by an administrator. + +**A plain repository import is not a port.** If an existing platform identity must survive, use the guided port flow — the migration is signed in the eID wallet and the old public listing stays in control until activation. + +## `.w3ds/platform.json` + +```json +{ + "schemaVersion": 1, + "platformName": "example-platform", + "displayName": "Example Platform", + "description": "A short description of the platform.", + "version": "0.1.0", + "ename": null, + "url": "https://example.invalid", + "logoUrl": "https://example.invalid/logo.png", + "domains": [""], + "inSubmission": false, + "submissionVersion": "", + "isDraft": true +} +``` + +**Never hand-edit these** — they are managed, and editing them breaks the identity: + +- `platformName` — the stable machine-facing slug, immutable after identity creation. The friendly `displayName` changes freely; this does not. +- `ename` — starts `null`, then written by the publisher or preserved by the port flow. **Immutable once assigned.** Never invent one, never copy one from another platform. +- `version` — synchronized from the latest stable semantic release. Do not bump it by hand. +- `inSubmission` / `submissionVersion` and any proof fields — managed by the signed PPA workflow. Never fabricate, copy between platforms, or hand-edit cryptographic proof material. + +Editable by permitted users: `displayName`, `description`, `url` (required before PPA), `logoUrl`, `domains`, and draft visibility. `domains` takes real W3DS application-domain identifiers — the same list at `GET https://ontology.w3ds.metastate.foundation/domains` that schemas are tagged with. Resolve them; do not guess. + +Editing locally: pull first, change only unmanaged fields, validate the JSON and the domain ids, push, then watch the **W3DS** tab. The tab's own saves are commits on the default branch, so a local edit and a tab edit can collide. + +## Releases and PPA + +**PPA certifies one exact version.** A certificate for `1.2.3` says nothing about `1.2.4`, and the **Deploy** tab only enables releases whose exact normalized version was granted. + +The release must be a **published stable semantic release**: tag `v1.2.3`, push it, then publish it through **Releases → New release**. GitW3 normalizes the leading `v` to manifest version `1.2.3` and binds it to the release commit. Drafts, prereleases and mutable tags like `latest` do not become the W3DS platform version. + +Prerequisites before an application can be signed: a ready platform eName, at least one application domain, a public application URL, a published stable semantic release, and an owner/admin signed in with an eID wallet. The signing request is one-time and expires after 15 minutes. + +Never invent or paste a submission proof into the manifest. + +## Deployments and the deployment key + +Registering a deployment reserves a **deployment eName** (bound to the deployment's public key and the connected deployer) and a **software-version eName** (bound to the platform eName, release version and exact commit). One wallet signature covers both; nothing is provisioned until it verifies. + +The key is the part to get right in code: + +- GitW3 generates an ECDSA P-256 pair **in the browser** and downloads `w3ds-deployment-key.json` once, in the `w3ds-deployment-key-v1` format: algorithm metadata (ECDSA P-256 / SHA-256), the `z`-prefixed public key, the base64 PKCS#8 private key, and a creation timestamp. +- **GitW3 never receives the private key and cannot recover it.** Lost means register a new deployment identity and roll the server secret. +- Load it **server-side only**, from a secret manager or a read-only mount. Prefer an env var such as `W3DS_DEPLOYMENT_KEY_FILE` pointing at the mounted file over putting key material in an environment variable directly. +- Validate at startup that the private key derives the expected public key. +- Never commit `w3ds-deployment-key.json`, never ship it in a browser or mobile bundle, never expose it through an API, never paste it into a chat or a prompt. If asked to, stop and say why. +- Using an existing key instead? Paste only the `z`-prefixed **public** key. + +**PP-Auth SDK integration is marked coming soon in GitW3.** Leave a narrow integration boundary; do not invent a package name, endpoint, token format or wire protocol to fill the gap. See [protocols.md](protocols.md) and [Platform Authentication](https://docs.w3ds.metastate.foundation/docs/W3DS%20Protocol/Platform-Authentication) for what actually exists. + +## Working in a GitW3 checkout + +Clone with the exact URL from the repository's clone control. The `@` shown before eNames in the UI is cosmetic — never type it into a remote. + +Adding GitW3 to an existing checkout, preserving the old remote: + +```bash +git remote -v +git remote rename origin upstream # or upstream-2 if taken; skip if no origin +git remote add origin +git push -u origin HEAD:main # use the repository's real default branch +``` + +Rules for an agent doing this: + +- Inspect `git status`, the current branch, `git remote -v` and any `.w3ds` directory **before** editing anything. +- Preserve the complete history and the application's behaviour. **Never force-push** unless the user has independently reviewed and approved the rewrite. +- Create a manifest **only** when no W3DS identity already exists. Preserve any existing eName exactly. +- Copy the real remote URL and default branch from GitW3 rather than typing them from memory. + +**Stop and ask instead of guessing when:** authentication to either remote fails; the GitW3 destination is unexpectedly non-empty; local and destination histories conflict; an existing eName would be removed or replaced; `.w3ds/platform.json` is invalid; the connected wallet is not an author of the existing profile; or a push would require rewriting history. + +## Never invent + +An eName, a platform token, a migration proof, an ontology or domain identifier, an endpoint, or a credential. These fail silently or destructively, and a wrong one can transfer or break a live platform identity. The general rule from [SKILL.md](../SKILL.md#when-you-cannot-verify) applies: name what you could not verify, mark it in code, and do not substitute a plausible value. + +## References + +- [GitW3 overview](https://docs.w3ds.metastate.foundation/docs/GitW3/overview) +- [Create a new platform](https://docs.w3ds.metastate.foundation/docs/GitW3/create-a-platform) +- [Port an existing application](https://docs.w3ds.metastate.foundation/docs/GitW3/port-an-existing-application) +- [Work with repositories](https://docs.w3ds.metastate.foundation/docs/GitW3/work-with-repositories) +- [Platform manifest and W3DS workspace](https://docs.w3ds.metastate.foundation/docs/GitW3/platform-manifest-and-workspace) +- [Releases and PPA certification](https://docs.w3ds.metastate.foundation/docs/GitW3/releases-and-ppa) +- [Register a deployment](https://docs.w3ds.metastate.foundation/docs/GitW3/deploy-a-release) +- [Troubleshooting GitW3](https://docs.w3ds.metastate.foundation/docs/GitW3/troubleshooting) +- [Sign in and manage your account](https://docs.w3ds.metastate.foundation/docs/GitW3/sign-in-and-account) diff --git a/skills/w3ds/reference/platform.md b/skills/w3ds/reference/platform.md index 6505188d2..af245d02a 100644 --- a/skills/w3ds/reference/platform.md +++ b/skills/w3ds/reference/platform.md @@ -11,6 +11,8 @@ This is the primary developer reference. A platform participating in W3DS needs | **Mapping files** | JSON per local table describing the global schema mapping | [Mapping Rules](https://docs.w3ds.metastate.foundation/docs/Post%20Platform%20Guide/mapping-rules) | | **Web3 Adapter** | Instance holding the mapping configs, mapping DB, and eVault client; call `handleChange(...)` after every DB write | [Web3 Adapter](https://docs.w3ds.metastate.foundation/docs/Infrastructure/Web3-Adapter) | +The platform also needs an identity, which is not code: a GitW3 repository holding `.w3ds/platform.json` and the permanent platform eName provisioned from it. See [gitw3.md](gitw3.md) — it is easier to start there than to retrofit an identity onto a repository already living somewhere else. + If your app is stateless — writes directly to eVaults and doesn't own a local DB — you can skip the Web3 Adapter entirely. The adapter exists only to keep a platform DB in sync with eVaults, and for a small application stateless is both less code and more obviously W3DS-native. Suggest it before building a sync layer nobody asked for. **Before you build any of this**, settle where the data lives. The eVault is the source of truth; the platform DB is a projection of it. Every entity you persist needs an ontology, an owner eName, and a write path to that owner's eVault — run the pre-flight in [SKILL.md](../SKILL.md#pre-flight--before-writing-any-w3ds-code) and read [w3ds-native.md](w3ds-native.md) if the answer to any of the four is unclear. The mechanics below assume that question is already answered; getting it wrong produces a conventional application with sync bolted on, which is the failure this reference exists to prevent.