A statement keeps the document's words

[WaylandYang]

Cut 1 of 0044 (#710): the open graph lands in the ledger, and extraction can write only that layer. Follows the sequence in #725 (queues come after cuts 1 and 2) and the time record 0045 (#724), whose interpretation columns wait for a later cut.

What the ledger gets

An open statement is what a document says, in its own words: a subject, a relation phrase as written, an object entity or a literal value, qualifiers keyed by the document's role words, the time words it mentions, and provenance to the chunk. Today an unbound relation is representable (0010: predicate_id NULL, phrase on fact_evidence.proposed_predicate), but it dedups badly (NULL-blind key, every observation inserts a row), qualifiers must be keyed by an ontology attribute (0037), a described thing cannot be an entity (canonical_name and the known_as fact are mandatory), time exists only as computed dates, and nothing marks the layer.

Migration 0061_a_statement_keeps_the_documents_words.sql (0060 is taken by #721). Additive, no backfill; there is no released version.

ALTER TABLE knowledge_bases ADD COLUMN open_extraction BOOLEAN NOT NULL DEFAULT FALSE;

ALTER TABLE facts
  ADD COLUMN layer TEXT NOT NULL DEFAULT 'typed' CHECK (layer IN ('typed', 'open')),
  ADD COLUMN phrase TEXT,
  ADD CONSTRAINT facts_open_statement_shape
    CHECK (layer <> 'open' OR (phrase IS NOT NULL AND predicate_id IS NULL));
CREATE INDEX facts_open_by_subject ON facts (kb_id, subject_id, phrase)
  WHERE layer = 'open' AND invalidated_at IS NULL;

ALTER TABLE fact_evidence ADD COLUMN quote_start INT, ADD COLUMN quote_end INT;  -- char offsets into chunks.text; NULL = not located

ALTER TABLE entities ADD COLUMN description TEXT;  -- a described, unnamed thing

CREATE TABLE statement_qualifiers (
  fact_id   UUID NOT NULL REFERENCES facts (id) ON DELETE CASCADE,
  role      TEXT NOT NULL,                         -- the document's word
  value     JSONB,
  entity_id UUID REFERENCES entities (id) ON DELETE CASCADE,
  PRIMARY KEY (fact_id, role),
  CHECK ((value IS NOT NULL) <> (entity_id IS NOT NULL))
);

CREATE TABLE time_mentions (
  id          UUID PRIMARY KEY,
  kb_id       UUID NOT NULL,
  fact_id     UUID NOT NULL REFERENCES facts (id) ON DELETE CASCADE,
  chunk_id    UUID NOT NULL REFERENCES chunks (id) ON DELETE CASCADE,
  text        TEXT NOT NULL,                        -- verbatim words, never a computed date
  char_start  INT  NOT NULL,
  recorded_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  UNIQUE (fact_id, chunk_id, char_start)
);

Open statements are facts rows with layer = 'open', so every read path that already tolerates a NULL predicate (graph, entity detail, paths, export, RDF, tools, tests/no_predicate_still_shows.rs) shows them labelled by the phrase without change. Both clocks: attested_from (record time) is always set; attested_at comes from doc_time only when doc_time_source IN ('content', 'source'), never from upload time (#714); no valid_* is written for an open statement.

Store

• graph::insert_open_statement(kb, subject, phrase, object | value, attested_at, confidence): dedup on (kb, subject, phrase, object IS NOT DISTINCT FROM value) among live open rows; writes layer, phrase, and proposed_predicate = phrase on the evidence so present displays stay the same.
• graph::add_evidence_located(fact_id, chunk_id, quote, span); add_evidence delegates. Offsets are computed on the server with a text search, never taken from the model.
• graph::add_statement_qualifier, statement_qualifiers_for; new module time_mentions with record and for_facts.
• resolution::create_described(kb, description, kind_word): an entity with description set and no known_as fact, so it never becomes a recall bridge (0041).
• proposed_predicates and adopt are restricted to layer = 'typed', so an open phrase is never auto-adopted as a relation type.
• corroborating_candidate, containment_reviews, refresh_disambiguators switch to LEFT JOIN relation_types with COALESCE(key, phrase), so adjudication profiles read open statements.
• kbs::update and UpdateKbReq gain open_extraction.

The compact extraction contract

New utopia_extract::open: build_open_messages(filename, known, opening, chunk_text) and parse_open_response. No ontology in the prompt, no document date, no dates asked for. The reply is arrays with short keys and no repeated quotes:

{"q": ["verbatim sentence", "..."],
 "e": [[0, "name or description", "kind word", 1]],
 "s": [[0, "relation phrase as written", 1, null, {"role": "words"}, [0], 0]],
 "t": [[0, "verbatim time words", 0]],
 "n": [[0, "another name", 0]]}

e: id, name or description, the document's kind word, named (1) or described (0). s: subject ref, phrase, object ref or null, literal value or null, qualifiers keyed by the document's role words or null, time-mention ids or null, quote index. t: id, verbatim words, quote index. n: entity ref, other name, quote index.

Server checks, each a drop reason in extraction_drops: every quote is in the chunk (QUOTE_NOT_IN_CHUNK), every name and time text is in its quote (TIME_NOT_IN_QUOTE), every ref exists (UNKNOWN_REF). Offsets come from the located quote.

Write path

New utopia-server::extraction_open::run_open. In extraction::run, after the chat client is built: when kb.open_extraction is on and the document is not a memory document awaiting a nod, run the open path and return; otherwise today's path runs unchanged. The open path reuses the chunk loop, epoch checks, retrying chat, origin_ceiling, mark_chunk_extracted, incomplete_reason, name recording (0041), and the govern and adjudicate_entities enqueues. It skips the ontology lists, the predicate index, record_miss, the temporal reconcile, signature_breaks, resolve_types and bootstrap_ontology. Flag off changes nothing: one if.

Not in this cut

• Alignment, bindings, implication rules, and typed facts derived from open statements (cut 2, Review queues are organised by what a decision changes #725 queue 2).
• Time interpretation (shape, anchor, offset, granularity) and document time context (0045 cuts).
• Identity profiles and constrained clustering (0044 decision 6).
• A "layer" marker in the UI; open rows render as today's unnamed relation until cut 2 gives them a name.
• Retiring the typed extraction path; it stays behind the flag until the open path passes the bench on all corpora.

Tests and measurement

• utopia-store/tests/an_open_statement_keeps_the_documents_words.rs: an open statement with a role qualifier and a time mention shows in overview, entity detail, neighborhood, paths and export under its phrase; a second observation reuses the row and adds evidence; proposed_predicates excludes it.
• a_described_thing_is_an_entity_without_a_name.rs: create_described writes no known_as fact and name recall does not find it.
• a_time_mention_is_words_not_a_date.rs: the words and char_start resolve inside chunks.text; the statement has no valid_from.
• Unit tests in open.rs: compact arrays parse, truncated output repairs, a bad quote index is a drop, the prompt carries no relation list.
• Bench: scripts/bench/recall.mjs --open flips the flag and scores entity-pair recall on the same corpora as today (SEC, pharma-en, ai-timeline; Re-DocRED via a fetch script, never as prompt examples), two runs per configuration. A judge_open.mjs samples open statements against chunks.text and reports the not-stated rate. Thresholds from 0044: not-stated at most 2%, entity-pair recall no lower than today's, prompt tokens per document reported.

Sequence

Claim 0061 early (0041 cut 2 also waits for the next number). One PR to dev. Cut 2 follows on the same tables: typed rows gain a from_statement_id, and the alignment cache keys on the signature.

[WaylandYang]

Contract change, measured on the branch (#731): statements now name their subject and object and carry their own verbatim quote; the numeric ids (e ids, s refs, t time ids, q quote indices) are gone.

Why: on a dense earnings-release chunk, deepseek-v4-flash mixed up the ids. Two replies to the same passage gave "NVIDIA has reached AI" and "tokens are tokens" where the text says "AI has reached its inflection point"; 15 refs pointed at nothing. The prototype, which named things in words and quoted per statement, fabricated 0 of 333.

Effect on the same 30 reference units (judge: deepseek-v4-pro, stated / misworded / not stated):

corpus	statements	stated	misworded	not stated
FDA approvals, ids	131	83.2%	13.7%	3.1%
FDA approvals, names	141	94.3%	2.8%	2.8%
CN statistics, ids	104	89.4%	9.6%	1.0%
CN statistics, names	139	98.6%	1.4%	0.0%
NVDA units, ids	71	88.7%	11.3%	0.0%
NVDA units, names	108	93.5%	6.5%	0.0%

Also tightened: a time mention is recorded only when its words occur in the statement's own quote (the model was attaching "week 4" to unrelated sentences). The year in a bulletin's title no longer attaches to every statement; that is the document time context of 0045, not the statement's.

Remaining FDA "not stated" cases are the unfolding of descriptions the prompt asks for ("Rezdiffra use — of — Rezdiffra") and the judge's strictness on "patients with hemophilia A"; neither is an invented fact. Rule 4 of the prompt may need "unless the description only names what it belongs to".

[WaylandYang]

Rounds 3–6 on the branch (prompt rule 4 refined; then temperature 0 for open extraction):

round	FDA approvals	CN statistics bulletin	NVDA 8-K and release
r3 rule 4	98 / 91.8% / 1.0% / 48.6%	101 / 97.0% / 0.0% / 56.2%	102 / 93.8% / 0.0% / 56.7%
r4 rule 4	143 / 95.8% / 0.7% / 59.2%	124 / 97.6% / 0.0% / 55.5%	101 / 80.2% / 0.0% / 71.6%
r5 temp 0	142 / 94.4% / 0.7% / 58.1%	128 / 93.8% / 6.3% / 41.6%	99 / 80.8% / 0.0% / 64.5%
r6 temp 0	131 / 93.1% / 1.5% / 52.5%	130 / 92.3% / 0.8% / 58.4%	96 / 97.9% / 0.0% / 56.7%

(statements / stated / not stated / stated-fact coverage; NVDA full-document recall 46/52 and 45/52.)

Rule 4 ("do not unfold a description that only names what it belongs to") brought FDA under the 2% line without costing coverage; r3's sparseness was sampling variance at the endpoint's default temperature, so open extraction now samples at temperature 0 and per-unit density is stable between runs. Two follow-ups outside this cut: financial tables (the period column of a cell is not captured; all of NVDA's remaining misworded rows), and figures in dense sentences (the model states "purchase price payable to stockholders" and drops the $11.9 billion; a second pass asking which figures and dates are not yet in a statement).