Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions crates/utopia-extract/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -771,6 +771,13 @@ Names:\n\
\"DeepMind\", \"OpenAI Ireland Ltd\" is not \"OpenAI\", \"Microsoft AI\" is not \
\"Microsoft\"), a project, programme, team, app or component. Never merge a version into \
its family or a part into its whole.\n\
- A document, agreement or filing is cited in many ways and stays one thing through its \
amendments: \"the Lease\", \"Lease Agreement\", \"Lease Agreement dated May 16, 2016\" and \
\"Lease Agreement dated May 16, 2016, as amended\" are one agreement when their parties and \
subject do not contradict each other. When it was signed and that it was amended describe \
the agreement; they do not make a second one. Each amendment is a document of its own, and \
an agreement for other premises, another phase or other parties (\"Phase 2 Lease\") is a \
different agreement.\n\
- A phrase that merely contains a name is not that name: \"Sam Altman's efforts\", \
\"psychological abuse from Sam Altman\", \"share sale led by Thrive Capital\", \"leaked \
letter from the National Data Guardian\", \"ChatGPT played a role in the campaign\", \"a \
Expand Down
60 changes: 57 additions & 3 deletions crates/utopia-server/src/governance.rs
Original file line number Diff line number Diff line change
Expand Up @@ -370,7 +370,10 @@ fn pair_of(item: &ReviewItem, p: &Precedents) -> utopia_extract::AdjudicationPai
}

/// 第一层没定(没判决、置信度不到线),或者同名、大类不冲突、模型却说不同——那是它
/// 最爱错的一种:都让第二层带着全部事实与原文再看一遍。硬规则拦下的不进:再看也不会改规则
/// 最爱错的一种:都让第二层带着全部事实与原文再看一遍。名字的形状让人起疑的合并(版本
/// 尾巴、含着名字的一长串)也再看:名字只能让人起疑,定不了是不是一个东西——「Lease
/// Agreement dated May 16, 2016, as amended」与「Lease Agreement」形状上是一句话,事实上
/// 是同一份租约。类型冲突这条硬规则拦下的不进:再看也不会改规则
fn wants_second_look(item: &ReviewItem, p: &Precedents, look: &Look) -> bool {
let types_conflict = gov::types_conflict(
item.left.type_label.as_deref(),
Expand All @@ -381,12 +384,38 @@ fn wants_second_look(item: &ReviewItem, p: &Precedents, look: &Look) -> bool {
&& !types_conflict
&& look.same == Some(false)
&& look.calls == 0;
let doubted_merge =
name_doubts(shape) && !types_conflict && look.same == Some(true) && look.calls == 0;
((gov::gate(look.same, look.conf, types_conflict, shape, p) == Gate::Propose
&& look.uncertain())
|| doubted_split)
|| doubted_split
|| doubted_merge)
&& p.reverts.is_empty()
}

/// 名字形状让人起疑的合并:版本尾巴,或含着另一个名字的一长串
fn name_doubts(shape: gov::NameShape) -> bool {
matches!(shape, gov::NameShape::Version | gov::NameShape::Phrase)
}

/// 闸门看的名字形状。第二层带着两边的事实与原文看过、仍说是同一个的,形状的疑点已经由
/// 证据答过了,不再按形状拦——拦的只剩没看过证据的第一层
fn shape_for_gate(item: &ReviewItem, look: &Look) -> gov::NameShape {
settled_shape(
gov::name_shape(&item.left.name, &item.right.name),
look.calls > 0,
)
}

/// 形状的疑点在第二层读过证据之后就答完了
fn settled_shape(shape: gov::NameShape, evidence_read: bool) -> gov::NameShape {
if evidence_read && name_doubts(shape) {
gov::NameShape::Unrelated
} else {
shape
}
}

/// 裁决器的入口(0028):治理关着,攒批判不定的对也带工具再看一遍——同一个循环、
/// 同一份预算。每次调用一个 run_id:一次裁决任务就是一次 run
pub(crate) async fn look_again(
Expand Down Expand Up @@ -446,7 +475,7 @@ async fn apply(ctx: &Ctx<'_>, item: &ReviewItem, p: &Precedents, look: Look) ->
item.left.type_label.as_deref(),
item.right.type_label.as_deref(),
);
let shape = gov::name_shape(&item.left.name, &item.right.name);
let shape = shape_for_gate(item, &look);

let action = look.action();
let precedents = gov::precedents_json(p);
Expand Down Expand Up @@ -908,3 +937,28 @@ pub async fn after_human_decision(
_ => {}
}
}

#[cfg(test)]
mod tests {
use super::*;

#[test]
fn a_doubtful_shape_stops_counting_once_the_evidence_was_read() {
let lease = gov::name_shape(
"Lease Agreement dated May 16, 2016, as amended",
"Lease Agreement",
);
assert!(name_doubts(lease), "{lease:?}");
assert_eq!(settled_shape(lease, false), lease);
assert_eq!(settled_shape(lease, true), gov::NameShape::Unrelated);

let version = gov::name_shape("Claude Mythos 5", "Claude Mythos");
assert!(name_doubts(version), "{version:?}");
assert_eq!(settled_shape(version, false), version);

// 形状本来就不起疑的,读没读过证据都照旧
let same = gov::name_shape("OpenAI", "OpenAI");
assert!(!name_doubts(same));
assert_eq!(settled_shape(same, true), same);
}
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 0025 · Governance reads the ledger before it decides

- **Status**: decision 3 revised 2026-09-07 (three clusters a round, #458) · decision 10 added 2026-09-07 (identity rules, name shapes, type families, the labeled set and `govern.mjs`) · decision 4 revised 2026-09-06 (the agent's own confidence decides, history moves the bar or blocks) · cut 1 implemented (#434) · migration 0035 adds `knowledge_bases.governance` and `agent_decisions`; `governance` in the store holds the precedent families, the first-in-first-out queue with its clusters, the gate and the table; the `govern` job in the server reads the switch, calls the model with precedents and applies or proposes; `?queue=agent`, `ReviewCounts.agent` and `POST /kbs/{id}/review/agent/{decision_id}` on the API · cut 3 (UI, #437) implemented: the switch in base settings, the Agent queue with its rows and answers, the proposal chip on a duplicate card whose Merge / Keep answers the proposal, the Agent section on the Overview · cut 2 implemented: migration 0036 adds `question`, `trace` and `calls` to `agent_decisions`; `governor` in the extract crate holds the tools, the opening and the step reader; `investigate` in the server job runs the loop for pairs the batch could not settle (decision 8) · cut 4 implemented: migration 0037 adds `knowledge_bases.governance_since`; `fuse` in the server job turns the switch off after two reverts of the agent's merges since it was turned on, within seven days, raises `governance.tripped` and writes the ledger (decision 9) · [0026](0026-a-decision-records-why.md) gives a decision its stated reason, hands the batch adjudicator the same precedents and quotes the reason into them · [0028](0028-the-adjudicator-looks-before-it-asks.md) opens the loop of cut 2 to the adjudicator with governance off and gives it a `consequences` tool
- **Status**: decision 10 revised 2026-09-14 (a shape that raises doubt goes to the second look, and what that look finds is applied, #700) · decision 3 revised 2026-09-07 (three clusters a round, #458) · decision 10 added 2026-09-07 (identity rules, name shapes, type families, the labeled set and `govern.mjs`) · decision 4 revised 2026-09-06 (the agent's own confidence decides, history moves the bar or blocks) · cut 1 implemented (#434) · migration 0035 adds `knowledge_bases.governance` and `agent_decisions`; `governance` in the store holds the precedent families, the first-in-first-out queue with its clusters, the gate and the table; the `govern` job in the server reads the switch, calls the model with precedents and applies or proposes; `?queue=agent`, `ReviewCounts.agent` and `POST /kbs/{id}/review/agent/{decision_id}` on the API · cut 3 (UI, #437) implemented: the switch in base settings, the Agent queue with its rows and answers, the proposal chip on a duplicate card whose Merge / Keep answers the proposal, the Agent section on the Overview · cut 2 implemented: migration 0036 adds `question`, `trace` and `calls` to `agent_decisions`; `governor` in the extract crate holds the tools, the opening and the step reader; `investigate` in the server job runs the loop for pairs the batch could not settle (decision 8) · cut 4 implemented: migration 0037 adds `knowledge_bases.governance_since`; `fuse` in the server job turns the switch off after two reverts of the agent's merges since it was turned on, within seven days, raises `governance.tripped` and writes the ledger (decision 9) · [0026](0026-a-decision-records-why.md) gives a decision its stated reason, hands the batch adjudicator the same precedents and quotes the reason into them · [0028](0028-the-adjudicator-looks-before-it-asks.md) opens the loop of cut 2 to the adjudicator with governance off and gives it a `consequences` tool
- **Written**: 2026-09-06 (conventions in the [README](README.md))
- **Related**: [0016](0016-close-the-open-seams-before-cutting-new-ones.md) C2 gave a base its first automation switch, `auto_type_resolution`, and this record copies its shape; #428 asked for bulk and automatic handling of same-name pairs and got the batch path (#429, #430) this builds on; [0020](0020-an-auditor-reads-it-without-us.md) made the ledger complete enough to be read back; [0015](0015-recording-a-sentence-is-not-asserting-a-fact.md) keeps a person's own sentences out of any machine's reach.

Expand Down Expand Up @@ -75,7 +75,7 @@ Both revert paths count: the Agent queue's Revert and the Merges page's Revert o

The first big-file run (decision 4's revision note) showed two things at once: a fresh base needs the agent to decide on its own, and what the model decides on its own is wrong in patterned ways. So on 2026-09-07 every pair the resolver raised on one governed import of eight interlocking articles (615 pairs, 411 distinct name pairs) was labeled by hand from each side's facts and source passages: `same`, `different`, or `unknown` where no reader could tell either (a bare "judge"). The set lives in `scripts/bench/truth/ai-timeline.duplicates.json`, keyed by the two names, and `scripts/bench/govern.mjs` resets a base's agent decisions without touching the ledger, runs the governor, and scores: decided on its own, agreed with the labels, wrong merges, wrong keeps, left for people.

The patterns became rules, written once (`IDENTITY_RULES`) and read by both prompts: a version, edition, tier, division, subsidiary or legal entity added at the end of a name is a different thing; a phrase that contains a name is not the name; a list is not its members; a common noun is not a proper name; a qualifier dropped from the front, a corporate suffix, a surname for a full name, a parenthetical acronym are the same thing; a different type label, a missing fact, a different partner or role is not a contradiction. Two of the rules are also mechanical, so the model cannot talk its way past them: `name_shape` classifies a pair as identical, abbreviation, suffix, version, extension, phrase or unrelated, and a `same` verdict on a version or a phrase is never applied on its own; `type_family` folds the extractor's labels into person, organization, place and event, and only a difference across families counts as a conflict. Identical names whose labels fall in one family are shown to the model with one label, since the label difference was the reason it gave most often for a wrong split; and a `different` verdict on identical, compatible names goes to the tool loop for a second look before it is applied.
The patterns became rules, written once (`IDENTITY_RULES`) and read by both prompts: a version, edition, tier, division, subsidiary or legal entity added at the end of a name is a different thing; a phrase that contains a name is not the name; a list is not its members; a common noun is not a proper name; a qualifier dropped from the front, a corporate suffix, a surname for a full name, a parenthetical acronym are the same thing; a different type label, a missing fact, a different partner or role is not a contradiction. Two of the rules are also mechanical, so the model cannot talk its way past them: `name_shape` classifies a pair as identical, abbreviation, suffix, version, extension, phrase or unrelated, and a `same` verdict on a version or a phrase is never applied on the first look alone; `type_family` folds the extractor's labels into person, organization, place and event, and only a difference across families counts as a conflict. Identical names whose labels fall in one family are shown to the model with one label, since the label difference was the reason it gave most often for a wrong split; and a `different` verdict on identical, compatible names goes to the tool loop for a second look before it is applied. Revised 2026-09-14 (#700): a `same` verdict on a version or a phrase goes to that second look too, and once the loop has read both sides' facts and text and still says `same`, the gate reads the pair without its shape. The shape raises a doubt, and only the evidence can settle it: "Lease Agreement dated May 16, 2016, as amended" and "Lease Agreement" are a phrase by shape and one lease by their facts. The identity rules gain the matching sentence: a document, agreement or filing stays one thing through its amendments, each amendment is its own document, and an agreement for other premises, another phase or other parties is a different one. On the labeled set, both runs reset and re-decided on the same extraction: agreement 96.3% → 98.1%, wrong merges 2 → 1, wrong keeps 4 → 2, pairs left for people 30 → 34.

The same 589 pairs, reset and re-run after each change:

Expand Down
5 changes: 4 additions & 1 deletion scripts/bench/govern.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,10 @@ async function fresh() {
await api("POST", `/api/v1/kbs/${kb}/ingest`, body);
}
await until(async () => {
const done = num(`SELECT count(*) FROM documents WHERE kb_id='${kb}' AND graph_status IN ('done','failed')`);
// 失败了但抽取任务还排着的(本体索引没嵌完、延后重试)不算完
const done = num(`SELECT count(*) FROM documents d WHERE d.kb_id='${kb}' AND (d.graph_status = 'done'
OR (d.graph_status = 'failed' AND NOT EXISTS (SELECT 1 FROM jobs j WHERE j.kind = 'extract_document'
AND j.status IN ('queued','running') AND j.payload->>'document_id' = d.id::text)))`);
if (done >= docs.length) return true;
const chunks = num(`SELECT count(*) FROM chunks WHERE kb_id='${kb}' AND extracted_at IS NOT NULL`);
log(` 抽取 ${chunks} 块 / ${done} 篇完成`);
Expand Down
Loading