Skip to content

Editing an existing ontology definition leaves negative alignment bindings cached #773

Description

@Maya-Kid

Editing an existing ontology definition leaves negative alignment bindings cached

Verified against dev@03417131d151bbb9454210461f236995c78a03ea on Linux x86_64 with Rust 1.98.1, PostgreSQL 16.15 and pgvector 0.8.6. All 69 migrations ran successfully on a fresh isolated database; a second migration run succeeded. No live model was used.

Observed behavior

After a kind word or phrase signature has an automatic none or undecided binding, editing an existing class/property description does not mark that binding stale. The ontology edit already queues alignment; the old negative binding is excluded from its work list.

Both type_bindings::stale and phrase_bindings::stale use:

bound: selected definition.updated_at > binding.decided_at
none / undecided: max(definition.created_at) > binding.decided_at

The existing-key filter in type alignment and phrase alignment requires a non-person binding and membership in the stale set.

Deterministic reproduction

  1. Create an isolated KB, an existing definition, entities and an open statement. Read signatures through the store API.
  2. Use the real decide API to record an automatic none or undecided result.
  3. Order the fixture timestamps explicitly: definition creation/update at 2000-01-01, binding decision at 2000-01-02.
  4. Call the real ontology::update_entity_type or ontology::update_relation_type, changing the description without creating a new definition. Assert that creation stays before the decision and update becomes later.
  5. Call the real stale API and apply the existing-key work-list predicate.

The four regression expectations fail on this baseline:

class_edit_reconsiders_none          FAILED
class_edit_reconsiders_undecided     FAILED
property_edit_reconsiders_none      FAILED
property_edit_reconsiders_undecided FAILED
actual in each case: stale=false, equivalent_todo=false

The controls pass: positive bindings become stale after definition edits; both negative statuses become stale after new definitions are added; person decisions stay excluded from automatic work and reject agent overwrites. The existing a_kind_word_binds_to_a_class integration test also passes, including its human entity-type protections.

Save the reproduction below as crates/utopia-store/tests/negative_binding_definition_edit.rs and run it after the normal migrations:

UTOPIA_TEST_REQUIRE_DB=1 UTOPIA_DATABASE_URL='<isolated-db-url>' \
  cargo test --locked -p utopia-store --test negative_binding_definition_edit -- --nocapture

This executes Rust/PostgreSQL store paths and the equivalent work-list predicate. It does not execute the complete background worker or prove an eventual semantic binding.

Expected behavior and scope

Changing a relevant semantic definition should give old automatic negative bindings another evaluation opportunity. The result may correctly remain none. Person bindings and human entity-type decisions must remain protected.

As a diagnostic only, changing the two negative branches from max(created_at) to max(updated_at) makes all five new test functions pass and preserves the existing integration test. That experiment was reverted. It does not establish a complete invalidation design: timestamp-based invalidation may also rerun after cosmetic edits and does not address definitions changing during an in-flight model call.

Is the existing negative-cache lifetime intentional, or should definition edits invalidate these results? A narrow fix could cover this transition while reusing #741, #751, #754 and #757. Broader input revision tracking and observation-ledger work can stay with the separate design discussion around #725.

Complete regression test (Rust)
//! Regression expectations for editing an existing definition, without an LLM.
use sqlx::PgPool;
use utopia_store::graph::FactObject;
use utopia_store::{graph, ontology, phrase_bindings as pb, type_bindings as tb};
use uuid::Uuid;

async fn lifecycle(phrase: bool, status: &str, add: bool, person: bool) -> anyhow::Result<()> {
    let Some(url) = utopia_store::test_db::url() else {
        return Ok(());
    };
    let pool = PgPool::connect(&url).await?;
    let (org, ws, kb, definition) = (
        Uuid::now_v7(),
        Uuid::now_v7(),
        Uuid::now_v7(),
        Uuid::now_v7(),
    );
    sqlx::query("INSERT INTO organizations (id,name) VALUES ($1,'negative binding audit')")
        .bind(org)
        .execute(&pool)
        .await?;
    let result = async {
        sqlx::query("INSERT INTO workspaces (id,org_id,name) VALUES ($1,$2,'audit')").bind(ws).bind(org).execute(&pool).await?;
        sqlx::query("INSERT INTO knowledge_bases (id,workspace_id,name) VALUES ($1,$2,'audit')").bind(kb).bind(ws).execute(&pool).await?;
        let table = if phrase { "relation_types" } else { "entity_types" };
        // Explicit historical timestamps order the fixture without sleeps or clock races.
        sqlx::query(&format!("INSERT INTO {table} (id,kb_id,key,label,description,created_at,updated_at) VALUES ($1,$2,'candidate','candidate','old definition','2000-01-01','2000-01-01')"))
            .bind(definition).bind(kb).execute(&pool).await?;
        let (subject, object) = (Uuid::now_v7(), Uuid::now_v7());
        for entity in [subject, object] {
            sqlx::query("INSERT INTO entities (id,kb_id,canonical_name,specific_type) VALUES ($1,$2,$3,'candidate wording')")
                .bind(entity).bind(kb).bind(entity.to_string()).execute(&pool).await?;
        }
        graph::insert_open_statement(&pool,kb,subject,"candidate wording",FactObject::Entity(object),None,0.9).await?;
        let sig = pb::signatures(&pool,kb).await?.into_iter().find(|s| s.phrase=="candidate wording").expect("real open statement produces a signature");
        assert!(tb::signatures(&pool,kb).await?.iter().any(|s| s.kind_word=="candidate wording"));
        let votes = serde_json::json!({});
        let chosen = (status == "bound").then_some(definition);
        let actor = if person { "person" } else { "agent" };
        if phrase {
            assert!(pb::decide(&pool,kb,&sig,pb::Decision { relation_type_id: chosen, direction: chosen.map(|_| "forward"), status, votes: &votes, decided_by: actor }).await?);
        } else {
            assert!(tb::decide(&pool,kb,"candidate wording",&[],chosen,status,&votes,actor).await?);
        }
        let bindings = if phrase { "phrase_bindings" } else { "type_bindings" };
        sqlx::query(&format!("UPDATE {bindings} SET decided_at='2000-01-02' WHERE kb_id=$1")).bind(kb).execute(&pool).await?;
        if phrase { assert!(pb::stale(&pool,kb).await?.is_empty()); }
        else { assert!(tb::stale(&pool,kb).await?.is_empty()); }

        if add {
            sqlx::query(&format!("INSERT INTO {table} (id,kb_id,key,label) VALUES ($1,$2,'new','new')"))
                .bind(Uuid::now_v7()).bind(kb).execute(&pool).await?;
        } else if phrase {
            ontology::update_relation_type(&pool,kb,definition,"candidate","state",Default::default(),"expanded definition",None,None,None,None).await?;
        } else {
            ontology::update_entity_type(&pool,kb,definition,"candidate",None,"circle",&[],"expanded definition").await?;
        }
        let (created_before, updated_after): (bool,bool) = sqlx::query_as(&format!("SELECT created_at < '2000-01-02'::timestamptz, updated_at > '2000-01-02'::timestamptz FROM {table} WHERE id=$1"))
            .bind(definition).fetch_one(&pool).await?;
        assert!(created_before);
        assert_eq!(updated_after,!add);
        let (stale,todo) = if phrase {
            let stale = pb::stale(&pool,kb).await?.iter().any(|b| b.key()==sig.key());
            let existing = pb::bindings(&pool,kb).await?;
            let binding = existing.iter().find(|b| b.key()==sig.key()).unwrap();
            (stale,binding.decided_by!="person" && stale)
        } else {
            let stale = tb::stale(&pool,kb).await?.contains(&"candidate wording".to_string());
            let existing = tb::bindings(&pool,kb).await?;
            let binding = existing.iter().find(|b| b.kind_word=="candidate wording").unwrap();
            (stale,binding.decided_by!="person" && stale)
        };
        eprintln!("DB ASSERTION phrase={phrase} status={status} add={add} person={person}: stale={stale}, equivalent_todo={todo}");
        if person {
            assert!(!todo);
            let changed = if phrase {
                pb::decide(&pool,kb,&sig,pb::Decision { relation_type_id: Some(definition),direction: Some("forward"),status:"bound",votes:&votes,decided_by:"agent" }).await?
            } else { tb::decide(&pool,kb,"candidate wording",&[],Some(definition),"bound",&votes,"agent").await? };
            assert!(!changed,"automatic decision must not overwrite a person");
        } else {
            anyhow::ensure!(stale && todo,"changed definition must make {status} eligible: stale={stale}, todo={todo}");
        }
        Ok::<_,anyhow::Error>(())
    }.await;
    sqlx::query("DELETE FROM organizations WHERE id=$1")
        .bind(org)
        .execute(&pool)
        .await?;
    result
}

#[tokio::test]
async fn controls() -> anyhow::Result<()> {
    for phrase in [false, true] {
        lifecycle(phrase, "bound", false, false).await?;
        for status in ["none", "undecided"] {
            lifecycle(phrase, status, true, false).await?;
            lifecycle(phrase, status, true, true).await?;
            lifecycle(phrase, status, false, true).await?;
        }
    }
    Ok(())
}

#[tokio::test]
async fn class_edit_reconsiders_none() -> anyhow::Result<()> {
    lifecycle(false, "none", false, false).await
}
#[tokio::test]
async fn class_edit_reconsiders_undecided() -> anyhow::Result<()> {
    lifecycle(false, "undecided", false, false).await
}
#[tokio::test]
async fn property_edit_reconsiders_none() -> anyhow::Result<()> {
    lifecycle(true, "none", false, false).await
}
#[tokio::test]
async fn property_edit_reconsiders_undecided() -> anyhow::Result<()> {
    lifecycle(true, "undecided", false, false).await
}
Baseline regression output
running 5 tests
DB ASSERTION phrase=false status=bound add=false person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=false status=none add=false person=false: stale=false, equivalent_todo=false
DB ASSERTION phrase=true status=none add=false person=false: stale=false, equivalent_todo=false
DB ASSERTION phrase=false status=undecided add=false person=false: stale=false, equivalent_todo=false
Error: changed definition must make none eligible: stale=false, todo=false
test class_edit_reconsiders_none ... FAILED
Error: changed definition must make undecided eligible: stale=false, todo=false
test class_edit_reconsiders_undecided ... FAILED
Error: changed definition must make none eligible: stale=false, todo=false
test property_edit_reconsiders_none ... FAILED
DB ASSERTION phrase=false status=none add=true person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=true status=undecided add=false person=false: stale=false, equivalent_todo=false
Error: changed definition must make undecided eligible: stale=false, todo=false
test property_edit_reconsiders_undecided ... FAILED
DB ASSERTION phrase=false status=none add=true person=true: stale=true, equivalent_todo=false
DB ASSERTION phrase=false status=none add=false person=true: stale=false, equivalent_todo=false
DB ASSERTION phrase=false status=undecided add=true person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=false status=undecided add=true person=true: stale=true, equivalent_todo=false
DB ASSERTION phrase=false status=undecided add=false person=true: stale=false, equivalent_todo=false
DB ASSERTION phrase=true status=bound add=false person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=true status=none add=true person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=true status=none add=true person=true: stale=true, equivalent_todo=false
DB ASSERTION phrase=true status=none add=false person=true: stale=false, equivalent_todo=false
DB ASSERTION phrase=true status=undecided add=true person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=true status=undecided add=true person=true: stale=true, equivalent_todo=false
DB ASSERTION phrase=true status=undecided add=false person=true: stale=false, equivalent_todo=false
test controls ... ok

failures:

failures:
    class_edit_reconsiders_none
    class_edit_reconsiders_undecided
    property_edit_reconsiders_none
    property_edit_reconsiders_undecided

test result: FAILED. 1 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.97s

error: test failed, to rerun pass `-p utopia-store --test negative_binding_definition_edit`
Diagnostic predicate substitution output
running 5 tests
DB ASSERTION phrase=false status=none add=false person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=true status=none add=false person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=false status=undecided add=false person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=false status=bound add=false person=false: stale=true, equivalent_todo=true
test property_edit_reconsiders_none ... ok
test class_edit_reconsiders_undecided ... ok
test class_edit_reconsiders_none ... ok
DB ASSERTION phrase=false status=none add=true person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=true status=undecided add=false person=false: stale=true, equivalent_todo=true
test property_edit_reconsiders_undecided ... ok
DB ASSERTION phrase=false status=none add=true person=true: stale=true, equivalent_todo=false
DB ASSERTION phrase=false status=none add=false person=true: stale=true, equivalent_todo=false
DB ASSERTION phrase=false status=undecided add=true person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=false status=undecided add=true person=true: stale=true, equivalent_todo=false
DB ASSERTION phrase=false status=undecided add=false person=true: stale=true, equivalent_todo=false
DB ASSERTION phrase=true status=bound add=false person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=true status=none add=true person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=true status=none add=true person=true: stale=true, equivalent_todo=false
DB ASSERTION phrase=true status=none add=false person=true: stale=true, equivalent_todo=false
DB ASSERTION phrase=true status=undecided add=true person=false: stale=true, equivalent_todo=true
DB ASSERTION phrase=true status=undecided add=true person=true: stale=true, equivalent_todo=false
DB ASSERTION phrase=true status=undecided add=false person=true: stale=true, equivalent_todo=false
test controls ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.97s

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions