Skip to content

fix: undated uploads using extraction-time recording anchors - #730

Closed
AaronProbha18 wants to merge 1 commit into
deeplethe:devfrom
AaronProbha18:fix/undated-upload-recording-anchor
Closed

AaronProbha18 wants to merge 1 commit into
deeplethe:devfrom
AaronProbha18:fix/undated-upload-recording-anchor

Conversation

@AaronProbha18

Copy link
Copy Markdown

What changed

documents.create stores doc_time = now() with doc_time_source = 'upload_time'
for an undated upload — a record-axis timestamp standing in a world-axis column.
extraction.rs read doc.doc_time at five sites and never looked at
doc_time_source, so an upload-time value was used both as "the document's own
date" in the prompt and as the recording anchor for every fact and name fact
extracted from it. A memo uploaded today that says "last year" resolved to last
year from today, and every fact it produced anchored on today.

The two sites want different things, so they stop sharing a value:

  • The prompt genuinely wants no date. prompt_doc_time is None when
    doc_time_source == "upload_time"; build_messages already handles that.
  • The anchor must never be absent. attest_earlier no-ops on None ("now
    can't be earlier than any existing evidence"), so a null anchor would leave
    close_with_unknown_end nothing to close against and an ended relation would
    go on reading as still holding. recording_anchor falls back to this run's
    Utc::now() — 0022 section 3's "the moment of recording", now meaning the
    moment of extraction rather than of upload. Computed once per run so every
    fact from one document shares an anchor.

All four anchor sites take Some(recording_anchor); no site reads
doc.doc_time directly any more. graph.rs, temporal.rs and names.rs are
unchanged — their logic was already right given a real anchor; the bug was
entirely in what extraction handed them. No schema change.

Dated documents (doc_time_source of content or source) behave exactly as
before.

0022 section 3 gets a dated revision note.

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up. The prompt half is exactly what #714 asked for. The anchor half needs another pass, and a few requirements from the issue thread are still missing.

1. Prompt: keep it. prompt_doc_time is None for upload_time, so a "last year" in an undated upload no longer resolves against the upload day. 👍

2. Anchor: please drop the Utc::now() fallback. Extraction time is still a transaction-axis timestamp, the same kind of value #714 is about. It just comes from a different clock.

  • In the normal flow, extraction runs minutes after upload, so the anchors barely change.
  • On re-extraction, the anchor drifts later.
  • The direction we've taken is recorded in 0045 (#724): attested time stays null for an undated document, and closing a timeline no longer depends on it. That work lands with 0045's cuts.
  • So please leave the four anchor sites as they are on dev for now, and remove the 0022 revision note (it would record the opposite direction).

3. Tests. Please add the ones asked for in the thread, scoped to what this PR now changes:

  • an undated upload (doc_time_source = 'upload_time') builds a prompt with no document date;
  • a dated document (content / source) still gets its date in the prompt.

4. CI.

  • cargo fmt --all --check fails on the validity_of call.
  • The commit has no Signed-off-by, so DCO is red. git commit --amend -s and a force-push will fix it.

With 1–4 done this is a small, safe merge.

@AaronProbha18

Copy link
Copy Markdown
Author

All four done — thanks, the 0045 pointer is the part I'd missed.

1. Kept, and pulled out into prompt_doc_date(doc_time, doc_time_source)
so it's testable without a document. upload_timeNone, everything else →
the formatted date. build_messages already says the date is unknown on None,
so nothing changed there.

2. Dropped. The four anchor sites are back to doc.doc_time exactly as on
dev, and the 0022 revision note is gone (git diff dev -- docs/ is empty).
You're right that extraction time is the same kind of value — it's a different
clock, not a different axis, and on re-extraction it drifts later for no reason
the ledger can defend. This PR is now the prompt half only.

3. Two unit tests beside it:

  • an_undated_upload_gives_the_prompt_no_dateupload_time with a real
    doc_time yields no date;
  • a_dated_document_still_carries_its_date_into_the_promptcontent and
    source both keep 2024-02-20, and a genuinely dateless document still
    gives None.

Extracting the helper was mostly so these could be plain unit tests rather than
standing up a document and an LLM for a one-line branch.

4. The validity_of call is back to its dev form, so the fmt failure goes
with it. Commit is signed off — I squashed the DCO remediation commit into the
real one rather than leaving both, so the branch is a single commit with one
Signed-off-by. Force-pushed.

`documents.create` writes `doc_time = now()` with `doc_time_source =
'upload_time'` for an upload that carries no date of its own. Extraction read
`doc.doc_time` straight into the prompt, so the model was told an undated memo
was written on the day it was uploaded, and a "last year" in it resolved
against that day (deeplethe#714).

`prompt_doc_date` gates the prompt's date on the source: `upload_time` means
no date, and `build_messages` already says the date is unknown when it gets
None. Dated documents (`content` / `source`) are unchanged.

The recording-axis anchors are left alone. Extraction time is a transaction-axis
value too, only from a different clock, and 0045 (deeplethe#724) has the direction for
those: attested time stays null for an undated document, and closing a timeline
stops depending on it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Aaron Personal <aaronprobha@gmail.com>
@AaronProbha18
AaronProbha18 force-pushed the fix/undated-upload-recording-anchor branch from 5290ccf to 13c0ffa Compare September 17, 2026 06:16
@WaylandYang

Copy link
Copy Markdown
Contributor

Thank you for this. It is superseded by #740 (0045): the open contract no longer sends any document date to the model, and migration 0064 sets doc_time to NULL with source none for every document whose only date was the upload time or the file modification time, so #714 is closed there at the ledger rather than at the prompt. The typed path this patch edits (extraction::run) is deleted in #736. Closing in favour of that stack.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants