A lease amendment chain answers as-of questions - #683
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: WaylandYang <wayland0916@gmail.com>
This was referenced Sep 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the lease-chain bench used to find and measure #679 and #680. It is scripts only; no filings are committed, the same as the NVIDIA corpus.
The recall bench (
recall.mjs) asks whether a fact reached the graph. This one asks whether, after a contract is rewritten by a chain of amendments, its terms answer correctly at a moment. The Blackbaud HQ lease's Phase 2 exercise deadline was extended five times in 2020 before the tenth amendment tied it to a trigger date. The property was sold to a Blackbaud affiliate that became the landlord, and the price was cut after inspection. There are 18 questions about those values on particular dates.What is added, under
scripts/bench/corpora/contracts/edgar-manifest.jsonlists 7 contract chains and 62 EDGAR filings. For each it records CIK, accession, filing date, role, and a title read from the document's opening.fetch.pyfetches them intoedgar/, as HTML plus plain text. It uses curl for the local proxy and needsSEC_USER_AGENT. The redacted CSG/Comcast chain is fetched only with--with-redacted.lease_bench.py setupcreates a fresh base and turns off the switches that edit the graph after extraction, asrecall.mjsdoes. It declares a generic lease and property-sale ontology, with single-valued terms marked functional, and ingests the 13 amendments (the 145-chunk base lease only with--with-base). It logs in likerecall.mjs:BENCH_BASE, andBENCH_TOKENorBENCH_EMAIL/BENCH_PASSWORD.lease_bench.py scoreanswers the questions the way the product reads. It takes each lease and purchase-agreement entity's facts and keeps those whose server-computedholds_from/holds_tocontains the moment. A question is right when the values holding then are exactly the true one.README.mdcovers how to run it, the truth table, and the 2026-09-13 runs..gitignoreignores the fetchededgar/,cuad/and the per-runlease-kb-*.json.A run is one fresh base and takes about 10 minutes. Runs vary by about 3 questions, so a build needs at least two.
🤖 Generated with Claude Code