Uh oh!
There was an error while loading. Please reload this page.
review: 0.2.9–0.2.11 — everything since 0.2.8: local mode, agent tools, local chat, Flash, two-sided client - #400
Open
rejojer wants to merge 186 commits into
Open
review: 0.2.9–0.2.11 — everything since 0.2.8: local mode, agent tools, local chat, Flash, two-sided client#400rejojer wants to merge 186 commits into
rejojer wants to merge 186 commits into
Conversation
* feat: add local mode to the PageIndex SDK client
One PageIndexClient, two backends. With api_key: the 0.2.x cloud SDK,
request for request (with the reviewed fixes: bounded timeouts on JSON
endpoints, none on uploads, URL-encoded ids, 401 key hint, empty DELETE
body tolerated). Without api_key: the same methods run locally —
page_index builds the tree in submit_document (mode="flash" uses
PageIndex Flash), documents are stored as plain JSON per doc under
storage_path, submit_query is LLM tree search with retrieve_model, and
chat_completions answers over the retrieved nodes with OpenAI-style
responses and streaming.
Local responses mirror the cloud wire shapes verified against the server
source: tree nodes rename start_index to page_index and drop end_index,
a non-leaf summary becomes prefix_summary, and the metadata/list/delete/
retrieval envelopes match key for key. Cloud-only features (folders,
beta_headers, enable_citations) raise instead of pretending.
Replaces the demo-only workspace client (index/get_document_structure/
get_page_content had no real users) and its retrieve.py helpers.
page_index_main gains an optional logger param so the SDK can keep
./logs out of the caller's working directory; pymupdf import is now lazy
(only the optional PyMuPDF parser path needs it).
* chore: package pageindex 0.3.0.dev4 for PyPI
Poetry packaging for the combined SDK + local pipeline: every production
import is a declared dependency (openai and requests join requirements.txt
for the same reason), config.yaml and the flash data tables ship in the
wheel, the benchmark PNG does not. pymupdf drops to an optional note now
that its import is lazy. dev4 follows the already-published 0.3.0.dev1-3;
pip still resolves plain 'pip install pageindex' to 0.2.8 until a final
0.3.0 — install with --pre.
* docs: add SDK section to README; move the agentic demo onto the SDK
The demo keeps its flow and the post-cutoff demo paper, swapping the
removed workspace client for PageIndexClient local mode (list_documents
for the doc-id cache, get_tree/get_ocr behind the agent tools). The old
examples/workspace JSONs demoed the removed format and go with it.
* refactor: rebuild cloud_api on the 0.2.8 client text
Ray's rule for the cloud half: his 0.2.8 code is the base; a Kylin-lineage
change survives only when strictly better — invisible on healthy traffic
while fixing a real failure mode. Kept under that bar: request timeouts
(dead connections hung forever; uploads still pass none, exactly like
0.2.8), the upload handle closed via with (leaked on request errors),
URL-encoded path ids (a crafted id could reroute the URL), the
empty-DELETE-body guard, and stream hardening (choices guard,
response.close in finally). Reverted as not strictly better: the
lowercase summary param (the server accepts both spellings), the 401
message hint (visible text change; AUTH_HINT dropped from errors.py), and
the _request/requests.request reorganization — every method body,
docstring, and section comment is 0.2.8's text again.
diff -w against ../pageindex_sdk/pageindex/client.py now reads as that
surgical patch plus plumbing: CloudAPI reads BASE_URL/api_key through the
owning client, PageIndexAPIError comes from errors.py, and
is_retrieval_ready lives verbatim on PageIndexClient shared by both
modes. A mocked-requests harness driving 0.2.8 and this file through 19
identical calls shows the only remaining request-level difference is
timeout.
* refactor: drop the local retrieval endpoints — cloud-only, deprecated
Ray: the cloud already marks POST /retrieval/ and GET /retrieval/{id}/
deprecated in favor of chat completions, so local mode should not grow a
fresh implementation of a retiring surface. submit_query/get_retrieval
now raise in local mode with a pointer to chat_completions; cloud mode is
untouched (the endpoint still works there and 0.2.8 code keeps running).
The tree search that backed them stays as chat_completions' retrieval
engine; the retrievals/ storage goes away. This also closes the one real
cross-mode parity gap — the retrieved_nodes inner shape — by removing
its local half.
* feat: manifest.json — one-file document listings for the local store
Ray wanted a metafile that shows every document in one place instead of
per-directory reads. It is a cache, never a second source of truth:
writers update it best-effort after save/delete (atomic replace, no
locks), and list_metas trusts it only while its id set matches the docs/
directory names — documents are immutable, so matching names imply valid
content. Any mismatch (lost concurrent update, crash, corrupt or deleted
manifest) rebuilds it from the doc.json files, reading only the missing
entries. Incomplete dirs (no doc.json) stay invisible and are never
recorded, so a save that completes later is still picked up.
1000-doc listing: 37ms of per-dir reads -> 2.3ms warm (scandir names +
one manifest read); one-time rebuild 160ms.
* fix: align local doc_id prefix and createdAt format with the cloud
Local doc ids now carry the cloud's pi- namespace prefix (random token
stays uuid4 hex — nothing parses cuid internals, the prefix is the
contract; chat ids already mirrored chatcmpl-). createdAt now matches
the server byte for byte: the cloud emits the DB datetime's bare
isoformat — naive UTC, second precision — while we emitted microseconds
plus +00:00.
* feat: optional metadata tags on submit_document (both modes)
Ray's call after the alignment review: the metadata key in tree/OCR
envelopes and list entries should carry real data, and the only honest
way is exposing the field the server already accepts. Cloud mode
forwards it as the existing metadata form field; local mode validates it
early (a JSON-serializable dict, checked before any LLM spend), stores
it in doc.json, and returns it from the same three places. get_document
still omits it, mirroring the server, whose metadata-endpoint SQL never
selects that column. Scope stays deliberately narrow: set at submit and
read back — no metadata_filter, no update API.
* docs: state that createdAt is UTC and show how to localize it
The value is naive UTC in both modes (the cloud column is timestamp
DEFAULT CURRENT_TIMESTAMP on a UTC server, emitted via bare isoformat).
Wall-clock display is the consumer's layer: emitting local time under
the same format would silently change meaning per machine, and adding an
offset marker would break both the byte-format parity and string-order
sorting.
* fix: createdAt carries milliseconds, matching the cloud's datetime(3)
The earlier second-precision alignment was reasoned from the postgres
schema file, but production is MySQL (DATABASE_BACKEND defaults to
mysql) and its FilePageIndex.createdAt is datetime(3) DEFAULT
CURRENT_TIMESTAMP(3) — so the server isoformat()s a millisecond-
precision naive-UTC datetime, emitting .XXX000 fractions (bare seconds
only when the millisecond happens to be zero). Local now generates
through the same mechanism. The docs' 2024-01-15T10:30:00.000Z sample is
a JS-style string Python isoformat cannot produce — not evidence.
* fix: createdAt at millisecond precision, matching the timestamp(3) column
The cloud column is timestamp(3); its datetimes render through bare
isoformat() as six fractional digits ending in 000 (or no fraction when
the millisecond is exactly zero). Truncate to the millisecond and render
the same way, replacing the second-precision guess from cd44ef0. Worth
one confirmation against a live cloud response when a key is around.
* chore: trim non-essential comments
Alignment narration (mirrors the cloud, matches 0.2.8, trailing-period
notes) moves out of the code — that rationale lives in the commit
history. Kept only constraints the code can't show: the storage commit
protocol and manifest trust rule, the id-escape guard, the silent
logger's reason to exist, the client-reference indirection, and the
tree-node reshape spec.
* fix: close the local_store crash and corruption holes found in review
Adversarial review reproduced three edge holes in the no-lock design:
a torn delete (doc.json unlinked, rmtree unfinished) left a ghost the
manifest kept listing forever; one truncated doc.json crashed every
listing with a raw JSONDecodeError; and two concurrent deletes of the
same id could crash the loser's rmtree.
doc.json is now the existence marker in both directions — written last
on save, unlinked first on delete — and list_metas serves an entry only
after confirming it still exists, instead of trusting the manifest/dir
name-set proxy. Atomic writes fsync before replace, closing the
power-loss truncation window. An unreadable JSON file is logged and
treated as absent; a document meta additionally falls back to the
manifest copy (immutable docs make the cache a valid replica). rmtree
runs with ignore_errors: the commit point has passed and cleanup is
best-effort, which also absorbs the double-delete race.
Also restores the reversed-page-range error in the demo's page parser
(silently empty since the retrieve.py removal). Warm 1000-doc listing
goes 2.3ms -> 8.3ms for the per-doc existence check; full parse remains
37ms.
* docs: correct two docstring claims and the pymupdf note
is_retrieval_ready reports only API errors as False — transport errors
propagate; delete_document may return {} on an empty cloud body; pymupdf
is also used by tree_optimize's page loading, not just get_page_tokens.
* chore: target 0.2.9 for the local-mode release
Ray's call: 0.2.x stays the no-collections line, so local mode ships as
0.2.9 and 0.3.0 stays reserved; plain pip installs never see the
0.3.0.devN pre-releases, and the cookbooks' existing 'pip install
--upgrade pageindex' will deliver the new SDK without any --pre
instructions.
* ci: publish to PyPI on version tags
Tag-driven releases: the pushed v-tag is the single source of truth for
the version — validated as PEP 440, injected into pyproject, built, and
published via OIDC trusted publishing with no stored credentials; a
GitHub Release with the artifacts is created alongside.
* chore: tighten the store docstring to essentials
* fix: contain invalid-UTF-8 corruption; fail loud on unreadable data files
The corruption guard caught JSONDecodeError but not the
UnicodeDecodeError a torn multi-byte write produces — the exact scenario
the guard targets whenever names or descriptions carry non-ASCII text —
so that flavor crashed listings and gets raw, and regressed the old
manifest read's broader ValueError guard. _read_json now catches
ValueError, which covers both.
Unreadable tree.json/pages.json under an intact doc.json previously
served an empty tree with retrieval_ready true — a silent lie; those
paths now raise 'stored document data is unreadable' and
is_retrieval_ready honestly reports False. delete_document survives a
doc.json tampered into a directory (cleans it, reports not-found)
while real unlink failures such as permissions stay loud.
* feat: LocalClient and CloudClient for explicit mode selection
PageIndexClient(api_key=os.getenv(...)) with an unset variable gets None
and silently falls into local mode — methods keep working against local
storage on the caller's own LLM bill, the silent mode flip the
empty-string guard can't see. The explicit classes close it at the type
level: CloudClient raises on a missing key, LocalClient has no api_key
parameter at all. Names per Ray.
* feat: explicit-mode clients PageIndexCloudClient and PageIndexLocalClient
PageIndexClient(api_key=os.getenv(...)) with an unset variable yields
None and silently lands in local mode — with both modes fully working,
that's a silent mode flip onto the user's own LLM bill. The explicit
classes pin the mode at construction: the cloud one refuses a missing or
empty key, the local one has no key parameter at all. Names follow the
package's PageIndex- prefix convention.
* fix local mode edge cases
* fix: keep the litellm/ prefix normalization the demo depends on
a108c02 added _normalize_retrieve_model because the agentic demo hands
client.retrieve_model straight to the OpenAI Agents SDK, which routes a
non-OpenAI provider only when the name carries a litellm/ prefix. The
normalization sat in client.py, so the demo line never had to change --
and this rewrite dropped the helper while leaving that lone consumer
untouched. A retrieve_model like anthropic/claude-sonnet-4-6, the form
config.yaml documents, then died at Agent() with "Unknown prefix".
Local mode is indifferent to which form it gets: llm_completion and
_chat_llm both removeprefix("litellm/"), count_tokens returns the same
count either way, and _is_openai_model classifies both as LiteLLM, so
_require_llm_key still asks for no OPENAI_API_KEY. Models without a
provider path (the packaged gpt-5.4 default) pass through untouched.
* fix: restore the published 0.2.8 helper signatures the cookbooks call
pyproject.toml makes this repo the source of the PyPI pageindex package,
so this utils.py replaces the published 0.2.8 one -- whose helpers are
the documented surface of the cookbook notebooks. Three had drifted:
remove_fields lost max_len, create_node_mapping lost
include_page_ranges/max_page, print_tree lost exclude_fields. Both
README-linked notebooks open with `pip install --upgrade pageindex` and
pass exactly those kwargs, so tagging v0.2.9 as-is would TypeError
every Colab run of vision_RAG_pageindex.ipynb.
remove_fields and create_node_mapping readopt the 0.2.8 bodies, strict
supersets of the current ones (no in-repo caller passes the new params).
print_tree keeps the outline view as its default and routes an explicit
exclude_fields= to the 0.2.8 pprint view -- the two versions disagree
on what the second positional means (indent vs exclude_fields), and the
notebooks pass it by keyword. call_llm, the fifth published name, stays
out: nothing imports it -- the one notebook using a call_llm defines
its own, with a different signature.
* perf: resolve the indexing stack lazily from pageindex/__init__
`import pageindex` eagerly pulled page_index, flash, and tree_optimize
-- 0.73s warm, numpy and pypdfium2 in-process -- while the published
0.2.8 package imported in 0.08s on requests+openai alone. A cloud-only
SDK user upgrading to 0.2.9 would pay for an indexing stack they never
call, on every interpreter start.
The eager surface shrinks to client and errors (2ms warm); everything
else resolves on first attribute access via PEP 562 and is cached in
the module namespace. `pageindex.page_index` stays the function, still
shadowing its submodule as the old star-import had it. __all__ now
names the public surface, so `from pageindex import *` binds the same
working set as before instead of 124 names including stdlib modules.
A TYPE_CHECKING block keeps real signatures visible to IDEs.
The test suite's sys.modules lookup assumed the eager import chain;
it now imports pageindex.page_index explicitly.
* fix: wrap PDF read failures in PageIndexAPIError on local submit
_extract_page_texts sat one line outside the try that wraps everything
else in submit_document, so a corrupt PDF surfaced as a raw
PyPDF2.errors.PdfReadError and a password-protected one as
FileNotDecryptedError -- while a blank PDF, checked on the very next
line, got a clean PageIndexAPIError. Callers handling the SDK error
type crashed on exactly the malformed downloads and encrypted files
an ingest loop sees most.
The extraction gets its own wrap rather than joining the indexer try
below, whose except would re-prefix the blank-PDF error into "Failed
to submit document: Failed to submit document: ...". FileNotFoundError
stays native, asserted by test_submit_rejections as cloud parity.
* fix: address code review findings across local mode and publish workflow
- _parse_json_reply: switch from extract_json to _reply_json (dead
try/except, global None→null substitution, wrong error messages)
- client.py: config override filter uses `is not None` instead of
truthiness, so empty-string model args are no longer silently dropped
- llm_completion/llm_acompletion: raise RuntimeError after retries
exhausted instead of returning empty string
- _require_llm_key: extend to anthropic/gemini/mistral providers
- _chat_llm: reuse _openai_sync_client singleton from utils
- _tree_search: drop redundant deepcopy before non-mutating remove_fields
- _stream_chunks: move final chunk inside try so usage data is reachable;
close stays in finally
- _validate_chat_messages: accept system messages, merge them into the
internal system prompt for cloud/local parity
- _build_chat_context: accept pre-read metas and pass structure through
to _tree_search, eliminating double get_meta and double tree.json reads
- _index_standard: reuse ConfigLoader from construction; reject empty
structure (parity with flash mode)
- extract_json: bare except: → except Exception: (no longer swallows
KeyboardInterrupt)
- Replace all str.removeprefix() with _strip_prefix() helper to restore
Python 3.7 compatibility; pyproject.toml back to python >= 3.7
- publish.yml: add test job (py3.10 + py3.13) gating the publish job
- Remove unused bare `import pageindex` from tests
* fix: tighten CI permissions, timestamp format, import style, and docstring
- publish.yml: add permissions: {contents: read} to the test job
- local_api: emit 3-digit ms timestamps via isoformat(timespec="milliseconds")
- local_api: use relative import for pageindex.utils in _chat_llm
- local_api: note the node_summary gate in _format_tree_node docstring
- test_client: update createdAt regex to match the new 3-digit format
* fix: accept GOOGLE_API_KEY for Gemini; mark pre-releases in GitHub
- _require_llm_key: Gemini provider now accepts either GEMINI_API_KEY
or GOOGLE_API_KEY (LiteLLM supports both)
- publish.yml: set prerelease flag on rc/dev/alpha/beta tags so they
are not shown as regular releases on GitHub
* fix: preserve print_tree backward compat with 0.2.8 positional call
Detect list passed as second positional arg (old 0.2.8 signature) and
treat it as exclude_fields instead of indent.
* refactor: print_tree param order — exclude_fields second for 0.2.8 compat
Move exclude_fields back to the second position (matching 0.2.8) instead
of detecting list-as-indent. Recursive call uses indent= keyword arg.
* refactor: remove _require_llm_key pre-check entirely
Let OpenAI SDK and LiteLLM report their own missing-key errors instead
of maintaining a parallel provider-to-env-var map. The except Exception
wrapper in chat_completions already converts these to PageIndexAPIError.
* fix: let LLM provider errors propagate instead of wrapping them
OpenAI/litellm auth, rate-limit, and other provider errors now reach the
caller as their original type (e.g. openai.AuthenticationError) instead
of being wrapped in PageIndexAPIError. PageIndexAPIError stays reserved
for PageIndex's own errors (bad doc_id, invalid params, indexing failures).
* refactor: catch only RuntimeError instead of isinstance check on openai
Only our own _tree_search logic raises RuntimeError (bad JSON, missing
node_list). Provider errors and unexpected bugs propagate naturally.
* fix: restore createdAt to 6-digit .177000 format matching cloud DATETIME(3)
Revert the timespec="milliseconds" that a linter introduced — it output
.177 (3 digits) while the cloud server's isoformat() outputs .177000
(6 digits). Verified against pageindex-compute server/lib/db/planet.py:
DATETIME(3) column + bare isoformat() = .177000.
* fix: resolve 15 review findings from PR #389
- Rename page_index.py → page_index_classic.py to fix __getattr__
shadowing (function permanently replaced by submodule after import)
- Add return_exceptions=True to verify_toc, generate_summaries, and
summarize_tree gathers so one LLM failure doesn't abort the batch
- Gracefully degrade generate_doc_description to "" on failure instead
of discarding the entire completed index
- Normalize file_path with str()/expanduser/abspath to support
pathlib.Path and tilde paths
- Strip text from tree.json at save time; reconstruct from pages.json
on read via _load_tree_with_text
- Filter empty-string model overrides in PageIndexClient constructor
- Guard empty choices list before indexing response.choices[0]
- Wrap streaming iteration errors as PageIndexAPIError inside the
generator
- Catch PermissionError in _read_json alongside FileNotFoundError
- Set max_retries=3 for OpenAI and num_retries=3 for litellm in
_chat_llm to match the retry behavior of the indexing path
- Replace _SilentLogger with logging.getLogger(__name__)
- Add .github/workflows/tests.yml for PR and push-to-main test runs
* fix: close publish workflow injection and detect flash silent summary failure
1. publish.yml: pass VERSION via os.environ instead of shell interpolation
into python -c, eliminating the command injection vector.
2. summarize_tree: raise RuntimeError when every node's summary generation
fails (e.g. missing LLM credentials), instead of silently saving a
document with all-empty summaries marked as completed.
* refactor: make chat_completions cloud-only until agent-based local chat lands
The local implementation was a tree-search RAG engine (retrieve prompt +
context stuffing) that diverged from the cloud /chat/completions design,
where an agent navigates documents through MCP tools. Rather than ship
the divergent engine in 0.2.9, remove it; local chat returns as an agent
loop built on the agent-tools layer in the 0.2.10 line.
Also from the PR #389 review:
- get_tree fails loud on unreadable pages.json instead of silently
serving textless nodes
- drop the wasted deepcopy in get_tree (_format_tree_node builds new dicts)
- generate_doc_description catches only RuntimeError so provider errors
(bad key, unknown model) propagate instead of storing ""
- _read_json treats IsADirectoryError as unreadable
- list_metas skips directory names that fail _is_safe_id
- constructing with api_key plus local-only args raises PageIndexAPIError
(was ValueError) to match the empty-api_key path
* fix: verification follow-ups for the chat removal commit
- retrieve_model docstring no longer promises the deleted tree-search
machinery; the param is reserved for the coming agent-based local chat
- empty pages.json ([]) fails loud like unreadable pages: a stored
document can never legitimately have zero pages, and get_tree's
"nodes always carry text" promise held only for the None case
- README: chat example moved to its own cloud-only block so the local
quickstart no longer ends in a raise
- tests: pin IsADirectoryError loud-fail, list_metas unsafe-name skip,
empty-pages loud-fail, and generate_doc_description's swallow-vs-
propagate boundary
* fix: detect classic-path silent summary failure like flash already does
generate_summaries_for_structure absorbed every per-node error to
summary: "" with no systemic check, so a bad key or model name produced
an all-empty-summary tree with zero errors on the default CLI config
(if_add_node_summary: yes, if_add_doc_description: no). Mirror flash's
summarize_tree guard: partial failures still absorb, all-failed raises.
* fix: accurate wording for retrieve_model docstring and empty-pages error
- retrieve_model is not "unused": the agent demo drives its model from
client.retrieve_model today; say so instead
- an empty pages.json is invalid, not unreadable — dedicated
_require_pages raises "stored document has no page content" so the
operator reindexes instead of hunting for file corruption
* chore: trim non-essential comments, fix three docstring issues
Review fixes:
- client.py: remove unverified "pending" from get_document status enum
- cloud_api.py: update stale "kept line-for-line" module docstring
- cloud_api.py: add node_summary to get_tree Args
Comment trimming across __init__.py, client.py, cloud_api.py, errors.py,
local_api.py, local_store.py — module docstrings shortened, explanatory
inline comments removed, multi-line class docstrings collapsed to one line.
* feat: add get_page_content and get_tree include_text parameter
- client.get_page_content(doc_id, pages): convenience method wrapping
get_ocr + page filtering; _parse_pages moved from the demo into client.py
- client.get_tree(..., include_text=False): skips node text for
structure-only views; local reads the stored tree directly (no text
to begin with), cloud strips client-side via remove_fields
- demo simplified: tools now call the new client methods directly
* simplify: drop redundant try/except in demo get_page_content tool
* feat: add get_document_structure convenience method
* test: cover get_page_content, get_document_structure, include_text=False
* fix: guard against four edge-case crashes found in PR #389 review
- Demo: use getattr for retrieve_model so cloud clients don't AttributeError
- utils: return empty string when start/end page index is None instead of TypeError
- local_store: clean up temp file on _write_json_atomic failure
- local_api: re-raise PageIndexAPIError before the catch-all Exception block
* chore: trim verbose optional-dep comments in requirements.txt
* revert: restore README.md to main — SDK section deferred to next version
* fix: eliminate double PDF parse in local standard indexing
_extract_page_texts already reads the PDF via PyPDF2; pass the
pre-extracted texts as page_list to page_index_main so it skips
its own get_page_tokens call. Token counts are computed once via
litellm.token_counter in _index_standard.
* test: assert page_list is passed and correctly shaped
* fix: wire include_text to cloud API and guard get_page_content on processing docs
cloud_api.py now sends include_text as a query param so the server can
omit node text from tree responses, saving bandwidth. Backward-compatible:
old servers ignore the param, client-side remove_fields still strips.
get_page_content raises PageIndexAPIError instead of TypeError when the
document is still processing (get_ocr returns result: null).Four new client methods make PageIndex documents available to agent frameworks, in both modes, with the mode decided solely by the client constructor: - agent_tools(): plain functions (browse_documents, get_document, get_document_structure, get_page_content) matching the PageIndex cloud MCP server's tools/list — same names, schemas, descriptions, and JSON response envelopes — so agent prompts port unchanged between the cloud MCP connection and these in-process tools. Tools never raise; errors come back in the same envelope. remove_document ships behind include_management=False. - as_openai_tools(): the same tools wrapped for the OpenAI Agents SDK. - as_claude_mcp(): one mcp_servers entry for the Claude Agent SDK — cloud clients get the remote MCP config (the framework connects to api.pageindex.ai/mcp and discovers the full cloud tool set), local clients get an in-process SDK MCP server. - agent_instructions(doc_id=None): orchestration guidance for the agent's system prompt; doc_id (same shape as chat_completions) appends the target documents. submit_document() gains wait=True: poll get_document status until completed, raise on failed or after 30 minutes — the manual polling loop every cloud caller writes today spins forever on a failed document. Neither framework becomes a dependency: imports happen at call time with actionable errors, and pageindex[openai] / pageindex[claude] extras are floor-only pins. tests/data/cloud_mcp_contract.json freezes the tool contract; a parity test guards against drift. 36 new tests (95 total), plus a live OpenAI Agents SDK run over a seeded local store verifying the structure-first navigation flow end to end.
…mantics - Large-doc next_steps now says structure-first, consistent with tool descriptions and agent instructions - _remove_document fetches document list once instead of per-name - call_tool returns error envelope for unknown names instead of raising - _not_ready_error timed_out flag reflects actual wait outcome - openai_agents.py docstring corrected to match default (FunctionTools) - Removed unused ModelSettings import from demo
…data merge - McpBridge reads session/protocol headers under the lock (now RLock: _ensure_initialized posts while holding it). openai-agents runs sync tools on threads and executes parallel tool calls concurrently, so bridge functions genuinely race; a torn read sent a new session id with a stale protocol header. Measured: one session expiry under 8 threads cost 4 initializations before, minimal 2 after. - Session-expiry retry also resets the negotiated protocol version, so the re-handshake carries no stale MCP-Protocol-Version header. - browse_documents time sort pages list_documents natively instead of fetching the whole library to slice one window (relevance still needs the full list for scoring). - _await_completion: a status refetch that nulls out metadata no longer clobbers the listing's copy (setdefault was a no-op on existing None). - Structure tool reads the raw stored tree via a named LocalAPI raw_tree() seam instead of reaching into _api._store internals; drop the redundant deepcopy before _format_structure (store re-reads from disk, formatting builds fresh containers). - Shared pageindex/_version.py replaces _sdk_version duplicated in mcp_bridge and the Claude integration. Left as-is after source verification against the cloud MCP: first-page budget bypass, pageNum falsy-zero, and the page-gap fallback text are letter-for-letter cloud behavior — parity wins over local repair.
…lience, contract drift - _parse_page_spec bounds the requested span arithmetically (10k pages) before materializing it; pages="1-1000000000" previously expanded to a billion integers inside the caller's process. - Local submit_document uniquifies document names the way the cloud upload does (taken name -> _1.._99, then reject with the cloud's own message). Same-name duplicates broke name-addressed tools: resolution always picks the newest, so older duplicates were unreachable. - agent_instructions(doc_id=...) now fails loud when the pinned doc's name is shadowed by a newer same-name document (legacy stores predate the rename) — it previews resolution with the same _resolve_document the tools use, so the check cannot drift from actual behavior. - submit_document(wait=True) tolerates transient network errors, not just API errors; a dropped connection at minute 25 of a 30-minute wait no longer kills it. Third strike wraps into PageIndexAPIError per the documented contract. - The live contract-parity test compares full per-param schemas, not just names and descriptions. It immediately caught real drift the shallow check had been passing: the server now emits nullables as anyOf unions and stamps MAX_SAFE_INTEGER maxima on offset/part. Contract and snapshot updated to the served wire form; _annotation_for learned anyOf so bridge signatures stay Optional[str] instead of degrading to Any. Adjudicated, not changed: the allowed_tools wildcard example stays (docstring advice covers scoping; Ray's call), and raw-length response accounting stays (letter-for-letter cloud behavior, parity wins).
Compute PR #558 makes /doc/ return {"doc_id", "name"} carrying the
post-dedup-rename name. Mirror it end to end: local submit returns the
stored name, the client warns when it differs from the uploaded file
name (read via .get so older cloud servers stay compatible), the local
name-exhaustion check runs before indexing instead of after the LLM
spend, and the demo caches doc_id in a file instead of name-matching —
a renamed document made the name lookup re-index on every run.The cloud MCP server publishes its agent instructions in the initialize result, adapted to each key's tool set. agent_instructions() previously returned the SDK's local-subset text in both modes — a silently forked copy that lacks the guidance for cloud-only tools (search_documents escalation, folders, images) and drifts as the server's prompt evolves. Cloud clients now serve the server's live instructions, captured from the initialize handshake on a per-client bridge shared with agent_tools() (one session, no extra request). An empty server response raises instead of silently substituting the subset text — same posture as the annotation-regression guard. The local constant stays as the honest subset for the in-process tools, with its provenance noted and a consistency test that every tool it names exists in the local registry.
sort="relevance" is cloud-side semantic ranking; the local substring imitation could satisfy the letter of the interface while silently missing semantically relevant documents. Per the honest-subset rule (same treatment as folders), local now returns the "not available here" envelope for sort="relevance" or a stray query, and the local instructions steer discovery through name/description matching plus full-library paging instead of prescribing a capability that does not exist here. The tool schema keeps the cloud contract verbatim, like folder_id: honesty lives in the runtime answer, not a forked contract.
"Not available here" read as a broken feature; the honest framing is that folders and semantic ranking exist on PageIndex cloud and are not in local mode yet. Both envelopes now say so and name the cloud client in next_steps, so agents relay an accurate story to the user.
The cloud-verbatim browse_documents description invites sort="relevance" and folder drilling, so a local agent's first semantic search attempt was a guaranteed dead end discovered only from the runtime error envelope. Local registration now appends a LOCAL MODE note to the description — the agent learns what is cloud-only before calling; the runtime envelope stays as the backstop for prompts that ignore descriptions. The cloud-facing contract stays byte-verbatim.
Appending a retraction to the cloud-verbatim description left the model parsing an instruction and its negation — and kept the cloud text recommending search_documents and get_folder_structure, tools that are not registered locally (get_page_content likewise pointed at get_document_image). Guidance now adapts to the local surface the way AGENT_INSTRUCTIONS already does: schema structure stays byte-identical to the contract (mechanically asserted by a strip-descriptions test), while local description strings teach only what works here and point to PageIndex cloud for the rest. A dead-reference test forbids local guidance from naming tools outside the local registry, so a contract refresh that reintroduces a cloud-only reference fails loudly.
folder_id, sort, query, and recursive were exposed locally with localized "cloud-only" descriptions, leaving the dead-end calls expressible and discovered at runtime. Schema constraints beat guidance: the local surface now serves the contract minus these parameters, so strict-schema frameworks make the calls inexpressible and a prompt that insists on sort="relevance" degrades to the bare call (the correct local behavior) instead of an error round-trip. The implementations still accept the hidden parameters and answer with the guided "works on PageIndex cloud" envelope — the backstop for direct call_tool callers and hosts without schema enforcement. wait_for_completion stays: seeded or torn stores can hold documents that are genuinely not completed. The structural guard now asserts the local schema equals the contract minus the documented hidden set, descriptions aside.
Three independent review passes over the agent-instructions increment surfaced six fixes: - The per-client bridge moved off the instance into a weak-keyed, lock-guarded module cache: cloud clients stay picklable (threading.RLock no longer rides on the client) and concurrent first calls can no longer construct duplicate bridges/sessions. - Blank or non-string initialize.instructions now hit the same honest error as a missing one — a whitespace-only or structured value could previously become the system prompt (or crash the doc_id append with a raw TypeError). - The invalid-sort envelope no longer prescribes sort="relevance" — the one error text that still taught the cloud-only value it would then reject. - "Page through the rest of the library" is emitted only when has_more is true; a fully-listed library no longer instructs a pointless call. - The mandatory full-library paging step now says limit: 50 — 6 calls instead of 30 on a 300-document library. - Docstrings and comments rescoped to what is actually true: the never-raise contract covers invocations the signatures accept (unknown params fail at the Python boundary; call_tool answers them with the guided envelope), recursive is accepted as the identity rather than errored, lenient framework arg models drop hidden params pre-call, and the module header no longer claims full schema parity. The capability-phrase guard now covers every local docstring, not just browse_documents.
The frozen contract guards tools/list, but the response envelopes the local tools emit were hand-built to mirror the cloud's and had no drift detector. A key-gated live test now asserts every field local emits exists in the live cloud response for the analogous call (top-level keys, next_steps, document entries, structure nodes, content entries). Guidance wording is deliberately localized and not compared. Verified green against the live server: local and cloud field structures currently match exactly.
….10) Local mode gains managed document QA: an agent over the #393 local tool set, reachable through three wire protocols, each 1:1 with the backend and with no translation layer. - chat_completions(): standard chat.completions semantics on any OpenAI-compatible backend (openai-agents engine). Final answer only, cross-turn aggregated usage, streaming as text pieces or chunk dicts (the existing cloud signature, now implemented locally; model and max_turns are local-only additions). - responses(): the agentic surface — OpenAI Responses format, the tool process is standard output items, streaming forwards native events (tool outputs emitted as response.output_item.done, the way the platform streams its own server-side tools). Round-tripping output into the next input keeps provider prompt-cache prefix continuity and the agent's memory — live-verified: the follow-up call answered from round-tripped tool output with zero new tool calls. - messages(): Anthropic-native via the SDK's own tool runner (new pageindex[anthropic] extra, floor 0.68.0 verified for tool_runner/beta_tool(input_schema)). tool_use/tool_result round-trip is the format's native behavior; the envelope is the final message with aggregated usage plus the full new-turn sequence; the managed system blocks carry cache_control breakpoints. Shared skeleton: thin chat header + the local AGENT_INSTRUCTIONS (caller system content is appended, not rejected), the doc_id targeting block as a leading context item (factored out of build_agent_instructions), read-only toolset, structural-only validation (no arbitrary caps — backend limits govern), sampling params passed through, per-run tracing disabled, enable_citations rejected as cloud-only. Design basis is industry-standard formats rather than the cloud chat endpoint; responses()/messages() raise on cloud clients until the cloud converges. Tests run the real engines against scripted backends (a Model fake for openai-agents, a mock HTTP transport under the real anthropic SDK) with real tool execution against a seeded store, including the round-trip prefix-extension assertions on both engines.
Three independent review passes (bug scan, claims-vs-code, adversarial runtime probes) over the local-chat increment; every fix below was reproduced before being fixed. messages(): - A max_turns cut no longer duplicates the final assistant turn: the runner has already appended it when iterations exhaust, so the round-trip history carried a duplicate tool_use id and ended on an unanswered tool_use — a guaranteed 400 on continuation. The append now keys on stop_reason, and truncation reads natively as stop_reason: "tool_use" with a continuable history. - The envelope is JSON-serializable end to end: runner-stored turns carry pydantic content blocks; everything is dumped to plain dicts, excluding SDK-internal __api_exclude__ fields (parsed_output) that the API rejects on round-trip. - Bounded by default (max_iterations 10, like the OpenAI surfaces); usage aggregation now preserves the final turn's native fields and sums the token counters None-safely; empty caller system strings are skipped; non-dict message entries and bad doc_id types raise PageIndexAPIError; anthropic < 0.68 gets an actionable version error; the doc block no longer spends a cache_control breakpoint. chat_completions()/responses(): - MaxTurnsExceeded wraps into PageIndexAPIError on all four run paths. - responses(stream=True) is one logical response: per-turn backend lifecycle events are collapsed (a canonical consumer previously stopped at turn 1's response.completed and never saw the answer), sequence numbers are reassigned monotonically, and the synthesized tool-output event carries output_index/sequence_number. - The responses envelope carries the real request surface (instructions, the actual function tool definitions, tool_choice, parallel_tool_calls, error/incomplete_details). - RunConfig(group_id) pins a stable prompt_cache_key: openai-agents otherwise stamps each run with a fresh key, tagging round-tripped prefixes as different cache groups and defeating the feature the round-trip exists for. - Abandoning a stream now cancels the run: a watchdog task lets the cancellation land even while the pump awaits the backend, and the per-call AsyncOpenAI client is closed before its loop ends (fixes "Task exception was never retrieved" noise). The opening role chunk is emitted even for empty outputs; empty responses() input and enable_citations-before-extra ordering fixed. Docs rescoped to what is true: finish_reason/status reflect loop completion on the OpenAI surfaces (the engine does not surface per-turn backend reasons); chat streaming yields visible narration including pre-tool text; messages(stream=True) forwards the Anthropic SDK's native event objects (not wire-verbatim); the doc block is a leading conversation item on OpenAI surfaces and a system block on messages(). Tests: 25 in the file (11 new), with per-extra skip sections so a machine with only one framework still covers the other surface; without-frameworks matrix re-verified; live smoke re-run green with a clean exit.
Fills the last cell of the agent-connection matrix: users driving their own anthropic tool_runner loop get runnable tools directly. Cloud wraps the live MCP tool set with input schemas passing through verbatim (MCP inputSchema is the Messages API schema shape); local exposes the same set messages() runs internally. The beta_tool wrapping moves from local_chat into integrations/anthropic_sdk.py, parallel to openai_agents.py, and messages() now consumes the shared builder. agent_tools grows _bridge_invoker/_read_only_tools so the plain-function and beta_tool cloud paths share invocation containment and the read-only gate.
Adversarial + best-practice review of 4590dd8 (three independent passes) surfaced two holes. The export was sync-only: AsyncAnthropic's runner accepts only BetaAsyncFunctionTool and splices anything else into the request body unserialized, so the first call died with an opaque TypeError — asynchronous=True now builds beta_async_tool runnables (present since the 0.68.0 floor) that run the blocking bridge/store call in a worker thread, keeping I/O off the caller's event loop. And beta_tool stores input_schema by reference, so cloud tools aliased the bridge's cached metas while the local path deep-copied — the builder now copies, and the passthrough test asserts equal-but-not-aliased so it can no longer compare an object with itself. Docstring fixes from the same round: the MCP-connector pointer now carries the full live-verified shape (authorization_token was missing — following it literally gave a 401), and the manual messages.create loop's to_dict() serialization is documented. Tests pin the runnable flavor both ways (isinstance), which existing tests could not distinguish.
…onversation The targeting block doc_id adds is re-set on every call and sits in the cached prompt prefix, so a round-trip that drops (or changes) doc_id silently diverges the prefix and loses the cache continuation. State the rule on all three chat surfaces' doc_id docs, and pin it with a prefix test that passes the same doc_id on both calls.
query + doc_id is the minimal PageIndex contract, so it now works uniformly: chat_completions and messages accept a plain string (one user message), as responses always did per its wire format. The wrap is input sugar at the SDK surface, not a translation layer — the outgoing wire is unchanged, and managed agent surfaces taking strings is the ecosystem convention (Runner.run, claude_agent_sdk.query). Cloud chat_completions gains the same acceptance; blank strings raise on every path.
The Messages API requires a per-turn output budget on the wire, but that is table-setting, not a PageIndex-layer user obligation — the simple call is now a question + model + doc_id. The knob stays overridable (passthrough intact); model stays required because no cross-vendor default is honest to guess.
max_tokens is a cap, not consumption, so the default should be the highest universally safe value: 4096 could truncate long-form answers (whole-document summaries), while 8192 is the output ceiling every non-EOL Claude model accepts and stays under the SDK's non-streaming long-request threshold.
Inserting tests above decorated ones absorbed their @needs_agents markers, so two tests ran (and failed) in the without-frameworks CI job. Both simulated-bare and full runs are green again.
Tool layer: - anthropic adapter: failed tool calls raise ToolError so the runner emits tool_result is_error:true; McpBridge.call_tool returns (text, is_error) and surfaces the server's MCP isError marking - as_openai_tools builds FunctionTool with the contract/server schema verbatim (strict off) — function_tool() regenerated schemas from signatures, dropping items/enum/pattern/bounds and aborting the whole list on object-typed params; shared _tool_specs() feeds both adapters - remove_document validates every name before deleting anything; call_tool classifies only bind-time TypeErrors as INVALID_INPUT - unknown-tool envelope formatted with _dumps like every other envelope Local chat: - doc_id is enforced at the tool layer (allowlist threaded through call_tool and the adapters), not just prompted; the shadow check runs inside the scope - _openai_model routes litellm/ and provider/ paths via LitellmModel and strips openai/ — the normalized retrieve_model 404'd as a raw wire name - responses() reports the backend's real terminal status (recorded at the transport client; the framework discards Response.status) and wraps framework exceptions in PageIndexAPIError - chat_completions streaming yields its opening chunk inside try, so an abandoned iterator still cancels the run and closes the backend - prompt-cache group_id is per-conversation (model+instructions+first item) instead of one global constant pooling every user - messages() max_tokens default resolves per model (claude-3 caps at 4096) Packaging / surface: - __init__ registers the 0.2.10 modules in _SUBMODULES; unknown names raise AttributeError instead of eagerly importing page_index_classic - anthropic floor 0.84.0: first release with ToolError whose runner also executes the final turn's tools on a max_iterations cut - client docstrings caught up with local chat landing Claude Agent SDK gate: - claude_allowed_tools(mcp_servers) derives mcp__<key>__<tool> entries from the caller's own registration map (live server annotations on cloud, the contract locally) — no name is ever spelled twice - claude_agent_config() bundles the three slots as one-call sugar over the explicit form Examples: - demo runs against cloud again (getattr for local-only attrs) and finds an existing indexed copy by name before re-indexing Tests: monkeypatches replace the consuming module's binding instead of mutating the shared time/requests modules; 185 -> 211.
claude_agent_config() gets two symmetric siblings, so each framework's front door is a single splat over the same explicit primitives: - openai_agent_config(): Agent(**...) kwargs — instructions, tools, and the local retrieve_model (cloud omits model for the framework default) - anthropic_runner_config(): tool_runner(**...) kwargs — system, tools, and the messages() defaults (per-model max_tokens, 10-iteration bound); only the user's messages remain Bundles stay pure sugar: doc_id rides agent_instructions, no extra semantics over the explicit form, docstrings point both ways. The demo agent shrinks to Agent(**client.openai_agent_config(doc_id=...)). Construction is pinned against the real frameworks in tests (Agent and tool_runner both built offline), so an upstream kwargs rename fails loudly; 211 -> 215 tests.
…lation, output_index axis - get_page_content: the summary is additive, not either/or — a call that both truncates for size and has out-of-range pages reported only the latter, telling the agent every in-range page was returned (#2) - McpBridge._extract_result: strict request-id correlation only; the eager fallback could hand back a stale or mis-correlated JSON-RPC message as this call's reply (#16) - responses() streaming: output_index now addresses the logical response.output — backend per-turn indexes are re-based past prior turns' items and the SDK-injected tool outputs take the next slot on that axis, instead of reusing the event-sequence counter (#15) 215 -> 217 tests.
claude_agent_config(server_name=...) applied the name to the mcp_servers key and the allowed_tools pre-approval but build_claude_mcp hardcoded create_sdk_mcp_server(name="pageindex"), so a renamed server introduced itself under the default identity in the MCP handshake. server_name now threads through as_claude_mcp into the server declaration; default unchanged, cloud entries carry no name and are untouched.
Ruling: users may install either major and both must work. 228426d had raised the floor to >=5 to close the dual-font-name-semantics install window; that trade is reversed — the floor returns to >=4.30.0 and the mild cross-major tree variance (per-face vs family font names, 3 of 9 corpus docs differ in title text only) is accepted. What already held: both compat branches were in place (embedded_toc's per-major bookmark API, geometry's font-name getattr chain), the 5.x semantics sentinel already version-skips, and the full suite passes on 4.30.0 (330 passed, E2E extraction verified on the bookmark and detected paths). What was missing: the pyproject floor blocked 4.x installs, and no test touched the 4.x bookmark branch — read_bookmarks was only ever exercised through stubs. New: a bookmark-parity test pinning identical entries from a real PDF (4.30 proven to take the PdfOutlineItem branch, byte-identical output to 5.13), and a pdfium-4 CI leg (py3.10, with frameworks) in tests.yml, mirrored in publish.yml's release gate.
page.get_textpage().raw leaves the PdfTextPage unreferenced; whenever the cycle collector ran mid-extraction its finalizer closed the handle, every per-char FFI call read back 0, all 17 chars failed object attachment, and the test failed with an empty extraction. That was the nondeterministic py3.10-leg CI failure: GC timing, shifted by the installed-package set, decided each run. Proven deterministically both ways with gc.set_threshold(1) — the temporary yields 0 chars, a held reference 17. Product code already holds its textpage (pipeline.py); this was the only temporary-handle site in the tree.
rejojer
commented
Aug 21, 2026
MemberAuthor
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. 🤖 Generated with Claude Code - If this code review was useful, please react with 👍. Otherwise, react with 👎. |
…own lane, backstop message names reachable causes --summary-model was silently dropped on --md_path: md_to_tree had no summary-model parameter, so node summaries and the doc description always billed the resolved index model while the flag's help promised the PDF chain (summary -> index -> model -> config.yaml). md_to_tree now takes summary_model (defaulting to model, so every existing caller is unchanged), and the markdown branch feeds the flag through ConfigLoader's existing _resolve_models chain exactly like the PDF branch. Red-verified: the new CLI test saw MODELS_SEEN=["INDEX-DECOY"] before the fix. The all-nodes summary backstops still said "check LLM credentials" — but since the typed retry ladder (LLMRetriesExhausted, non-400 fatal), credential failures raise out of visit()/gather before either backstop can run. What still reaches them is per-prompt 400 exhaustion, ladder-foreign errors, and all-empty replies, so the message now says that instead of pointing at the one cause it can no longer be.
ProcessPoolExecutor.__init__ eagerly allocates its call/result queues, so environments without working POSIX semaphores (slim containers, sandboxes, Lambda-class hosts) raise at construction — which sat outside the try/except that promises the sequential rerun, so a >=64-page PDF on such a host failed the whole indexing instead of degrading. Construction now lives inside its own guard: refusal falls back to parse_charlevel_meta, except in a bootstrapping spawn child, which re-raises like the sibling handler — a sequential rerun there would duplicate the whole run per worker.
…skips the retry ladder, five silent-wrong-value edges
summarize_tree's witness now counts a call as answered only when it returns
text: a backend whose replies carry empty content (content filter, spent
output cap) no longer stores a retrieval-ready document whose every
model-written summary is blank — a raw-text short leaf cannot vouch for it.
One blank reply among good ones stays absorbed per the per-node policy.
The retry ladder raises 400 immediately (_NO_RETRY_STATUS): the prompt will
not shrink, so retrying context_length_exceeded burned 10 wire calls and 9s
of sleep per node before failing the same way. The exception leaves the
ladder raw, so consumers classify it per-prompt exactly as before;
_is_unrecoverable is untouched.
generate_doc_description absorbs that same per-prompt 400 (empty
description) instead of discarding a fully indexed document over its one
unbounded whole-tree prompt; every other failure keeps propagating per the
no-swallow rule.
_dump_block passes exclude_unset like the SDK's own request serializer:
tool_use blocks no longer come back with "caller": null, which the Messages
request schema has no null variant for — the documented append-verbatim
continuation broke on the caller's second turn.
_parse_pages restores 0.2.10's whitespace tolerance (" 1-3", "5 - 7",
"1-3\n") on the SDK surface; the tool layer stays on the strict contract
pattern.
The flash CLI resolves the summary model through ConfigLoader's chain like
the standard and markdown branches instead of a hand-rolled ladder that
silently outranked a file-supplied summary_model with --model, and rejects
an empty flash structure like the SDK does instead of writing
"structure": [] and reporting success.
submit_document scrubs a surrogate-escaped basename at entry so the
returned name is byte-for-byte the stored name (the rename warning now
fires), and validates metadata with allow_nan=False so NaN/Infinity are
rejected at the gate instead of leaking as bare literals into the store and
every tool envelope.
All eight red-verified; suite 388 green.A literal replacement character in source is indistinguishable from actual mojibake, and the sibling scrub at _extract_page_texts already spells it �; the new basename scrub and its test now match. The pre-existing literal in the PyPDF2 extraction test is untouched.
Two call sites (the stored basename and the extracted page texts) repeated the regex-plus-replacement-char pair; the policy — what counts as unencodable and what it becomes — now has one owner, _scrub_surrogates.
Ruling: the two metadata files point users at one major — pyproject returns to >=5, agreeing with requirements' 5.13.0 pin — while the 4.x compat branches remain as insurance for environments an external pin or downgrade puts on the 4.x line. This supersedes 74de96f's floor half (both-majors-advertised); its semantics half (per-face font names) and its guards stay: the bookmark-parity test and the pdfium-4 CI leg now exist to keep the insurance honest — untested insurance rots — not to hold an advertised floor.
…ropic client per backend, dead backend param dropped Every cloud_api non-200 now raises with status_code=response.status_code (get_document already did; the other ten paths made callers parse message text to tell a 429 from a 401). messages() reuses one anthropic.Anthropic per backend: construction paid ~45 ms of SSL-context build per call (the same waste 6a9104a evicted from the old direct indexing lane) and reused no connections. The per-run finally closes only per-call constructions — cached clients stay open, caller-owned http_clients survive as before, and a backend whose values defeat hashing (or the tail past 8 distinct backends) constructs per call exactly as today. The fake-transport test seam is untouched. _litellm_model loses its backend parameter — never read since 78d44b4 moved credential judgment to LiteLLM itself; three call sites threaded it for nothing. Two doc truths: _await_completion notes local documents are stored already terminal (the wait never engages there), and page_index_flash's optimize doc says expand needs readable page text, so bookmark-only and scanned PDFs run the merge half only (expands reports 0).
…g it Two threads missing the cache on the same backend key both construct; plain assignment let the second store evict the first client while other threads could already be running requests on it — whose per-run finally would then close it mid-flight for someone else. setdefault keeps whichever client landed first; the loser instance drops its only reference and closes on refcount, the same lifecycle 6a9104a documented for the old per-call clients.
…spect model output ceilings The flash empty-structure errors (SDK and CLI) now point at mode='standard', which builds the structure with the model — the refusal stays, the recourse is in the message. _default_max_tokens clamps a lifted thinking default to the model's output ceiling from LiteLLM's bundled capability map (no hardcoded table): claude-opus-4-1 with a 30000 budget now sends 32000, not a wire-rejected 38192. Models the map does not know keep today's budget+8192, and a bool budget no longer counts as one.
…421) Indexing: dead credentials or a missing model fail the run instead of storing a document with blank summaries; a 400 (context_length_exceeded) skips the retry ladder — the prompt will not shrink — and stays a per-prompt failure the run absorbs; all-empty model replies can no longer store a retrieval-ready document; the one-sentence doc description absorbs its own context overflow instead of discarding a fully indexed document; the heading-less flash refusal points at mode='standard'. Chat: messages() output is append-verbatim clean — unset response-only defaults are dropped (no "caller": null the request schema rejects); Claude cache marks follow the wire routing; model_settings and name are openai_agent_config parameters; one Anthropic client per backend; lifted thinking defaults are clamped to the model's output ceiling from LiteLLM's capability map. Store and inputs: lone surrogates are scrubbed from page text and the stored basename, so the returned name is byte-for-byte the stored name and the rename warning fires; NaN/Infinity metadata is rejected at the gate; every cloud error now carries its HTTP status. CLI: the flash lane resolves the summary model through ConfigLoader like the standard and markdown lanes; an empty flash structure errors like the SDK instead of writing "structure": [] with exit 0; --summary-model reaches the markdown lane; the SDK page-spec surface keeps 0.2.10's whitespace tolerance while the tool layer stays strict. pypdfium2 stays on the 5.x line for every install; the 4.x code paths are tested compatibility insurance with their own CI leg; process-pool construction failure falls back to the sequential parse; a py3.10 GC flake in text extraction is fixed. Port of feat/local-chat 0667e3b..1993740 (28 commits); README and assets untouched.
The expand loop awaited one propose_children at a time — 20-30 nodes at ~3s each put 1-3 minutes of pure round-trip latency on every default local submit. Nodes waiting in a wave are all frontier leaves whose decisions cannot affect each other, so the model half now runs concurrently (EXPAND_CONCURRENCY = 8) while the apply half stays serial in wave order: decisions, log entries, and child ids land exactly as before, and children attach into the next wave. A fatal classification still aborts the run right after the wave's gather. Benchmarked on real PDFs with a fixed-latency fake model: 408 pages 21.1s -> 3.0s, 758 pages 28.2s -> 3.5s (7-8x); final trees byte-identical to the serial pass on both. The cap stays low on purpose: expand treats an exhausted retry ladder as fatal, and a wide burst on a rate-limited account would trip exactly that — 8 already collapses minutes to seconds.
A child's only prerequisite is its own parent's apply, so each kept node gathers its children directly rather than waiting for its whole generation to finish. Same recursive shape as summarize_tree; the semaphore still caps in-flight proposals at 8; trees are unchanged.
Cap sweeps on six real documents put the speed plateau at 32: the ready frontier tops out at 21-28 nodes on few-hundred-page PDFs, so 64 buys nothing while doubling the burst. Live runs at 32 cut the expand phase 24-30% on the two documents wide enough to feel it, with zero ladder retries anywhere - and summaries already burst twice as wide through the same ladder.
…osals (#422) * perf: expand proposes a wave of nodes concurrently The expand loop awaited one propose_children at a time — 20-30 nodes at ~3s each put 1-3 minutes of pure round-trip latency on every default local submit. Nodes waiting in a wave are all frontier leaves whose decisions cannot affect each other, so the model half now runs concurrently (EXPAND_CONCURRENCY = 8) while the apply half stays serial in wave order: decisions, log entries, and child ids land exactly as before, and children attach into the next wave. A fatal classification still aborts the run right after the wave's gather. Benchmarked on real PDFs with a fixed-latency fake model: 408 pages 21.1s -> 3.0s, 758 pages 28.2s -> 3.5s (7-8x); final trees byte-identical to the serial pass on both. The cap stays low on purpose: expand treats an exhausted retry ladder as fatal, and a wide burst on a rate-limited account would trip exactly that — 8 already collapses minutes to seconds. * perf: expand schedules dependency-exact instead of in waves A child's only prerequisite is its own parent's apply, so each kept node gathers its children directly rather than waiting for its whole generation to finish. Same recursive shape as summarize_tree; the semaphore still caps in-flight proposals at 8; trees are unchanged. * perf: expand admits thirty-two concurrent proposals Cap sweeps on six real documents put the speed plateau at 32: the ready frontier tops out at 21-28 nodes on few-hundred-page PDFs, so 64 buys nothing while doubling the burst. Live runs at 32 cut the expand phase 24-30% on the two documents wide enough to feel it, with zero ladder retries anywhere - and summaries already burst twice as wide through the same ladder.
… home (#424) * feat: the client grows two sides — documents and chat each pick their home One client, two independent switches: api_key decides where documents live (the PageIndex cloud, or the local store); a configured chat model decides who answers (your own model in your process, or the managed cloud chat). Their free combination opens the bridge — cloud documents, your model — and the fourth cell stays unspellable. - index=/chat= slots: string shorthand or grouped dict, 1:1 with the flat arguments; one spelling per side, sides mix freely - optional "type" everywhere (top-level and in either dict): always omittable, checked against the content, meaningful alone — type="cloud" is a keyless cloud spelling - PAGEINDEX_API_KEY is read only when the code explicitly says cloud (PageIndexCloudClient(), type="cloud", "pageindex-cloud", {"type": "cloud"}); a bare PageIndexClient() stays local - bare mode words ("cloud", "local", …) are reserved: they error with the real spellings instead of silently parsing as model names - bridge chat runs the in-process agent over the live cloud MCP tools and instructions; doc_id targets at the prompt level; citations stay managed-only; an auth-shaped backend failure explains whose credentials run the model - typed shapes (IndexConfig, ChatConfig) ship as optional annotations Every previously working program is byte-for-byte unchanged: the only behavioral deltas are error paths — reworded guidance, and the api_key+chat_model combination graduating from an error into the bridge. * fix: the constructor refuses empty and mistyped values on every spelling - .env keys reach all four keyless-cloud spellings: utils' import-time load_dotenv now runs before every PAGEINDEX_API_KEY read - an empty chat-side value ("", {}) errors instead of silently selecting own-model chat on the default model; None-valued slot keys mean absent, exactly like the flat arguments - _local_chat derives from chat_model, so a post-construction assignment switches the whole client, never half of it - model= beside a slot gets the split guidance (index_model=/chat_model=) instead of "two spellings of the same thing" - the messages door wraps provider failures through _model_backend_error, and 401s count as auth-shaped even without "api key" in the text - keyless-cloud hints name the spelling that actually combines; slot strings are stripped; wrong-typed values raise PageIndexAPIError - retrieve_model/chat_backend docs drop the stale "Local mode only"; the local-scope refusal no longer claims bridge tools are server-scoped * fix: type= cross-checks the index slot; the cloud pinned class frees its chat side - type= beside index= now does what the docstring promises: agreement passes, disagreement errors, and a mistyped value reports the vocabulary error instead of a spelling collision - PageIndexCloudClient grows the chat-side arguments (chat=, chat_model, retrieve_model, chat_backend), so "pin the index side" is literally true and the chat surfaces' construct-with-chat_model guidance is followable on it - the four chat doors' doc_id entries carry the enforcement split the config helpers already state (local: tool-layer allowlist; cloud: prompt-level / server-side) - types.py stops claiming slot keys share the flat names — the side prefix is factored out, index={"model"} is index_model= * docs: bridge-reachable wording — dependency errors say own-model chat, hints name a chat= model - the three framework-missing errors said "in local mode", which is wrong on a bridge client (cloud documents + own model) — they now explain the dependency the way the surfaces do: your own chat model - the construct-with guidance reads "(or a chat= model)": a bare chat="pageindex-cloud" is also chat= but selects the managed side - the mechanical Local-only → Own-model-chat-only substitution left orphan fragments and two overlong lines; those paragraphs re-flowed * fix: managed chat reads None; the bridge stops paying per-turn tool lists - a managed-chat cloud client stores chat_model/chat_backend as None, so the documented attribute reads instead of raising AttributeError; _local_chat derives from "is a chat model configured" - McpBridge caches tools/list per session — every chat turn rebuilds the tool set, and the round trip was pure latency; the 404 session-expiry reset drops the cache with the session - run_messages builds tools before the transport: on a bridge client that build is network I/O, and a failure there stranded a per-call anthropic client ahead of the try/finally * refactor: the side declaration is spelled mode=, not type= "type" is Python's own word — a builtin, and "data type" beside the TypedDict shapes; "mode" is what the SDK already calls the two sides ("local mode", "cloud mode"). Same grammar everywhere the declaration appears: the top-level argument, the index dict, the chat dict, the typed shapes. The rename also frees the builtin inside the constructor, so the shape-check error names the offending class through type() again. "type" in a slot dict is now an ordinary unknown key. * fix: the reserved-word errors stop calling "cloud" not a mode word With the declaration key spelled mode=, 'index="cloud" is not a mode word' contradicted its own remedy, index={"mode": "cloud"} — "cloud" is exactly a mode value. The four bare strings are reserved words; the message now says so. * fix: a blank tools/list is not cached; the auth note's managed exit is chat-lane only - McpBridge.list_tools caches only a non-empty list — a transient blank (a deploy blip, a gate misconfiguration) would otherwise run every later turn with zero tools while the instructions still name them, and only a 404 session reset could clear it - the 401 architecture note appends "drop the chat model configuration" only on the chat lane: responses() and messages() refuse a client without an own model, so on those lanes the exit sent the caller in a circle - CloudIndexConfig says api_key is omittable only while mode: "cloud" stays — index={} refuses as an empty dict rather than reading the env * fix: the bridge fetches tools/list per call again; .env resolves from the cwd - McpBridge.list_tools no longer caches: the tool set is built once per SDK call (Agent(tools=...) ahead of Runner.run; build_anthropic_tools ahead of tool_runner), not per model turn, so the cache saved one round trip per later call while a mid-pagination 404 replayed a dead cursor into a duplicated (and cached) list, and the list went out by reference across a lock dropped between miss and store - utils.load_dotenv searches upward from the cwd: a bare load_dotenv() walked up from utils.py, which is site-packages for an installed SDK, so the four keyless-cloud spellings never saw a project-root .env; the package-relative walk stays as the fallback - the emptiness guard strips strings: chat_model=" " selected own-model chat, the silent flip the guard's own comment rules out - the _local_chat comment stops advertising post-construction assignment as a full mode switch * fix: the pinned classes take index=/chat=; "cloud"/"local" are mode words; a blank chat_model stays managed - PageIndexLocalClient takes index= and chat=, PageIndexCloudClient takes index= — the grouped spelling of the flat vocabulary each already took; their refusals name the class and an exit that class can take, and the mode cross-check runs before any environment read - "cloud" and "local" are accepted wherever "pageindex-cloud" was (index=, chat=, mode=, {"mode": ...}), case- and whitespace-insensitive; "hosted" and "managed" still refuse, pointing at the real word - every spelling strips its strings, and the slot spellings' type/empty errors name the slot key (index["model"]), not the flat argument - _local_chat treats a blank chat_model as managed: the constructor refuses "", so assignment agrees instead of opening the bridge on a nameless model; openai_agent_config carries no model then either - an empty MCP tools/list raises like empty instructions does — a zero-tool agent would answer from the model's own knowledge silently - enable_citations names the real gate (managed vs own chat), not "cloud-only", on a cloud own-model client - pageindex/py.typed: the exported config TypedDicts reach installed type-checked callers * test: the two framework-door tests skip without openai-agents as_openai_tools() and openai_agent_config() need the agents package, which the "without frameworks" CI legs do not install — the same importorskip every other test on those doors already carries.
The branch had nothing main lacks (its content was squash-ported in #421/#422); this records main as merged so PR #400 spans 0.2.9–0.2.11 with history kept. Claude-Session: https://claude.ai/code/session_01GZLsJ6jmAQvgotcbhQv85Q
…k chat_model refuses at the chat door; storage_path is typed PathLike (#428) * fix: .env stays unset when the cwd tree has none; a local client with a blank chat_model refuses at the chat door; storage_path is typed PathLike find_dotenv(usecwd=True) returns '' when nothing is reachable from the cwd, and `or None` turned that into load_dotenv's own upward walk from utils.py — the install-dir leak the cwd search was added to replace. A pip-installed SDK could load another project's .env from above site-packages, silently. _local_chat treats a blank chat_model as "managed chat", which a client without an api_key does not have: chat_completions() then reached for LocalAPI.chat_completions and raised a bare AttributeError. The managed branch now refuses as a PageIndexAPIError naming chat_model. py.typed made the annotations authoritative while storage_path was typed str; _ARG_TYPES accepts os.PathLike, so Path(...) ran fine and failed the user's type check. Both signatures and LocalIndexConfig now say so. Claude-Session: https://claude.ai/code/session_017Fd7jVm366S2Xamzhxv6yb * fix: the exported config shapes pass into index=/chat=; a comment and two docstrings stop overclaiming The slots were annotated dict[str, Any]. A TypedDict is consistent with Mapping[str, object], never with dict (PEP 589: a dict-typed receiver could write arbitrary keys through it), so the four shapes types.py exports — and py.typed advertises to installed callers' checkers — could not be passed to the one place they describe. pyright on a probe that does exactly that: 9 errors before, 0 after. The constructor only reads the slot (items(), then a fresh conf dict), so Mapping is the honest bound; a plain dict is a Mapping, and TypedDict instances are plain dicts at runtime, so nothing moves at runtime. The _ARG_TYPES comment said "every value" is shape-checked; api_key is not in the table (its empty check is separate, its type check stays unchecked by ruling), so the comment now speaks for the table only. _local_doc_scope and _require_local_scope still explained the cloud drop as "scoping is server-side" — true of the managed chat, which never reaches either function. What reaches them on a cloud client is own-model chat and the config helpers, whose cloud tools take no allowlist: targeting there is prompt-level only, as the error message between them already said. 434 passed; pyright on pageindex/ unchanged at 235 (0 in the touched files, before and after). Claude-Session: https://claude.ai/code/session_01TxG8u8x29XRnK4yscZVCch * test: the install-dir .env test is named for what it asserts Claude-Session: https://claude.ai/code/session_017Fd7jVm366S2Xamzhxv6yb
…alling cloud tool scoping server-side (#429) fix: the slots accept any Mapping at runtime, as their annotation admits; eight docstrings stop calling cloud tool scoping server-side 4e9c56c widened index=/chat= to Mapping[str, Any] so the exported TypedDicts pass a checker, but _resolve_index_slot/_resolve_chat_slot still dispatched on isinstance(..., dict): a MappingProxyType or ChainMap was pyright-clean and raised "must be a string or a dict" at construction. The resolvers now narrow on Mapping — the comprehension already copies, so a read-only proxy proves the caller's mapping is never mutated. 4e9c56c corrected three of eleven "scoping is server-side" sites; the remaining eight said the same untrue thing about doc_id on cloud (its tools carry no allowlist — targeting is prompt-level, as the runtime error already explains). Deleted rather than reworded. local_chat.py's module docstring predates own-model chat over the cloud bridge; storage_path's prose now names the PathLike 5e2dc9b typed. Claude-Session: https://claude.ai/code/session_01VQ6mruXZBgw9Hjii8KPbQP
Two post-0.2.11 follow-up squashes (b9a9a3b, 174f95f) land on the review branch the same way f6fa99b did: main's tree recorded as merged, history kept, so PR #400 keeps spanning 0.2.9 → current main. Claude-Session: https://claude.ai/code/session_01VQ6mruXZBgw9Hjii8KPbQP
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR holds the complete SDK diff since 0.2.8 — 0.2.9 (local mode, #389), 0.2.10 (agent tools, local chat, Flash default) and 0.2.11 (two-sided client configuration; cloud documents with your own model, #424) — base
pre-389-main= v0.2.8, head =mainat v0.2.11. The guide below is 0.2.10's; 0.2.11's configuration surface (index=/chat=/mode=, the pinned classes' slots, the bridge) is documented in #424.PageIndex SDK 0.2.10 added two things:
client.chat()for the answer, or three standard chat APIs for the full envelopes.Both work in local mode (runs on your machine, with your model keys) and cloud mode (runs on api.pageindex.ai, with a PageIndex API key).
(Review note: the code is already on
mainvia #389, #396, #402, #404, #405, #406, #409, #410, #411, #413, #421, #422 and #424. This PR is not meant to be merged; the tools layer's own review record is #393, the 0.2.9 diff alone is #403, the 0.2.11 diff alone is #424.)Install
0.2.11 is the current stable release — plain
pip install pageindexresolves it; the pin only guards against later releases. An extra only adds a vendor's own SDK surface; everything else ships with the base install.Quick start
One rule for keys: the key belongs to whatever model does the thinking — at indexing time the summary model, at chat time the agent's model. The navigation tools themselves make no LLM calls. A missing key fails fast, naming the variable to set; cloud mode needs only
api_keyfrom dash.pageindex.ai, no model keys. (PageIndexLocalClient/PageIndexCloudClientpin the mode explicitly instead of inferring it.)Two model knobs:
index_model(indexing — structure and summaries, defaultgpt-5.6-luna) andchat_model(every chat door, defaultgpt-5.6-sol), settable as constructor kwargs or in config.yaml.modelsets both at once, and the released role names (summary_model,retrieve_model) stay accepted — new names win over old, specific over general. Chat-lane model names are LiteLLM's grammar, verbatim: bare names are OpenAI-compatible shorthand (OPENAI_BASE_URLpicks the server),provider/modelreaches that provider, and no prefix triggers a side lane.Local indexing defaults to PageIndex Flash — the tree comes from the PDF's layout in seconds, the LLM only writes summaries.
mode="standard"for the fully LLM-built tree. CLI:python run_pageindex.py --pdf_path doc.pdf. Documents persist under./.pageindex— submit once, then reuse thedoc_id(client.list_documents()shows what is stored).Chat with your documents
chat()— question in, answer outchat()returns just the answer; the agent loop — tree navigation, page reads — runs inside. It is stateless: you keep the history.chat()'s knobs stay business-level on purpose — who answers (model), about what (doc_id), how hard it thinks (reasoning_effort). Sampling and wire-level controls live on the protocol surfaces.Three standard chat APIs
chat()is sugar overchat_completions(). When you need the envelope — usage accounting, streaming metadata, the tool-use process — call a protocol surface. Each speaks one standard wire format, so request and response look exactly like the API you already know; pass a plain string or full messages in that protocol's format.All three stream with
stream=Trueand do multi-turn the protocol's own way: append the previous reply to your next call's messages. Provider prompt caching keeps working across turns — Claude models (Anthropic direct, Bedrock, Vertex) get the managed prefix cache-marked automatically.doc_idtargeting is enforced by the tools, not just suggested to the model.Tuning rides each protocol's own vocabulary, forwarded verbatim with no defaults of ours: reasoning via
chat_completions(reasoning_effort=)/responses(reasoning={})/messages(thinking={}); sampling and output caps viatemperature,top_p, andmax_tokens(max_output_tokensonresponses()) — the caps bound each backend call in the agent loop, the waymax_turnsbounds the loop. Fields a method doesn't name go throughextra_body, merged into the request last so caller keys win.Connection config follows the same doctrine:
index_backend/chat_backendon the constructor (plus per-callbackendon the protocol doors, per-call keys winning) carry each lane's own connection vocabulary verbatim — LiteLLM params on the indexing lane andchat_completions(), openai SDK client params onresponses(), anthropic SDK client params onmessages()— so two clients can point at two providers without environment juggling. Credentials belong there, never inextra_body.extra_headerson all three doors sends raw HTTP headers (beta flags and the like); the one wire-probed exception is documented: LiteLLM's anthropic adapter ownsanthropic-beta, so Anthropic beta flags belong onmessages().Bring your own agent framework
One call returns everything the framework needs — instructions plus tools. Local and cloud clients work identically. Agent frameworks are async-native, so the snippets assume an async context, with
clientanddocfrom the quick start.OpenAI Agents SDK — ships with the SDK
More configuration options
Anthropic SDK tool runner —
pip install "pageindex[anthropic]"More configuration options
Claude Agent SDK —
pip install "pageindex[claude]"More configuration options
Any other framework — no extras needed
Drop to the explicit calls to customize. All of these accept
doc_id=...to point the agent at specific documents, andinclude_management=Trueto also expose document deletion (off by default).What works where
Bring your own agent — the tools, on every major surface:
agent_tools()— plain functions, any frameworkas_openai_tools()/openai_agent_config()— OpenAI Agents SDKhosted=True: execution on OpenAI's side, read-only endpoint by default)as_anthropic_tools()/anthropic_runner_config()— Anthropic SDK tool runneras_claude_mcp()/claude_agent_config()— Claude Agent SDK / Claude Codeapi.pageindex.ai/mcp(read-only:…/mcp?tools=read) — any MCP host, the Anthropic MCP connector, OpenAI hosted MCPManaged chat — the SDK runs the loop:
chat()chat_completions()chat_completions()responses()messages()tool_runnerLocal serves four read-only tools:
browse_documents,get_document,get_document_structure,get_page_content(remove_documentonly withinclude_management=True). Cloud addssearch_documents, folders, andget_document_image— discovered live from the server, never frozen into the SDK.What a run looks like
An actual run (local mode, OpenAI Agents SDK, over
examples/documents/q1-fy25-earnings.pdf):This is the intended loop: the agent reads the tree structure first, picks tight page ranges, and answers from tool output with page citations. No vector index, no chunking. The retrieval intelligence is your agent's own model — the navigation tools make no LLM calls.
Everything below is design rationale and the test record, written for reviewers. You don't need it to use the SDK.
Design — the tools layer
browse_documents/get_document/get_document_structure/get_page_content, doc_name-addressed, same input schemas, descriptions, and JSON response envelopes as the hosted MCP server'stools/list— agent prompts port unchanged between the cloud MCP connection and these in-process tools. Adapters hand the contract/server schema to the framework verbatim (FunctionTool(params_json_schema=…),beta_tool(input_schema=…)) — no regeneration from Python signatures, soitems/enum/pattern/bounds survive on every surface.tests/data/cloud_mcp_contract.jsonfreezes the contract; a parity test guards drift.search_documents,get_document_image) are not registered, mirroring the server's gating semantics. Cloud-only parameters (folder_id,sort/query,recursive) are hidden from the local surface entirely — strict-schema frameworks then cannot express the dead-end calls, and the call_tool/MCP path still answers direct calls with a guided "works on PageIndex cloud" envelope as the backstop. Local descriptions and instructions teach only that surface: the exposed schema is the contract minus the documented hidden set (mechanically asserted), and a dead-reference test keeps local guidance from naming cloud-only tools.remove_documentis off by default, behindinclude_management=True.as_claude_mcp(),as_openai_tools(hosted=True), the raw connector URL — point at the server's read-only endpoint (/mcp?tools=read) by default, so the URL itself is the gate and works identically in every MCP client. The in-process surfaces (agent_tools(),as_openai_tools(),as_anthropic_tools()) expose only tools the server marksreadOnlyHint; local withholdsremove_documentat registration.include_management=Trueis the one switch that opens the complete list in either mode, on every surface. All four tool exports stay polymorphic: on a cloud client the live tool set — including new server-side tools — arrives without an SDK release.{"error", "errorCode", "next_steps"}envelope the cloud emits, flagged through each channel that has one (MCPisErrorpropagated, Anthropic tool runneris_error: trueviaToolError), so the model can always tell a failed call from data. Destructive calls validate every argument before acting — a rejection envelope means nothing was deleted.agent_instructions(doc_id=None)supplies the retrieval playbook for the agent's system prompt. Cloud: the live instructions the MCP server serves for the key's tool set, captured from theinitializehandshake over the same bridge session — server-side guidance updates arrive without an SDK release, and an empty server response raises instead of silently substituting. Local: the built-in playbook for the in-process tools, a trimmed subset with a consistency test that every tool it names exists locally.doc_id(str or list) appends the target documents — in the run above it is what let the agent skip discovery.openai-agents, the chat engine (chat() is the front door; ~15 MB on a base tree already carrying litellm + openai; the>=0.18.1floor is the live-probed minimum that works with current openai, and the latest release passes the full suite).claude-agent-sdk/anthropicstay call-time imports behind vendor extras with actionable errors; the[openai]extra remains declared but empty so existing install commands keep resolving. Thelitellmfloor rises to1.97.0— the release whose bridge routes sol-class chatcmpl+tools calls onto/v1/responsesautomatically (A/B-verified against 1.96.2), so the default chat model works out of the box. That release also ships a Python 3.10 defect —Message/Deltaannotations whose nested forward refs don't resolve, killing everycompletion()(upstream [Bug]: pydantic.errors.PydanticUserError:Messageis not fully defined BerriAI/litellm#36384) — repaired by a version-gated, best-effort model rebuild at our completion gateways that no-ops on 3.11+ and on fixed releases.submit_document(wait=True)polls with growing intervals; returns oncompleted, raises onfailedor after 30 minutes — the manual polling loop cloud callers write today spins forever on a failed document.summary_model).page_index_flash()takesoptimize="full"(default) /"merge"/False;Trueis accepted as"full"for backward compatibility, unknown values raise instead of silently degrading. The CLI's--mode {flash,standard}replaces--flash(kept as a hidden compatibility alias); standard-only tuning flags now error in flash mode instead of being silently ignored; the missing-key pre-check (litellm.validate_environment, all providers) runs only when an LLM will actually be called. Both modes emit identical output schemas end to end.Design — chat on the tools
/chat/completionsquirks (history flattening, bespoke prompt, stateless re-reading, arbitrary caps) are not mirrored; local targets the standard formats and becomes the reference the cloud can later converge toward.responses()/messages()raise on cloud clients until then.AGENT_INSTRUCTIONS; callersystemcontent appended, not rejected; thedoc_idtargeting block leads the conversation and the tool layer enforces it), tool execution (read-only local set, scoped todoc_id), and billing (usage aggregation, envelope ids). Per-run tracing is disabled; prompt-cache routing keys are per-conversation, never pooled across users.tool_runnerformessages()(floor0.108.0— the first release whose runner stops at a refusal carrying atool_useblock instead of executing the tool; verified by probing mock transports against 0.84.0 through 0.108.0). The chat lane routes every model through LiteLLM — bare names are OpenAI-compatible shorthand (env config unchanged), and no prefix triggers a direct lane, so a routing decision can never hide in a model-name spelling;responses()stays on the OpenAI SDK natively (LiteLLM cannot speak the Responses wire format). Rule of the layer: engines = each vendor's official thin loop; agent hosts (Claude Code et al.) only ever get tools.responses()carries the backend's real terminalstatus/incomplete_details(recorded at the transport layer — the engine discards them), a partial page read names every omitted page, and framework exceptions surface asPageIndexAPIError, never as raw engine types.cache_controlbreakpoints sit on the managed system blocks only. The same decision reaches the LiteLLM lane: Claude models routed through LiteLLM — Anthropic direct, Bedrock, and Vertex, resolved by LiteLLM's ownget_llm_provider— pass LiteLLM'scache_control_injection_points(via the Agents SDK'sextra_args, both documented parameters), so the managed prefix (tools + instructions + doc block) caches there too. Each channel live-verified with a write→read cycle (anthropic per-turn reads through the full stack; Bedrock 7264 and Vertex 4842 tokens read on the second call).chat()is output sugar, not a fourth protocol. It returns the answer string and hides the envelope; the wire underneath ischat_completions()unchanged, so it works on every backend in both modes. Its contract is the one surface not pinned to a wire format — a futureengine=selector is a non-breaking add — while a merged multi-protocol method stays rejected: round-trip formats are protocol-specific, so a switch parameter abstracts nothing.ConfigLoader.load()fills index/summary/chat from whichever names were given — new names win over old, specific over general,modelsets every role, and the built-in defaults close each chain. The packaged config.yaml ships no model keys (comments only), so key presence is what distinguishes a user's choice from a default; every name that ever shipped in a release stays accepted, without deprecation warnings.reasoning_effort/reasoning/thinking;max_tokens/max_output_tokens), values forwarded untranslated with no defaults of ours — capability rules are the backend's (LiteLLM refuses unsupported combinations loudly, wrapped asPageIndexAPIError). The named sampling knobs ride ModelSettings fields — the one channel clean on every lane — whileextra_bodycovers fields the methods don't name: OpenAI destinations get a true body merge, LiteLLM providers take the keys as LiteLLM's own params,messages()hands them to the Anthropic SDK's nativeextra_body. Merged last, so caller keys win.index_backend/chat_backend(and per-callbackend) hand each stack its own connection params untranslated: the indexing gateways take the dict as LiteLLM call kwargs scoped by a contextvar (the env-key pre-check yields to it; the openai-SDK fast path normalizesapi_base), the chat lane liftsapi_key/base_urlintoLitellmModel's two pinned constructor slots and rides the rest as call kwargs,responses()/messages()construct their SDK clients from the dict (unknown keys raise, wrapped uniformly). Per-call wins over per-client;chat()takes no per-call backend but honors the client's; config bundles deliberately carry no credentials — they run in your environment.enable_citationsraises as cloud-only (citations need block-level OCR data local mode does not store).messages()resolves itsmax_tokensdefault per model (8192, or 4096 for the claude-3 generation), so the simple call needs only a question on any model.Verification
Modelfake under openai-agents, a mock HTTP transport under the real anthropic SDK); contract parity vs the frozen snapshot; framework-missing/-installed behavior both ways; streaming on every chat surface; thechat()front door (answer extraction, streamed chunks, multi-turn history passthrough, cloud envelope unwrap); the round-trip prefix-extension assertions on both engines; doc_id scoping, error-marking, and envelope-honesty regressions; a model-resolution matrix with one row per released knob generation; per-door passthrough delivery (reasoning channels, sampling knobs on ModelSettings,extra_bodyasserted on the captured anthropic wire body); backend delivery per lane (contextvar scoping on both gateway paths, the env-precheck yield, constructor lift and kwargs remainder on the LiteLLM lane, SDK-client construction on responses/messages, merge precedence) andextra_headerson every door (anthropic-betaasserted on the anthropic wire).agent_tools()andas_anthropic_tools()discovered this key's gated tool set (7 read-only tools;include_management=Trueaddsremove_document); frozen-contract parity letter-for-letter; envelope field parity on the analogous calls; the server serves non-emptyinitialize.instructions.chat_completionsanswered with the structure-first loop;responsesround-trip answered the follow-up with zero new tool calls. Anthropic —messages()history was accepted verbatim by the real API, follow-up answered with zero new tool turns,cache_controlhit live (cache_read_input_tokens: 1826), native streaming; both the sync andAsyncAnthropictool runners drove the live cloud tools end-to-end; the Messages API MCP connector reachedapi.pageindex.ai/mcpserver-side (mcp_tool_use/mcp_tool_resultin a single call).d87fa89,b135711,eb1a230), the remainder triaged with rationale. Nine further review rounds followed (commit messages31c9150througheebed64), covering argument coercion, protocol terminal states, envelope honesty, provider error containment, CodeQL findings, and SDK dependency floors — each finding reproduced before the fix landed. The feat: model knobs, LiteLLM-verbatim chat lane, per-door passthrough params #409 wave (model knobs, routing flip, passthrough params) went through three more audits with wire-level probes: LiteLLM'smax_tokenstranslation verified on both of its paths (chatcmplmax_completion_tokens, bridgemax_output_tokens),extra_bodydelivery captured per lane, and the sol-bridge litellm floor bisected to the exact release by source and behavioral A/B. The backend wave (feat: backend connection overrides; extra_headers on every local door #411) ran three further audits backed by ~20 mock-server wire probes that mapped LiteLLM's channel semantics per adapter (kwargs vsextra_bodyvs headers), re-proved custom-header forwarding after catching a case-sensitivity bug in the probe itself, and isolated theanthropic-betadrop to LiteLLM'scompletion()plumbing — its transformation layer merges user betas when called directly. The post-dev5 hardening (Aug 18–19,03ffab3through49a24e1) ran five more maximum-effort rounds across the tool layer, chat lanes, and flash: the standaloneagent_instructionsshadow guard re-armed strict (the*_agent_configbundles keep the relaxed in-set check they can prove), caller-owned transports surviving the per-call backend closes, cloud discovery and instructions riding the gated?tools=readendpoint (live-verified: 7 annotated read-only tools; instructions byte-identical on both endpoints today), thinking-safe runner defaults, explicitoptimize=precedence over the deprecated modifier, mcp 2.0 compatibility, and an.env-independent suite — every finding reproduced before its fix, withdrawals and non-changes triaged with rationale in the commit bodies.responses()envelope — officialoutput(model items only) +items(full transcript for round-trip) + usage aggregated across turns, verified against the real OpenAI API; python floor declared>=3.10; staleanthropic>=0.84.0hints updated to the real 0.108.0 floor; the bridge's binary-stub behavior disclosed on the two image-advertising tool surfaces;_run_syncmoved off theexcept RuntimeErrorprobe so user exceptions stop carrying a phantom "no running event loop" context.Release gate — satisfied: the default cloud configs point at the read-only MCP endpoint, so VectifyAI/pageindex-chat#448 had to be deployed before 0.2.10 ships (an older server ignores the
tools=readparameter and would silently serve the full set behind a URL that promises read-only). Verified live before publishing0.2.10.dev1:…/mcp?tools=readserves 7 tools withoutremove_document,…/mcpserves 8 with it.Follow-ups (not in this PR): an
AsyncPageIndexClienttwin per the industry dual-client pattern — every layer around the SDK is already async-native (FastAPI server, agent engines, agent frameworks); the async chat path is the engines' native form (drops the sync bridge, streams pass through asasync for), and cloud transport gains an httpx track; a stdiopageindex-mcpentry point for non-Python MCP hosts; a publicdoc_idscope on the BYO tool exports (the chat surfaces already enforce it); the docs-site agent-integration page; cloud/responses·/messagesconvergence toward these surfaces.