Problem
openkb add currently reports a whole-file "added" outcome even when one or more individual
concepts/entities failed to generate during compilation. _compile_concepts in
openkb/agent/compiler.py collects exceptions from the per-concept/per-entity generation tasks
via asyncio.gather(..., return_exceptions=True), logs a [WARN] ... planned but only N written
line, and continues — but the file's hash is still registered and the source is still eligible for
auto_delete_added_files, so a partially-compiled document looks identical to a fully successful
one from the CLI/API caller's point of view. There is currently no config or CLI flag to change
this behavior.
Reproduction
openkb add some-large-doc.pdf with a transient LLM failure affecting a subset of planned
concepts (e.g. a rate limit or timeout hit on one of several concurrent concept-generation
calls).- Console shows
[WARN] N concept(s) planned but only M written for <doc> (...). - Console still ends with
[OK] <file> added to knowledge base. and the CLI/API status is
"added" — indistinguishable from a fully complete compile.
Kontext
openkb/agent/compiler.py: _compile_concepts (concept/entity generation gather), compile_short_doc,
compile_long_doc.openkb/cli.py: _add_single_file_locked / _add_single_file (outcome determination), the
existing Literal["added", "skipped", "failed"] status contract.openkb/mutation.py / openkb/add_coordinator.py: existing snapshot+journal rollback
infrastructure, already fully wired for any exception raised during commit_body.openkb/config.py: DEFAULT_CONFIG (e.g. auto_delete_added_files, debug) — the pattern to
extend with a new key.
Vorschlag
Add an insert_mode config key (per-KB, default "normal" = current behavior, unchanged) with two
opt-in strict variants:
"fail-fast": abort the compile as soon as the first concept/entity generation fails, without
waiting for other in-flight/queued concurrent generations — minimizing wasted LLM calls/tokens —
then roll back via the existing mutation snapshot and report "failed"."fail-at-end": let the compile run to completion (every planned concept/entity is attempted, so
the log captures every failure for the file in one pass), then — if any failed — roll back via the
existing mutation snapshot and report "failed", exactly like fail-fast in outcome.
In both strict modes: the raw file is preserved (existing "keep raw/ on failed" behavior already
covers this), the mutation rollback already reverts any wiki/concepts, wiki/entities, index, and
raw/wiki/sources changes, and the per-file debug log is already preserved on any non-"added"
outcome — no changes needed to those three mechanisms, only to how/when a compile signals failure.
For a batch add (directory), each file's outcome remains independent — one file failing in strict
mode does not stop the batch from continuing to the next file (existing behavior, unchanged).
This issue was drafted with the assistance of an AI assistant.
Problem
openkb addcurrently reports a whole-file"added"outcome even when one or more individualconcepts/entities failed to generate during compilation.
_compile_conceptsinopenkb/agent/compiler.pycollects exceptions from the per-concept/per-entity generation tasksvia
asyncio.gather(..., return_exceptions=True), logs a[WARN] ... planned but only N writtenline, and continues — but the file's hash is still registered and the source is still eligible for
auto_delete_added_files, so a partially-compiled document looks identical to a fully successfulone from the CLI/API caller's point of view. There is currently no config or CLI flag to change
this behavior.
Reproduction
openkb add some-large-doc.pdfwith a transient LLM failure affecting a subset of plannedconcepts (e.g. a rate limit or timeout hit on one of several concurrent concept-generation
calls).
[WARN] N concept(s) planned but only M written for <doc> (...).[OK] <file> added to knowledge base.and the CLI/API status is"added"— indistinguishable from a fully complete compile.Kontext
openkb/agent/compiler.py:_compile_concepts(concept/entity generation gather),compile_short_doc,compile_long_doc.openkb/cli.py:_add_single_file_locked/_add_single_file(outcome determination), theexisting
Literal["added", "skipped", "failed"]status contract.openkb/mutation.py/openkb/add_coordinator.py: existing snapshot+journal rollbackinfrastructure, already fully wired for any exception raised during
commit_body.openkb/config.py:DEFAULT_CONFIG(e.g.auto_delete_added_files,debug) — the pattern toextend with a new key.
Vorschlag
Add an
insert_modeconfig key (per-KB, default"normal"= current behavior, unchanged) with twoopt-in strict variants:
"fail-fast": abort the compile as soon as the first concept/entity generation fails, withoutwaiting for other in-flight/queued concurrent generations — minimizing wasted LLM calls/tokens —
then roll back via the existing mutation snapshot and report
"failed"."fail-at-end": let the compile run to completion (every planned concept/entity is attempted, sothe log captures every failure for the file in one pass), then — if any failed — roll back via the
existing mutation snapshot and report
"failed", exactly likefail-fastin outcome.In both strict modes: the raw file is preserved (existing "keep raw/ on failed" behavior already
covers this), the mutation rollback already reverts any wiki/concepts, wiki/entities, index, and
raw/wiki/sources changes, and the per-file debug log is already preserved on any non-
"added"outcome — no changes needed to those three mechanisms, only to how/when a compile signals failure.
For a batch add (directory), each file's outcome remains independent — one file failing in strict
mode does not stop the batch from continuing to the next file (existing behavior, unchanged).
This issue was drafted with the assistance of an AI assistant.