Skip to content

feat(cli): configurable insert_mode (fail-fast / fail-at-end) for strict compile-failure handling #239

Description

@sebastianbraun25

Problem

openkb add currently reports a whole-file "added" outcome even when one or more individual
concepts/entities failed to generate during compilation. _compile_concepts in
openkb/agent/compiler.py collects exceptions from the per-concept/per-entity generation tasks
via asyncio.gather(..., return_exceptions=True), logs a [WARN] ... planned but only N written
line, and continues — but the file's hash is still registered and the source is still eligible for
auto_delete_added_files, so a partially-compiled document looks identical to a fully successful
one from the CLI/API caller's point of view. There is currently no config or CLI flag to change
this behavior.

Reproduction

  1. openkb add some-large-doc.pdf with a transient LLM failure affecting a subset of planned
    concepts (e.g. a rate limit or timeout hit on one of several concurrent concept-generation
    calls).
  2. Console shows [WARN] N concept(s) planned but only M written for <doc> (...).
  3. Console still ends with [OK] <file> added to knowledge base. and the CLI/API status is
    "added" — indistinguishable from a fully complete compile.

Kontext

  • openkb/agent/compiler.py: _compile_concepts (concept/entity generation gather), compile_short_doc,
    compile_long_doc.
  • openkb/cli.py: _add_single_file_locked / _add_single_file (outcome determination), the
    existing Literal["added", "skipped", "failed"] status contract.
  • openkb/mutation.py / openkb/add_coordinator.py: existing snapshot+journal rollback
    infrastructure, already fully wired for any exception raised during commit_body.
  • openkb/config.py: DEFAULT_CONFIG (e.g. auto_delete_added_files, debug) — the pattern to
    extend with a new key.

Vorschlag

Add an insert_mode config key (per-KB, default "normal" = current behavior, unchanged) with two
opt-in strict variants:

  • "fail-fast": abort the compile as soon as the first concept/entity generation fails, without
    waiting for other in-flight/queued concurrent generations — minimizing wasted LLM calls/tokens —
    then roll back via the existing mutation snapshot and report "failed".
  • "fail-at-end": let the compile run to completion (every planned concept/entity is attempted, so
    the log captures every failure for the file in one pass), then — if any failed — roll back via the
    existing mutation snapshot and report "failed", exactly like fail-fast in outcome.

In both strict modes: the raw file is preserved (existing "keep raw/ on failed" behavior already
covers this), the mutation rollback already reverts any wiki/concepts, wiki/entities, index, and
raw/wiki/sources changes, and the per-file debug log is already preserved on any non-"added"
outcome — no changes needed to those three mechanisms, only to how/when a compile signals failure.

For a batch add (directory), each file's outcome remains independent — one file failing in strict
mode does not stop the batch from continuing to the next file (existing behavior, unchanged).


This issue was drafted with the assistance of an AI assistant.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions