feat(cli): configurable insert_mode (fail-fast / fail-at-end) for strict compile-failure handling

[sebastianbraun25]

Note

This PR was created in collaboration between a human and AI: implementation, tests, and
PR text were created by an AI assistant under the guidance and review of the human author.

Problem

openkb add currently reports a whole-file "added" outcome even when one or more individual
concepts/entities failed to generate during compilation. _compile_concepts in
openkb/agent/compiler.py collects exceptions from the per-concept/per-entity generation tasks
via asyncio.gather(..., return_exceptions=True), logs a [WARN] ... planned but only N written
line, and continues — but the file's hash is still registered and the source is still eligible for
auto_delete_added_files, so a partially-compiled document looks identical to a fully successful
one from the CLI/API caller's point of view.

Solution / Changes

• New insert_mode config key (.openkb/config.yaml) with three values:
  • "normal" (default): unchanged behavior — a concept/entity generation failure during compile
    is logged as a warning and the file is still reported "added".
  • "fail-fast": the first concept/entity generation failure cancels every other still-pending
    generation in the batch and immediately raises ConceptCompilationError — nothing from the
    batch is written.
  • "fail-at-end": every planned concept/entity generation is attempted (so every failure for the
    document is logged in one pass) before ConceptCompilationError is raised if anything failed.
• Both strict modes rely entirely on the existing mutation-snapshot rollback
  (openkb.add_coordinator/openkb.mutation) to discard the add and report it "failed" — no new
  rollback path needed. The existing "keep raw/ on failed" and "keep the debug log on a
  non-'added' outcome" behaviors already cover raw-file and log-preservation for strict mode.
• _compile_concepts's three early-return paths (unparseable plan, scalar plan,
  all-items-filtered-as-malformed) now also raise under a strict insert_mode, not just individual
  concept/entity generation failures — a genuinely empty plan (nothing was ever planned) still
  counts as complete success in every mode.
• compile_short_doc/compile_long_doc resolve insert_mode from the already-loaded KB config, so
  no CLI-level plumbing is needed.
• Backward compatible: default "normal" behavior is unchanged.

Issues

• Resolvesfeat(cli): configurable insert_mode (fail-fast / fail-at-end) for strict compile-failure handling #239

[sebastianbraun25]

Note

This PR was created in collaboration between a human and AI: implementation, tests, and
PR text were created by an AI assistant under the guidance and review of the human author.

Problem

openkb add currently reports a whole-file "added" outcome even when one or more individual
concepts/entities failed to generate during compilation. _compile_concepts in
openkb/agent/compiler.py collects exceptions from the per-concept/per-entity generation tasks
via asyncio.gather(..., return_exceptions=True), logs a [WARN] ... planned but only N written
line, and continues — but the file's hash is still registered and the source is still eligible for
auto_delete_added_files, so a partially-compiled document looks identical to a fully successful
one from the CLI/API caller's point of view.

Solution / Changes

• New insert_mode config key (.openkb/config.yaml) with three values:
  • "normal" (default): unchanged behavior — a concept/entity generation failure during compile
    is logged as a warning and the file is still reported "added".
  • "fail-fast": the first concept/entity generation failure cancels every other still-pending
    generation in the batch and immediately raises ConceptCompilationError — nothing from the
    batch is written.
  • "fail-at-end": every planned concept/entity generation is attempted (so every failure for the
    document is logged in one pass) before ConceptCompilationError is raised if anything failed.
• Both strict modes rely entirely on the existing mutation-snapshot rollback
  (openkb.add_coordinator/openkb.mutation) to discard the add and report it "failed" — no new
  rollback path needed. The existing "keep raw/ on failed" and "keep the debug log on a
  non-'added' outcome" behaviors already cover raw-file and log-preservation for strict mode.
• _compile_concepts's three early-return paths (unparseable plan, scalar plan,
  all-items-filtered-as-malformed) now also raise under a strict insert_mode, not just individual
  concept/entity generation failures — a genuinely empty plan (nothing was ever planned) still
  counts as complete success in every mode.
• compile_short_doc/compile_long_doc resolve insert_mode from the already-loaded KB config, so
  no CLI-level plumbing is needed.
• Backward compatible: default "normal" behavior is unchanged.

Issues

• Resolvesfeat(cli): configurable insert_mode (fail-fast / fail-at-end) for strict compile-failure handling #239

[sebastianbraun25]

Note

This PR was created in collaboration between a human and AI: implementation, tests, and
PR text were created by an AI assistant under the guidance and review of the human author.

Problem

openkb add currently reports a whole-file "added" outcome even when one or more individual
concepts/entities failed to generate during compilation. _compile_concepts in
openkb/agent/compiler.py collects exceptions from the per-concept/per-entity generation tasks
via asyncio.gather(..., return_exceptions=True), logs a [WARN] ... planned but only N written
line, and continues — but the file's hash is still registered and the source is still eligible for
auto_delete_added_files, so a partially-compiled document looks identical to a fully successful
one from the CLI/API caller's point of view.

Solution / Changes

• New insert_mode config key (.openkb/config.yaml) with three values:
  • "normal" (default): unchanged behavior — a concept/entity generation failure during compile
    is logged as a warning and the file is still reported "added".
  • "fail-fast": the first concept/entity generation failure cancels every other still-pending
    generation in the batch and immediately raises ConceptCompilationError — nothing from the
    batch is written.
  • "fail-at-end": every planned concept/entity generation is attempted (so every failure for the
    document is logged in one pass) before ConceptCompilationError is raised if anything failed.
• Both strict modes rely entirely on the existing mutation-snapshot rollback
  (openkb.add_coordinator/openkb.mutation) to discard the add and report it "failed" — no new
  rollback path needed. The existing "keep raw/ on failed" and "keep the debug log on a
  non-'added' outcome" behaviors already cover raw-file and log-preservation for strict mode.
• _compile_concepts's three early-return paths (unparseable plan, scalar plan,
  all-items-filtered-as-malformed) now also raise under a strict insert_mode, not just individual
  concept/entity generation failures — a genuinely empty plan (nothing was ever planned) still
  counts as complete success in every mode.
• compile_short_doc/compile_long_doc resolve insert_mode from the already-loaded KB config, so
  no CLI-level plumbing is needed.
• Backward compatible: default "normal" behavior is unchanged.

Issues

• Resolvesfeat(cli): configurable insert_mode (fail-fast / fail-at-end) for strict compile-failure handling #239

[sebastianbraun25]

Note

This PR was created in collaboration between a human and AI: implementation, tests, and
PR text were created by an AI assistant under the guidance and review of the human author.

Problem

openkb add currently reports a whole-file "added" outcome even when one or more individual
concepts/entities failed to generate during compilation. _compile_concepts in
openkb/agent/compiler.py collects exceptions from the per-concept/per-entity generation tasks
via asyncio.gather(..., return_exceptions=True), logs a [WARN] ... planned but only N written
line, and continues — but the file's hash is still registered and the source is still eligible for
auto_delete_added_files, so a partially-compiled document looks identical to a fully successful
one from the CLI/API caller's point of view.

Solution / Changes

• New insert_mode config key (.openkb/config.yaml) with three values:
  • "normal" (default): unchanged behavior — a concept/entity generation failure during compile
    is logged as a warning and the file is still reported "added".
  • "fail-fast": the first concept/entity generation failure cancels every other still-pending
    generation in the batch and immediately raises ConceptCompilationError — nothing from the
    batch is written.
  • "fail-at-end": every planned concept/entity generation is attempted (so every failure for the
    document is logged in one pass) before ConceptCompilationError is raised if anything failed.
• Both strict modes rely entirely on the existing mutation-snapshot rollback
  (openkb.add_coordinator/openkb.mutation) to discard the add and report it "failed" — no new
  rollback path needed. The existing "keep raw/ on failed" and "keep the debug log on a
  non-'added' outcome" behaviors already cover raw-file and log-preservation for strict mode.
• _compile_concepts's three early-return paths (unparseable plan, scalar plan,
  all-items-filtered-as-malformed) now also raise under a strict insert_mode, not just individual
  concept/entity generation failures — a genuinely empty plan (nothing was ever planned) still
  counts as complete success in every mode.
• compile_short_doc/compile_long_doc resolve insert_mode from the already-loaded KB config, so
  no CLI-level plumbing is needed.
• Backward compatible: default "normal" behavior is unchanged.

Issues

• Resolvesfeat(cli): configurable insert_mode (fail-fast / fail-at-end) for strict compile-failure handling #239

[sebastianbraun25]

Note

This PR was created in collaboration between a human and AI: implementation, tests, and
PR text were created by an AI assistant under the guidance and review of the human author.

Problem

openkb add currently reports a whole-file "added" outcome even when one or more individual
concepts/entities failed to generate during compilation. _compile_concepts in
openkb/agent/compiler.py collects exceptions from the per-concept/per-entity generation tasks
via asyncio.gather(..., return_exceptions=True), logs a [WARN] ... planned but only N written
line, and continues — but the file's hash is still registered and the source is still eligible for
auto_delete_added_files, so a partially-compiled document looks identical to a fully successful
one from the CLI/API caller's point of view.

Solution / Changes

• New insert_mode config key (.openkb/config.yaml) with three values:
  • "normal" (default): unchanged behavior — a concept/entity generation failure during compile
    is logged as a warning and the file is still reported "added".
  • "fail-fast": the first concept/entity generation failure cancels every other still-pending
    generation in the batch and immediately raises ConceptCompilationError — nothing from the
    batch is written.
  • "fail-at-end": every planned concept/entity generation is attempted (so every failure for the
    document is logged in one pass) before ConceptCompilationError is raised if anything failed.
• Both strict modes rely entirely on the existing mutation-snapshot rollback
  (openkb.add_coordinator/openkb.mutation) to discard the add and report it "failed" — no new
  rollback path needed. The existing "keep raw/ on failed" and "keep the debug log on a
  non-'added' outcome" behaviors already cover raw-file and log-preservation for strict mode.
• _compile_concepts's three early-return paths (unparseable plan, scalar plan,
  all-items-filtered-as-malformed) now also raise under a strict insert_mode, not just individual
  concept/entity generation failures — a genuinely empty plan (nothing was ever planned) still
  counts as complete success in every mode.
• compile_short_doc/compile_long_doc resolve insert_mode from the already-loaded KB config, so
  no CLI-level plumbing is needed.
• Backward compatible: default "normal" behavior is unchanged.

Issues

• Resolvesfeat(cli): configurable insert_mode (fail-fast / fail-at-end) for strict compile-failure handling #239

[sebastianbraun25]

Note

This PR was created in collaboration between a human and AI: implementation, tests, and
PR text were created by an AI assistant under the guidance and review of the human author.

Problem

openkb add currently reports a whole-file "added" outcome even when one or more individual
concepts/entities failed to generate during compilation. _compile_concepts in
openkb/agent/compiler.py collects exceptions from the per-concept/per-entity generation tasks
via asyncio.gather(..., return_exceptions=True), logs a [WARN] ... planned but only N written
line, and continues — but the file's hash is still registered and the source is still eligible for
auto_delete_added_files, so a partially-compiled document looks identical to a fully successful
one from the CLI/API caller's point of view.

Solution / Changes

• New insert_mode config key (.openkb/config.yaml) with three values:
  • "normal" (default): unchanged behavior — a concept/entity generation failure during compile
    is logged as a warning and the file is still reported "added".
  • "fail-fast": the first concept/entity generation failure cancels every other still-pending
    generation in the batch and immediately raises ConceptCompilationError — nothing from the
    batch is written.
  • "fail-at-end": every planned concept/entity generation is attempted (so every failure for the
    document is logged in one pass) before ConceptCompilationError is raised if anything failed.
• Both strict modes rely entirely on the existing mutation-snapshot rollback
  (openkb.add_coordinator/openkb.mutation) to discard the add and report it "failed" — no new
  rollback path needed. The existing "keep raw/ on failed" and "keep the debug log on a
  non-'added' outcome" behaviors already cover raw-file and log-preservation for strict mode.
• _compile_concepts's three early-return paths (unparseable plan, scalar plan,
  all-items-filtered-as-malformed) now also raise under a strict insert_mode, not just individual
  concept/entity generation failures — a genuinely empty plan (nothing was ever planned) still
  counts as complete success in every mode.
• compile_short_doc/compile_long_doc resolve insert_mode from the already-loaded KB config, so
  no CLI-level plumbing is needed.
• Backward compatible: default "normal" behavior is unchanged.

Issues

• Resolvesfeat(cli): configurable insert_mode (fail-fast / fail-at-end) for strict compile-failure handling #239

[sebastianbraun25]

Note

This PR was created in collaboration between a human and AI: implementation, tests, and
PR text were created by an AI assistant under the guidance and review of the human author.

Problem

openkb add currently reports a whole-file "added" outcome even when one or more individual
concepts/entities failed to generate during compilation. _compile_concepts in
openkb/agent/compiler.py collects exceptions from the per-concept/per-entity generation tasks
via asyncio.gather(..., return_exceptions=True), logs a [WARN] ... planned but only N written
line, and continues — but the file's hash is still registered and the source is still eligible for
auto_delete_added_files, so a partially-compiled document looks identical to a fully successful
one from the CLI/API caller's point of view.

Solution / Changes

• New insert_mode config key (.openkb/config.yaml) with three values:
  • "normal" (default): unchanged behavior — a concept/entity generation failure during compile
    is logged as a warning and the file is still reported "added".
  • "fail-fast": the first concept/entity generation failure cancels every other still-pending
    generation in the batch and immediately raises ConceptCompilationError — nothing from the
    batch is written.
  • "fail-at-end": every planned concept/entity generation is attempted (so every failure for the
    document is logged in one pass) before ConceptCompilationError is raised if anything failed.
• Both strict modes rely entirely on the existing mutation-snapshot rollback
  (openkb.add_coordinator/openkb.mutation) to discard the add and report it "failed" — no new
  rollback path needed. The existing "keep raw/ on failed" and "keep the debug log on a
  non-'added' outcome" behaviors already cover raw-file and log-preservation for strict mode.
• _compile_concepts's three early-return paths (unparseable plan, scalar plan,
  all-items-filtered-as-malformed) now also raise under a strict insert_mode, not just individual
  concept/entity generation failures — a genuinely empty plan (nothing was ever planned) still
  counts as complete success in every mode.
• compile_short_doc/compile_long_doc resolve insert_mode from the already-loaded KB config, so
  no CLI-level plumbing is needed.
• Backward compatible: default "normal" behavior is unchanged.

Issues

• Resolvesfeat(cli): configurable insert_mode (fail-fast / fail-at-end) for strict compile-failure handling #239

[sebastianbraun25]

Note

This PR was created in collaboration between a human and AI: implementation, tests, and
PR text were created by an AI assistant under the guidance and review of the human author.

Problem

openkb add currently reports a whole-file "added" outcome even when one or more individual
concepts/entities failed to generate during compilation. _compile_concepts in
openkb/agent/compiler.py collects exceptions from the per-concept/per-entity generation tasks
via asyncio.gather(..., return_exceptions=True), logs a [WARN] ... planned but only N written
line, and continues — but the file's hash is still registered and the source is still eligible for
auto_delete_added_files, so a partially-compiled document looks identical to a fully successful
one from the CLI/API caller's point of view.

Solution / Changes

• New insert_mode config key (.openkb/config.yaml) with three values:
  • "normal" (default): unchanged behavior — a concept/entity generation failure during compile
    is logged as a warning and the file is still reported "added".
  • "fail-fast": the first concept/entity generation failure cancels every other still-pending
    generation in the batch and immediately raises ConceptCompilationError — nothing from the
    batch is written.
  • "fail-at-end": every planned concept/entity generation is attempted (so every failure for the
    document is logged in one pass) before ConceptCompilationError is raised if anything failed.
• Both strict modes rely entirely on the existing mutation-snapshot rollback
  (openkb.add_coordinator/openkb.mutation) to discard the add and report it "failed" — no new
  rollback path needed. The existing "keep raw/ on failed" and "keep the debug log on a
  non-'added' outcome" behaviors already cover raw-file and log-preservation for strict mode.
• _compile_concepts's three early-return paths (unparseable plan, scalar plan,
  all-items-filtered-as-malformed) now also raise under a strict insert_mode, not just individual
  concept/entity generation failures — a genuinely empty plan (nothing was ever planned) still
  counts as complete success in every mode.
• compile_short_doc/compile_long_doc resolve insert_mode from the already-loaded KB config, so
  no CLI-level plumbing is needed.
• Backward compatible: default "normal" behavior is unchanged.

Issues

• Resolvesfeat(cli): configurable insert_mode (fail-fast / fail-at-end) for strict compile-failure handling #239