Skip to content

feat(opencode): support per-model compaction config - #43713

Open
joserdf wants to merge 1 commit into
anomalyco:devfrom
joserdf:feat/per-model-compaction
Open

feat(opencode): support per-model compaction config#43713
joserdf wants to merge 1 commit into
anomalyco:devfrom
joserdf:feat/per-model-compaction

Conversation

@joserdf

Copy link
Copy Markdown

Issue for this PR

Closes#43703

Type of change

  • New feature

What does this PR do?

Adds per-model compaction configuration so models with different context
windows can use different compaction thresholds.

Before: the top-level compaction block (auto, prune,
tail_turns, preserve_recent_tokens, reserved) was global — one
value applied to every model. The only per-model lever was
limit.context, which shifts the trigger point but cannot set an
independent buffer per model.

After: each model can override any compaction field via
provider.<id>.models.<model>.compaction. Model-level fields are merged
over the global config (model wins per-field), so a 32k local model can
compact early while a 128k frontier model keeps a larger usable window.

Example:

{
"provider": {
"my-provider": {
"models": {
"small-model": {
"limit": { "context": 32000, "output": 4096 },
"compaction": { "reserved": 4000 }
},
"big-model": {
"limit": { "context": 128000, "output": 8192 },
"compaction": { "reserved": 24000 }
}
}
}
}
}

Implementation:

  • Extracted the compaction schema into ConfigProviderV1.Compaction and
    reused it for both the global config and the per-model Model schema
    (single source of truth, no behavior change for existing configs).
  • Carried the resolved compaction config onto the runtime
    Provider.Model so the session layer can read it.
  • Added compactionConfig() in session/overflow.ts that merges
    model-level fields over the global block; usable(), isOverflow(),
    select() (tail_turns / preserve_recent_tokens) and the auto-compact
    call sites in session/prompt.ts now resolve through it.
  • prune intentionally stays on the global config: it is a background
    cleanup task keyed only by sessionID, and the model is not readily
    available at that call site.

How did you verify your code works?

  • tsgo --noEmit passes in packages/core and packages/opencode.
  • bun test test/session/compaction.test.ts — 59 pass, 1 skip, 0 fail
    (added 4 tests: per-model auto, per-model reserved, per-model
    tail_turns, and a regression check that models without an override
    keep using the global config).
  • bun test test/provider/ — 557 pass, 0 fail.
  • bun test test/config/ (core) — 50 pass, 0 fail.
  • The 5 failures in the full test/session/ run are pre-existing
    inotify_add_watch: No space left on device environment failures —
    they fail identically on the base dev branch without these changes.

Screenshots / recordings

N/A (not a UI change)

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Allow overriding the global compaction block per model via
provider.<id>.models.<model>.compaction. Model-level fields (auto,
prune, tail_turns, preserve_recent_tokens, reserved) are merged over
the global config, so models with different context windows can use
different compaction thresholds.
- Extract the compaction schema into ConfigProviderV1.Compaction and
reuse it for both the global config and per-model overrides
- Carry the resolved compaction config on the runtime Provider.Model
- Resolve compaction settings through compactionConfig() in overflow
and session compaction (reserved, auto, tail_turns,
preserve_recent_tokens)
- Resolve auto-compaction flag per model at the create call sites
- Add tests for per-model auto, reserved, and tail_turns overrides
@Enough1122

Copy link
Copy Markdown

AI code review — automated review for reference; please use your judgment.

  • packages/opencode/src/provider/provider.ts:1057-1063 — the inline Compaction struct duplicates ConfigProviderV1.Compaction but with weaker validation: here fields are plain Schema.Number (negatives accepted), while the core config schema uses NonNegativeInt. A models.json/provider-metadata entry with e.g. reserved: -1000 would validate here and then inflate usable() (input.limit.input - (-1000)), silently raising overflow thresholds. Reuse the core schema (export its type too) so both layers agree.
  • packages/opencode/src/provider/provider.ts:1542 — compaction: model.compaction ?? existingModel?.compaction replaces any previously-known per-model block wholesale on provider refresh, inconsistent with the sibling headers: mergeDeep(existingModel?.headers ?? {}, ...). A partial upstream update like {auto: false} will drop an earlier {reserved: 60000} instead of merging. Consider mergeDeep for parity, or document the replace semantics.
  • packages/opencode/src/session/prompt.ts:1166-1171 and 1329-1334 — auto: model.compaction?.auto ?? true conflates "the model opted out of auto-compaction" with the create() flag describing who initiated this run. Both call sites are automatic triggers, so with auto: false they now record a user-style compaction; enforcement arguably belongs in isOverflow (which already short-circuits) plus an early return here. Worth double-checking the intended semantics of the auto argument.
  • packages/core/src/v1/config/config.ts:146-150 — clean dedup of the inline struct into ConfigProviderV1.Compaction; field-for-field identical annotations, no behavior change. This makes the provider-layer duplication in the first bullet more avoidable.
  • packages/opencode/test/session/compaction.test.ts:566-632 — good override matrix (auto, reserved, tail_turns, unaffected-model isolation). Two gaps worth closing: provider-schema rejection of negative values, and refresh-time merge/partial-update behavior for an existing model's compaction block.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE]: per-model compaction threshold (reserved/buffer)

2 participants

@joserdf@Enough1122