ADFA-5187 | Dynamically size n_ctx based on model metadata and available RAM

[jatezzz]

Description

This PR updates the llama.cpp context size (n_ctx) to be dynamically calculated at model load, replacing the hardcoded 4096 value. It extends the GgufModelInspector to parse the <architecture>.context_length from the GGUF metadata header and checks the device's available system RAM. The optimal n_ctx is now computed as the minimum of the model's supported context length, the affordable context within the RAM budget, and a sane system ceiling. This change prevents unexpected Android low-memory kills on lower-end devices while unlocking the full context potential for models and devices that can handle > 4096 tokens. The implementation fails open, gracefully falling back to the default 4096 if metadata is missing or unreadable.

Details

Logic-related changes. Please review the Android logcat during model initialization; you will see logs indicating the parsed model context length, the available RAM snapshot, and the resulting computed n_ctx being passed to LLamaAndroid.configureContext(...) before context creation.

2026-08-21 11:12:52.065 28532-28789 AiAgentLoc...ma-android com.itsaky.androidide I model: loading from /data/user/0/com.itsaky.androidide/files/llm-models/1796906021_675710816_qwen2.5-0.5b-instruct-q8_0.gguf
2026-08-21 11:12:52.346 28532-28789 AiAgentLoc...ma-android com.itsaky.androidide I context: using 6 threads (batch=6)
2026-08-21 11:12:52.363 28532-28789 AiAgentLoc...ma-android com.itsaky.androidide I context: created with n_ctx = 16384 (requested 16384, model trained for 32768), n_batch = 2048
2026-08-21 11:12:56.272 28532-28789 AiAgentLoc...ma-android com.itsaky.androidide I prefill: n_len = 1024, n_ctx = 16384, n_kv_req = 2007
2026-08-21 11:12:56.278 28532-28789 AiAgentLoc...ma-android com.itsaky.androidide I prefill: 983 tokens (0 reused from cache) in 1 slice(s) of at most 2048

Ticket

ADFA-5187

Observation

The fallback mechanism is completely safe and mirrors the previous behavior (defaults to 4096) on any parse failure. The RAM snapshot excludes the currently loaded context to ensure the budget accurately reflects available memory.

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[hal-eisen-adfa]

Three findings from a review of the dynamic n_ctx change. All three are about the memory accounting around the new context sizing rather than the sizing itself.

[hal-eisen-adfa]

Three follow-ups on the latest round. The seven earlier threads all check out in 5d181b3 and the new tests cover the cases they promised, so nothing here reopens those. Two of these are consequences of the dedup fix and the new variable n_ctx respectively; the third is a doc inconsistency the weight-subtraction fix left behind.