ADFA-5188 | Enable KV cache quantization and flash attention - #76

Open

jatezzz wants to merge 4 commits into

fix/ADFA-5187-dynamic-n-ctxfrom

fix/ADFA-5188-kv-cache-quantization

Commits on Aug 26, 2026

feat(ai-agent-local): store the KV cache as q8_0 to double the context
jatezzz
committed
c45cce1

Commits on Aug 27, 2026

merge: bring ADFA-5187 weights accounting into the q8_0 KV cache
jatezzz
committed
24b31ac
fix(ai-agent-local): retry a failed context at the 4096 floor
jatezzz
committed
f570d09
merge: bring ADFA-5187 single-parse header into the q8_0 KV cache
jatezzz
committed
5d1f91c