fix: honest savings claims for read/git log filters (READ-I1, GIT-I1) - #134
Merged
Merged
Conversation
E2E audit (docs/audits/2026-05-22-e2e-review.md) found two filters whose
documented savings did not match measured reality. Diagnose-first pass:
neither needs a redesign — the headlines were oversold.
GIT-I1 — `git log` filter delivers ~2% over real runs vs an 80% claim.
Root cause: 99% of real invocations pass `--oneline`/`--format`, where
git has already compacted the output and `filter_log_output` is a
near-identity transform. The filter only earns savings on plain
`git log` (~1% of usage). Fixes:
- `log_run_is_passthrough` (pure, unit-tested) classifies a run: when
the filter produced no structural change, run_log records it via
`track_passthrough` (0/0 tokens) instead of `track`, so `gain` stops
blaming the filter for work it never had a chance to do.
- Claim revised in the init.rs template and hooks/README.md.
Empirically validated against a throwaway tracking DB:
git log -5 (plain) → 690→319 tokens, 53.8% tracked
git log --oneline -5 → 0/0 passthrough
git log -1 --format=%H → 0/0 passthrough
git log --format='%h %s' → 0/0 passthrough
Plus 6 unit tests covering the passthrough/tracked decision boundary.
READ-I1 — `read` filter delivers ~14% vs a 60% claim. Root cause: the
hook rewrites `cat` → `contextcrawler read` with no `--level`, which
defaults to passthrough. Forcing `--level minimal` would recover tokens
but strip code comments from everything the model reads — a real loss of
context value. Deliberately NOT done. Full passthrough by default is the
correct behaviour for an agent reading real code. Claim revised only:
"Code reading with filtering (60%)" → "Full file content; --level
minimal/aggressive to filter (opt-in)".
Also drop the non-English README translations (es/fr/ja/ko/zh) — stale
upstream-derived artifacts carrying the same oversold figures; this fork
maintains only README.md. Nothing references them.
2,539 tests pass (+6 new), clippy clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
noogalabs
pushed a commit
to noogalabs/contextcrawler
that referenced
this pull request
Jun 4, 2026
) * feat: tee raw output to file for LLM re-read without re-run (thehoff#86) When RTK filters command output, LLM agents lose failure details (stack traces, assertions) and re-run the same command 2-3x. The tee feature saves raw output to ~/.local/share/rtk/tee/ on failure and prints a one-line hint so the agent can read the file instead. - Add src/tee.rs: core module with tee_raw(), tee_and_hint(), rotation - Add TeeConfig to config.rs: enabled, mode, max_files, max_file_size - Integrate in 7 modules: cargo, runner, vitest, pytest, lint, tsc, go - Default: failures only, skip <500 chars, 20 file rotation, 1MB cap - Env overrides: RTK_TEE=0 (disable), RTK_TEE_DIR (custom directory) - 14 unit tests, 352 total tests passing, zero regressions Closes thehoff#86 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add tee.rs to ARCHITECTURE.md module count (47 modules) CI validate-docs requires main.rs module count == ARCHITECTURE.md count. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves audit findings READ-I1 and GIT-I1 from docs/audits/2026-05-22-e2e-review.md. Diagnose-first pass concluded neither filter needs a redesign — the headline savings were oversold.
GIT-I1 —
git logmeasured at ~2% vs the 80% claim. Root cause: 99% of real invocations pass--oneline/--format=…, where git already compacted the output andfilter_log_outputis a near-identity transform. The filter only earns savings on plaingit log(~1% of usage).log_run_is_passthrough()(unit-tested) classifies each run. No-op runs nowtrack_passthrough(0/0) instead oftrack, sogainstops blaming the filter for work it never had a chance to do.src/hooks/init.rstemplate andhooks/README.md.git log -5(plain) → 53.8% tracked;--oneline -5/--format=%H→ 0/0 passthrough.READ-I1 —
readmeasured at ~14% vs the 60% claim. Root cause: the hook rewritescat→contextcrawler readwith no--level, which defaults to passthrough. Forcing--level minimalwould recover tokens but strip code comments from everything the model reads — a real loss of context value. Deliberately NOT done. Full passthrough by default is correct behaviour for an agent reading real code. Claim revised only.Also drops the non-English README translations (es/fr/ja/ko/zh) — stale upstream-derived artifacts carrying the same oversold figures.
Tests: +6 unit tests covering the passthrough/tracked decision boundary, 2,539 pass, 0 fail, clippy clean.
🤖 Generated with Claude Code