Pi auto-rewrite must preserve shell pipeline and redirection semantics

[thehoff]

Summary

The Pi bash rewrite integration can rewrite commands inside shell pipelines. That changes downstream command semantics, because the pipeline consumer receives ContextCrawler-filtered output instead of the original command output.

For agent-visible display commands this saves tokens. For data pipelines it can produce wrong answers.

Confirmed runtime example

From the repo root on develop:

# Raw shell, bypassing ContextCrawler rewrite
contextcrawler proxy sh -lc 'grep -R "fn " src/analytics | wc -l'
# => 95

# Normal Pi bash command with auto-rewrite active
grep -R "fn " src/analytics | wc -l
# => 85

contextcrawler gain --history then records the grep command as filtered, confirming the pipeline producer was rewritten.

Why this is dangerous

The user asked for a count of raw grep matches. The rewritten pipeline counts filtered ContextCrawler output instead.

The same class applies to other shell-data patterns:

ls -la path | head -5
cat file | grep pattern | head -1
cargo check > build.log

The rewrite engine currently emits rewrites for at least some redirected commands too:

contextcrawler rewrite 'cargo check >/tmp/cc-cargo-check.txt'
# contextcrawler cargo check >/tmp/cc-cargo-check.txt

contextcrawler rewrite 'ls -la src/analytics > /tmp/cc-pi-ls.txt'
# contextcrawler ls -la src/analytics > /tmp/cc-pi-ls.txt

Even if the current live Pi runtime appears to skip some redirection cases, the rewrite engine output is unsafe if loaded verbatim by the extension.

Expected behaviour

Auto-rewrite should preserve shell semantics.

A safe default would be:

• Rewrite only top-level simple commands whose stdout goes directly to the agent/model.
• Do not rewrite commands whose stdout is piped into another command.
• Do not rewrite commands with stdout redirection to a file.
• Do not rewrite inside command substitution, process substitution, heredocs, or nested shell strings unless explicitly designed and tested.

Users can still opt in manually:

contextcrawler grep -R "fn " src/analytics | wc -l

But auto mode should not silently change answers.

Acceptance criteria

• grep -R "fn " src/analytics | wc -l returns the same result with Pi auto-rewrite enabled as it does via contextcrawler proxy sh -lc.
• contextcrawler rewrite 'grep -R "fn " src/analytics | wc -l' does not rewrite the grep producer by default.
• contextcrawler rewrite 'cargo check >/tmp/build.log' does not rewrite the command by default.
• Tests cover pipelines, redirections, command substitution, heredocs, and direct display commands.
• Direct display commands like git status, ls, and cargo check still rewrite normally when stdout goes to the model.

Notes

This was found while testing the new ContextCrawler Pi integration. Direct command chains with && appear to rewrite as expected; nested shells such as sh -lc 'git status' appear to bypass rewrite, which is safer than rewriting blindly but should be documented.