Background
contextcrawler is a shell-side filter: it compresses the output of commands the agent already ran. Some patterns — e.g. "count lines per .ts file in src/" — burn 10s of read calls + grep walks where a 20-line script would do the job once. contextcrawler doesn't currently offer a way to delegate that kind of imperative extraction to a sandboxed runtime.
Problem
Agent workflows that need lightweight aggregation across many files (counting, summarising, filtering) blow context through repeated tool calls when a single executable script would emit only the final result.
Proposed Design
New subcommand:
contextcrawler exec --lang python --intent "count lines per .ts file in src/"
The agent writes the script body, contextcrawler runs it in a subprocess with two contextcrawler-specific properties:
-
Tirith supply-chain gate INSIDE the sandbox — pip install evil inside the script body gets caught by the same machinery that catches it at the shell level. A sandbox with a verb gate is strictly more powerful than a sandbox alone.
-
Filter chain on script output — script stdout runs through contextcrawler's existing filters before returning. If the script emits JSON, the json filter compresses it. If it emits a file listing, the ls filter applies. The agent gets the script's output already-filtered.
Language runtimes for v1: Python + Node.js. Detect via which; gracefully no-op if missing.
Script body passed via stdin or --script-file.
Why now
- The two contextcrawler-specific properties (verb gate inside sandbox + filter chain on output) are genuine architectural advantages that depend on contextcrawler's existing supply-chain and filter infrastructure
- Opens the agent-side delegation lane contextcrawler currently doesn't address
- Optional / opt-in — doesn't change the existing CLI surface
Effort
M (1-2 weeks). Process spawn + isolation + stdout capture, supply-chain gate hookup inside argv, filter chain integration, language runtime detection, snapshot tests.
Dependencies
Out of scope (v1)
- 12 language runtimes (Ruby / Go / Rust / etc.) — defer
- Concurrent batch execution — defer
- Network sandboxing — out of scope for the lab-only profile; relies on Tirith's existing
curl | bash gate
Honest evaluation
This is the LEAST clear-cut feature on the roadmap. contextcrawler is positioned as a shell-side filter; exec moves into agent-side territory. Open question: are the verb-gate-inside-sandbox + filter-chain-on-output angles meaningful enough differentiators to justify entering that lane, or is the right call to stay best-in-class at shell-side?
Recommend deferring this until #157–#160 ship and the position is clearer.
Background
contextcrawler is a shell-side filter: it compresses the output of commands the agent already ran. Some patterns — e.g. "count lines per .ts file in src/" — burn 10s of
readcalls + grep walks where a 20-line script would do the job once. contextcrawler doesn't currently offer a way to delegate that kind of imperative extraction to a sandboxed runtime.Problem
Agent workflows that need lightweight aggregation across many files (counting, summarising, filtering) blow context through repeated tool calls when a single executable script would emit only the final result.
Proposed Design
New subcommand:
The agent writes the script body, contextcrawler runs it in a subprocess with two contextcrawler-specific properties:
Tirith supply-chain gate INSIDE the sandbox —
pip install evilinside the script body gets caught by the same machinery that catches it at the shell level. A sandbox with a verb gate is strictly more powerful than a sandbox alone.Filter chain on script output — script stdout runs through contextcrawler's existing filters before returning. If the script emits JSON, the
jsonfilter compresses it. If it emits a file listing, thelsfilter applies. The agent gets the script's output already-filtered.Language runtimes for v1: Python + Node.js. Detect via
which; gracefully no-op if missing.Script body passed via stdin or
--script-file.Why now
Effort
M (1-2 weeks). Process spawn + isolation + stdout capture, supply-chain gate hookup inside argv, filter chain integration, language runtime detection, snapshot tests.
Dependencies
Out of scope (v1)
curl | bashgateHonest evaluation
This is the LEAST clear-cut feature on the roadmap. contextcrawler is positioned as a shell-side filter;
execmoves into agent-side territory. Open question: are the verb-gate-inside-sandbox + filter-chain-on-output angles meaningful enough differentiators to justify entering that lane, or is the right call to stay best-in-class at shell-side?Recommend deferring this until #157–#160 ship and the position is clearer.