"Never send a human to do a machine's job." Agent Smith, The Matrix (1999)
Agent configuration is hard, fragile, and always changing. Agent Smith outsources that tuning work to your agent: it gathers empirical data from watching real sessions and recommends concrete changes to prompts, settings, and workflow.
Get a feedback loop based on how Claude Code, Codex, Gemini CLI, OpenCode, Pi, or Grok actually behave on your real work instead of guessing what might help.
Agent Smith:
- Collects hook and plugin telemetry from supported hosts
- Pulls the latest configuration schemas
- Emits events to a user-level SQLite database
- Generates agent-backed analysis reports by agent or project
- Helps apply safe config improvements and surfaces larger changes for review
- Bun
>=1.3.0 jqsqlite3python3
claude plugins marketplace add chetmancini/agent-smith
claude plugins install agent-smith@agent-smithInstall from a local clone:
bun run ./agent-smith-app/src/cli.ts install-codexOr:
make codex-installThen:
- Restart Codex.
- Open the Plugin Directory.
- Choose your personal marketplace.
- Install
Agent Smith. - Run
make app-doctor.
The installer enables features.codex_hooks = true, links the plugin into ~/.codex/plugins/agent-smith, and configures the personal marketplace entry. The checkout still needs to stay trusted so Codex can load the repo-local .codex/hooks.json.
Gemini currently ships as a local-checkout hook extension:
gemini extensions link ./gemini-extensionThis reuses the shared repo-root shell hooks and scripts.
Add Agent Smith to opencode.json:
{
"$schema": "https://opencode.ai/config.json",
"plugin": ["agent-smith-opencode"]
}For a local checkout instead of the published package:
{
"plugin": ["./path/to/agent-smith/opencode-plugin"]
}Install from a local clone:
bun run ./agent-smith-app/src/cli.ts install-piOr:
make pi-installThen:
- Start a new Pi session.
- Run
make app-doctor.
The installer adds this checkout to ~/.pi/agent/settings.json so Pi can discover Agent Smith from any working directory. The installed package exposes:
.pi/extensions/agent-smith/index.tsfor telemetry and slash-command aliasescommands/as prompt templatesskills/as Agent Skills
Grok uses a plugin + project .grok/ files for hooks (telemetry) and skills (analysis commands).
Install from a local clone:
bun run ./agent-smith-app/src/cli.ts install-grokOr:
make grok-installThen:
- Start (or restart) Grok sessions.
- If hooks from this checkout are encountered for the first time, approve the trust prompt (or run
/hooks-trust/ use the hooks modal). - Run
make app-doctor.
The installer runs grok plugin install ./grok-plugin --trust (or prints the equivalent if the grok binary is not on PATH). This registers:
- Hooks under the plugin for
grok-scoped session/tool events (reusing the shared shell emitters). - Bundled skills for
/analyze-config,/upgrade-settings, etc. (also available via project.grok/skills/when your working directory is inside this checkout).
The repo also provides .grok/hooks/hooks.json and .grok/skills/ directly for pure project-scoped use (no plugin registration required when cwd is the agent-smith tree).
| Surface | Claude Code | Gemini CLI | Codex | OpenCode | Pi | Grok |
|---|---|---|---|---|---|---|
| Install path in this repo | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
agent-smith doctor coverage | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Slash commands | ✓ | ✓ | ✓ | ✓ | (skills) | |
| Shared shell commands | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Feature | Claude Code | Gemini CLI | Codex | OpenCode | Pi | Grok |
|---|---|---|---|---|---|---|
| Session lifecycle | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Bash failure tracking | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Vague prompt guidance | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Rollup and analysis | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Schema validation | ✓ | ✓ | ✓ | ✓ | ✓ | (stub) |
| Tool failures | ✓ | ✓ | ✓ | ✓ | ✓ | |
| Permission denials | ✓ | ✓ | ✓ | (via PreToolUse) | ||
| Permission grants | ✓ | |||||
| Session errors | ✓ | ✓ (Stop) | ||||
| File-edited telemetry | ✓ | ✓ (PostToolUse) | ||||
| Context compression | ✓ | ✓ | ✓ | ✓ | ✓ (PreCompact) | |
| Edit-triggered test-loop detection | ✓ | ✓ | ✓ | ✓ | ✓ | |
| Subagent lifecycle | ✓ | ✓ | ✓ | ✓ |
As of April 22, 2026, Codex still exposes a narrower hook surface than Claude Code. Gemini is supported for install, doctor, and the shared shell workflow through the hook extension, but slash-command parity is still pending. OpenCode reaches its richer telemetry surface through the native TypeScript plugin. Pi support installs this checkout as a Pi package, which exposes the repo extension, bundled schema validation, and slash-command aliases across working directories; Pi currently records context compression without the Claude/Gemini auto-vs-manual trigger split because that reason is not exposed by Pi's extension events. Grok uses native .grok/hooks/ (or plugin hooks) with direct support for SessionStart/End, Pre/PostToolUse*, UserPromptSubmit, Stop, PreCompact, Subagent*; project hooks require one-time trust. Grok config is TOML and schema validation uses a permissive stub until an official schema is published.
Claude Code, Codex, OpenCode, and Pi currently expose these slash commands:
/agent-smith:analyze
/agent-smith:analyze-fast
/agent-smith:upgrade-settings
Gemini currently ships the hook extension plus the shared shell commands below. Slash-command parity can come later. Pi provides the same command names through the installed package's prompt-template aliases. Grok discovers the skills under .grok/skills/ (or via plugin) and exposes them as /analyze-config, /analyze-fast, /upgrade-settings (or full /agent-smith:analyze via skill name). The hooks provide the telemetry surface.
Useful commands:
bash scripts/metrics-rollup.sh
bash scripts/analyze-config.sh --sessions 50
bash scripts/analyze-config.sh --llm --sessions 50
bash scripts/refresh-schemas.sh
bash scripts/validate-agent-config.sh --tool codex --refresh
make agent-upgrade-settings TOOL=codex
make app-doctorThe standalone TypeScript CLI is the migration path for the shared runtime:
bun run ./agent-smith-app/src/cli.ts doctor
bun run ./agent-smith-app/src/cli.ts report
bun run ./agent-smith-app/src/cli.ts improve --tool codex
bun run ./agent-smith-app/src/cli.ts loop --tool codex
bun run ./agent-smith-app/src/cli.ts install-grok ┌───────────┐
│ COLLECT │◄────────────────────┐
│ hooks and │ │
│ plugins │ │
└─────┬─────┘ │
│ │
▼ │
┌───────────┐ │
│ ROLL UP │ │
│ JSONL to │ │
│ SQLite │ │
└─────┬─────┘ │
│ │
▼ │
┌───────────┐ │
│ ANALYZE │ │
│ raw or │ │
│ agent-led │ │
└─────┬─────┘ │
│ │
▼ │
┌───────────┐ │
│ APPLY │ │
│ safe fixes│ │
│ and review│ │
└─────┬─────┘ │
│ │
▼ │
┌───────────┐ │
│ BETTER │─────────────────────┘
│ AGENT │
│ LOOP │
└───────────┘
- Collect: hooks or plugins emit structured events into
~/.config/agent-smith/events.jsonl. - Roll up: the event stream is ingested into
~/.config/agent-smith/rollup.db. - Analyze: Agent Smith generates a raw local report or an agent-backed report in
~/.config/agent-smith/reports/. - Apply: safe changes can be applied automatically; larger configuration or workflow changes stay reviewable.
The default CLI path produces a local raw report. The slash command /agent-smith:analyze defaults to the smarter agent-backed path. Automatic analysis is disabled by default.
When --include-settings is enabled for agent-backed analysis, Agent Smith redacts obvious secret-bearing keys before sending the settings snapshot to the active agent.
| Event | Typical trigger |
|---|---|
session_start | Session begins |
session_stop | Turn or session ends |
session_error | OpenCode session crashes |
tool_failure | Tool execution fails |
command_failure | Shell command exits non-zero |
permission_denied | Permission request is denied |
permission_granted | OpenCode permission is granted |
file_edited | OpenCode edit telemetry fires |
clarifying_question | Prompt is vague or ambiguous |
test_failure_loop | Repeated test failures after edits |
context_compression | Host compacts or compresses context |
Not every host exposes every event. Metrics are tagged by initiating agent, and analysis stays scoped per agent.
When the host provides structured shell-failure payloads, Agent Smith records command text, exit code, output snippets, and turn or tool identifiers so the raw event stream stays actionable.
Token usage and estimated USD cost are calculated during rollup, not during the hook itself. Session start stores the transcript path, and rollup re-reads transcripts so repeated runs can capture partial progress from an in-flight session.
~/.config/agent-smith/
├── events.jsonl # Raw metric events
├── rollup.db # SQLite rollup database
├── reports/ # Generated analysis reports
├── .session_start_ts_* # Temporary timing files
├── .cost_snapshot_* # Temporary cost snapshots
├── .transcript_paths # Transcript lookup state
└── .test_fail_count_* # Consecutive test failure counters
| Path | Purpose |
|---|---|
agent-smith-app/ | Standalone TypeScript CLI and shared runtime |
.claude-plugin/ | Claude Code manifest |
.codex-plugin/ | Codex manifest |
gemini-extension/ | Gemini CLI extension |
opencode-plugin/ | Native OpenCode plugin |
.pi/extensions/agent-smith/ | Pi extension entrypoint packaged from this repo |
.codex/hooks.json | Repo-local Codex hook registration |
schemas/ | Bundled schema snapshots such as Pi settings |
hooks/ | Shared shell hook scripts and libraries |
scripts/ | Rollup, analysis, schema, and helper scripts |
commands/ | Slash-command prompts |
skills/ | Agent Smith skills |
tests/ | Bats and integration tests |
bunfor the standalone app, the OpenCode plugin, and TypeScript checksbatsfor the shell test suitesjqfor JSON validation and release helpersshellcheckfor shell lintingshfmtfor shell formatting checksmarkdownlintfor README, command, and skill docsghfor GitHub release creation
Run this before pushing or updating a PR:
make pre-pushTo install the tracked git hook for this clone:
make install-git-hooksTargeted local checks:
make test
make lint
make format-check
make typecheckAgent Smith now uses VERSION as the single release source of truth.
# one-command release: run tests, bump, commit, tag, push, and publish notes
make release VERSION=1.0.1If you edit VERSION by hand, run make sync-version to push that value into the Claude, Gemini, and Codex manifests plus agent-smith-app/package.json and opencode-plugin/package.json.
make release requires a clean git worktree, a freshly fetched local main that exactly matches origin/main, and an authenticated gh session. Before it mutates version files or creates tags, it runs make release-test (make deps + make test) and does not reuse the broader make pre-push hook path. Run it from main after git pull --ff-only origin main. If you only want to bump versioned release files without publishing yet, use make set-version VERSION=1.0.1.
The GitHub release flow does not publish the standalone CLI package to npm for you. After cutting the repo release, publish it separately from agent-smith-app/ when you are ready:
cd agent-smith-app
npm publish# Run skills through any agent
make agent-analyze TOOL=claude
make agent-analyze TOOL=gemini
make agent-validate-schemas TOOL=codex
make agent-upgrade-settings TOOL=codex
make agent-loop TOOL=opencode # validate-schemas then analyze-config
make agent-analyze TOOL=grok
make agent-loop TOOL=grok
# Override session window
make agent-analyze TOOL=codex SESSIONS=100
make grok-install
# Local schema tools
make refresh-schemas # Refresh all schema caches
make refresh-schemas TOOL=codex # Refresh one schema cache
make validate-agent-config TOOL=codex
# Full repo validation before push
make pre-push
make install-git-hooksThe Makefile keeps a single parameterized interface: use TOOL=claude|gemini|codex|opencode|pi|grok for the agent-backed agent-*, refresh-schemas, and validate-agent-config targets.
MIT