Skip to content

feat: runner-aware tools - #346

Closed
wan9chi wants to merge 1 commit into
mainfrom
runner-aware-tools
Closed

feat: runner-aware tools#346
wan9chi wants to merge 1 commit into
mainfrom
runner-aware-tools

Conversation

@wan9chi

@wan9chiwan9chi commented Apr 18, 2026

Copy link
Copy Markdown
Member

Set up a IPC channel between vite-task and the processes it spawns, so the spawned tools can declare at runtime what they actually read, wrote, or cared about, and then vite-task uses that to decide what to fingerprint in the cache.

Design notes: docs/runner-task-ipc/.

Problems this PR solves

Every example below is exercised by patches/vite.patch, which wires vite build into the IPC through @voidzero-dev/vite-task-client.

1. Dynamic tracked envs

Before: the user had to declare every relevant env in vite-task.json, statically:

{
"tasks": {
"build": { "env": ["NODE_ENV", "VITE_*"], "cache": true }
}
}

This duplicates knowledge the tool already has. Forgetting NODE_ENV silently skips cache invalidation on mode change. envPrefix-matching envs (VITE_* by default) get inlined into the bundle through import.meta.env.* — so changing envPrefix: 'MYAPP_' in vite.config.js without updating vite-task.json drifts: the runner still tracks VITE_* while the build output is driven by MYAPP_*.

After: the tool declares its envs at runtime, driven by its own config.

// vite's resolveConfigfetchEnv("NODE_ENV",{tracked: true});// vite's loadEnv, one call per configured prefix — these envs are// exposed to client code as import.meta.env.*, so their values are// baked into the bundlefor(constprefixofenvPrefix){fetchEnvs(`${prefix}*`,{tracked: true});}

The build task in vite-task.json needs no env: at all. Changing envPrefix in vite.config.js dynamically changes the set of envs the runner tracks, with zero config edits on the runner side.

2. Exclude tool's cache dir from input/output

Vite stores pre-bundled deps under node_modules/.vite/ and bundled configs under node_modules/.vite-temp/. Every build reads the cache metadata (to check staleness) and writes fresh entries when it isn't stale. Without intervention the runner sees:

  • the reads → implicit inputs, so the cache key depends on dep-cache contents
  • the writes → implicit outputs
  • the same directory both read and written → the runner refuses to cache the run at all (read-write overlap)

There is a workaround already in vite-plus: voidzero-dev/vite-plus#1096 plus its follow-up #1198 hardcode !node_modules/.vite-temp/**, !node_modules/.vite/**/results.json, and !dist/** as negative input globs on every vp subcommand (build, test, pack). That's not good enough:

  • Leaks vite internals into vp. Every time vite changes its cache layout (new path under .vite/, moved temp dir, new subcommand with its own transient files), vp has to ship a matching glob update. It's a lockstep coupling that design-wise shouldn't exist.
  • Input-only, not symmetric. The globs suppress reads for the input fingerprint (which is enough to break the read-write overlap check), but the writes are still captured as outputs — meaning transient cache contents get archived into the runner's cache and restored on every hit, bloating the cache store.
  • Per-subcommand, per-tool duplication. #1198 already had to retrofit the same glob into three subcommands. Any new subcommand, and any third-party tool with similar behavior (Nuxt's .nuxt/, SvelteKit's .svelte-kit/, Next's .next/), needs its own hand-maintained list — vp can't ship it generically.

After:

// in loadCachedDepOptimizationMetadataconstdepsCacheDir=getDepsCacheDir(environment);ignoreInput(depsCacheDir);ignoreOutput(depsCacheDir);

The declaration lives with the tool that owns the directory. The dep cache is vite's private concern.

3. Exclude output from input when a tool clears the folder before writing it

vite build calls emptyDir(outDir) before writing dist/. emptyDir has to read the directory entries to know what to delete — those reads look identical to genuine input reads. Since dist/ is also where vite writes its final output, the runner sees a read-write overlap on the same paths and refuses to cache.

After:

// in prepareOutDir, right before emptyDir()ignoreInput(outDir);

Only the writes count. The pattern generalizes: any tool that wipes-then-writes the same directory needs to tell the runner "my enumeration reads aren't inputs."

What's in this PR

  • Step 1 — Protocol (vite_task_ipc_shared): message types + serialization shared by both ends.
  • Step 2 — Transport (vite_task_server + vite_task_client): async server, sync blocking client, tested Rust-to-Rust.
  • Step 3 — Extract artifact crate out of fspy for dylib embedding. (Landed on main via refactor: extract materialized_artifact crate out of fspy #344 as materialized_artifact.)
  • Step 4 — JS bridge: vite_task_client_napi + @voidzero-dev/vite-task-client JS wrapper (fetchEnv single-name + fetchEnvs glob, with dedupe against already-set process.env).
  • Step 5 — Runner integration: server started per task execution, client dylib embedded/extracted, IPC envs injected via serve()'s returned iterator.
  • Step 6 — Cache integration: runner consumes reported ignored inputs/outputs, tracked env requests (single + glob), and disable-cache signals when fingerprinting.

Test plan

  • Rust integration tests for server/client transport (vite_task_server/tests/integration.rs)
  • E2E snapshot fixtures per client method: ignore_input, ignore_output, fetch_env, fetch_envs_glob, disable_cache
  • E2E test caching a real vite build via patches/vite.patch (vite_build_cache fixture): NODE_ENV-change invalidation, envPrefix-driven tracked-env set change, dist/ write restoration on cache hit

@wan9chi
wan9chiforce-pushed the runner-aware-tools branch from 4ba6a19 to 64f1651CompareApril 20, 2026 02:17
@wan9chi
wan9chi changed the base branch from main to graphite-base/346April 20, 2026 02:19
@wan9chi
wan9chi changed the base branch from graphite-base/346 to feat/output-restorationApril 20, 2026 02:19
@wan9chiGraphite App

wan9chi commented Apr 20, 2026

Copy link
Copy Markdown
MemberAuthor

This stack of pull requests is managed by Graphite. Learn more about stacking.

@wan9chiwan9chi changed the title feat(ipc): runner-aware tools — protocol + transport (partial)feat: runner-aware toolsApr 20, 2026
@wan9chi
wan9chiforce-pushed the runner-aware-tools branch from 64f1651 to f6605e3CompareApril 20, 2026 04:20
@wan9chi
wan9chiforce-pushed the feat/output-restoration branch from 0008bd7 to 994624aCompareApril 20, 2026 04:20
Comment threadpackages/vite-task-client/index.js Outdated
@socket-security

socket-securityBot commented Apr 23, 2026

Copy link
Copy Markdown

@wan9chi
wan9chiforce-pushed the runner-aware-tools branch 2 times, most recently from e18c21a to 09a310fCompareApril 23, 2026 07:42
@wan9chi
wan9chiforce-pushed the feat/output-restoration branch 2 times, most recently from 589a626 to 9df3054CompareApril 23, 2026 08:00
@wan9chi
wan9chiforce-pushed the runner-aware-tools branch from 09a310f to 87d8c32CompareApril 23, 2026 08:00
@wan9chi
wan9chiforce-pushed the feat/output-restoration branch from 9df3054 to 3a8b605CompareMay 7, 2026 08:08
@wan9chi
wan9chiforce-pushed the runner-aware-tools branch from 4edcd12 to cfa4282CompareMay 7, 2026 08:08
@wan9chi
wan9chiforce-pushed the feat/output-restoration branch from 3a8b605 to 6198f6bCompareMay 7, 2026 08:23
@wan9chi
wan9chiforce-pushed the runner-aware-tools branch 3 times, most recently from 24f5889 to ffca388CompareMay 7, 2026 08:38
@wan9chi
wan9chi changed the base branch from feat/output-restoration to graphite-base/346May 14, 2026 10:52
@wan9chi
wan9chiforce-pushed the graphite-base/346 branch from 6198f6b to c63db22CompareMay 14, 2026 10:52
@wan9chi
wan9chiforce-pushed the runner-aware-tools branch from ffca388 to 53d7bd1CompareMay 14, 2026 10:52
@wan9chi
wan9chi changed the base branch from graphite-base/346 to mainMay 14, 2026 10:52
@wan9chi

Copy link
Copy Markdown
MemberAuthor

@codex review

@chatgpt-codex-connectorchatgpt-codex-connectorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:53d7bd1883

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment threadpackages/vite-task-client/index.js Outdated
Comment threadcrates/vite_task/src/session/execute/mod.rs Outdated
@wan9chi
wan9chiforce-pushed the runner-aware-tools branch 15 times, most recently from e97de58 to 1a01a8aCompareMay 25, 2026 09:27
Set up an IPC channel between vite-task and the processes it spawns,
so the spawned tools can declare at runtime what they actually read,
wrote, or cared about, and vite-task uses that to decide what to
fingerprint in the cache.
Squashes the PR's commit history onto current `origin/main` after
the previous merge history (multiple merges with main as it evolved)
made a true commit-by-commit rebase impractical (heavy conflicts on
older commits like `30a97a2e` against main's parallel evolution of
the same fixtures).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@wan9chi
wan9chiforce-pushed the runner-aware-tools branch from 1a01a8a to a733f7bCompareMay 28, 2026 06:08
@wan9chi

Copy link
Copy Markdown
MemberAuthor

Superseded by the 5-PR stack: #404 (protocol+transport) → #405 (JS bridge) → #406 (execute refactor with placeholder server) → #407 (real integration + cache + first-party e2e) → #408 (real-vite e2e + playground).

@wan9chiwan9chi closed this May 28, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@wan9chi@cpojer