Uh oh!
There was an error while loading. Please reload this page.
Add dev-tmux skill for portless+tmux local Workflow SDK dev - #1916
Conversation
Adds two race workflows (sleepWinsRaceWorkflow, stepWinsRaceWorkflow) that exercise Promise.race between a step function and a sleep call. The current `sleepWinsRaceWorkflow` test fails — surfacing how the replay engine resolves a previously-completed step instantly while sleep still has to elapse. Also adds a `dev-tmux` skill that documents the 3-pane tmux + portless setup for testing workflows interactively in a worktree alongside the observability UI. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 8b7225b The changes in this PR will be included in the next version bump. This PR includes changesets to release 0 packagesWhen changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) workflow with 10 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) workflow with 25 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) workflow with 50 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) workflow with 10 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) workflow with 25 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) workflow with 50 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro Stream Benchmarks(includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) stream pipeline with 5 transform steps (1MB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) 10 parallel streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) fan-out fan-in 10 streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
❌ Some benchmark jobs failed:
Check the workflow run for details. |
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
|
Adds allowedDevOrigins entries so portless-style worktree-prefixed .localhost URLs (e.g. https://<branch>.turbopack.localhost) can hit HMR and dev-only endpoints without Next's cross-origin protection flooding the logs with warnings. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds new end-to-end coverage for Promise.race behavior between a long-running step and sleep(), and introduces a new dev-tmux skill to standardize a local 3-pane dev setup (workbench + observability UI + scratchpad) using portless for worktree-isolated .localhost URLs.
Changes:
- Add
sleepWinsRaceWorkflow/stepWinsRaceWorkflowworkflows to the workbench e2e workflow collection. - Add corresponding Vitest e2e tests asserting the expected race winner and duration bounds.
- Add
skills/dev-tmux/SKILL.mddocumentation for a tmux + portless local dev workflow; add a (currently empty) changeset file.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| workbench/example/workflows/99_e2e.ts | Adds two new workflows plus a shared step helper to exercise Promise.race between a step and sleep(). |
| packages/core/e2e/e2e.test.ts | Adds two e2e tests covering the new race workflows and asserting winner + duration. |
| skills/dev-tmux/SKILL.md | New skill documenting a portless-routed 3-pane tmux setup for local SDK dev + e2e runs. |
| .changeset/better-pets-reply.md | Adds a new changeset file, but it is currently empty/invalid. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| test('sleepWinsRaceWorkflow', { timeout: 60_000 }, async () => { | ||
| const run = await start(await e2e('sleepWinsRaceWorkflow'), []); | ||
| const returnValue = await run.returnValue; | ||
| expect(returnValue.winner).toBe('sleep'); | ||
| // Sleep is 1s; step would take 10s. Should resolve in ~1s, well under 5s. | ||
| expect(returnValue.durationMs).toBeLessThan(5_000); | ||
| }); |
Uh oh!
There was an error while loading. Please reload this page.
| tmux send-keys -t "$SESSION".1 \ | ||
| 'cd workbench/nextjs-turbopack && WORKFLOW_PUBLIC_MANIFEST=1 portless run --name turbopack pnpm dev' C-m | ||
| # Pane 2 (top-right): observability UI scoped to the workbench app | ||
| tmux send-keys -t "$SESSION".2 \ | ||
| 'cd workbench/nextjs-turbopack && portless run --name workflow-obs sh -c "pnpm workflow web --webPort \$PORT --noBrowser"' C-m | ||
| # Pane 3 (bottom-right): scratchpad at repo root | ||
| tmux send-keys -t "$SESSION".3 'echo "scratchpad: $(pwd)"' C-m |
There was a problem hiding this comment.
Fixed in 5a44360. The skill now captures each pane's ID at split time with -P -F '#{pane_id}' and uses those IDs as targets, so it's correct under both pane-base-index 0 and pane-base-index 1 regardless of the user's tmux config.
Uh oh!
There was an error while loading. Please reload this page.
TooTallNate
left a comment
There was a problem hiding this comment.
Verdict: Request Changes
The PR has two things in it: e2e race tests (good idea, surfacing a real V2-handler semantics question) and a personal-dev skill file. I have one blocker on the test, one scope concern on the skill, and one minor nit on the next.config change.
Summary of inline comments
packages/core/e2e/e2e.test.ts:587— Blocker.sleepWinsRaceWorkflowfails on every e2e job (56+ jobs across all frameworks × all environments). Needstest.fails(...),.skipwith TODO, or removal until the underlying race semantics are decided. The PR description correctly identifies this as intentional pending discussion, but the current state makes the e2e matrix unusable for any PR based on this one.skills/dev-tmux/SKILL.md:3— Scope. Activation phrases ("spin up the dev session", "start dev mode", "set up the workbench") will trigger for general contributor questions, then push them toward installing portless. Either narrow the phrases to portless-specific terms or move the skill to a user-level location.workbench/nextjs-turbopack/next.config.ts:13— Nit. Hardcoding*.turbopack.localhostin the workbench bakes one developer's convention into the shared config. Dev-only, so harmless — flagging for awareness, not blocking.
What I verified
- Diagnosis is correct: The V2 inline-execution model in
workflowEntrypoint(packages/core/src/runtime.ts) executes step bodies inline rather than enqueueing them. A 10-seconddelayMsStepblocks the handler for 10 seconds, which prevents the 1-secondsleep('1s')timer from firing on time. SoPromise.raceends up resolving with'step'because that's what the handler delivers first. Whether this should be considered correct behavior or a bug is a real question. - Symlinks: Confirmed
99_e2e.tsis symlinked fromworkbench/example/workflows/99_e2e.tsinto all other workbenches per AGENTS.md convention. ✅ - Empty changeset:
---\n---is the right pattern for tests-only PRs that don't ship package changes; matches existing examples (fix-vite-step-source-maps-e2e.md,tanstack-start-workbench.md). ✅ - Skill frontmatter:
version: '1.0'is fine for a new skill (existingmigrating-to-workflow-sdkis'0.2.0'). Format matches. - Typecheck: Clean across all 40 packages.
Suggested path forward
Smallest change that keeps the value of this PR:
- Mark
sleepWinsRaceWorkflowwithtest.fails(...)and add a code comment linking to an issue tracking the V2 race-semantics decision. That preserves the test as a regression detector — if the underlying behavior is ever fixed, the test will go red and force a follow-up. - Optional: open a separate small skill PR after clarifying scope, or move the skill to a personal location.
- Land
stepWinsRaceWorkflowas a regular passing test — it's a useful regression check on the easy direction.
Happy to re-review once the failing assertion is addressed.
| test('sleepWinsRaceWorkflow', { timeout: 60_000 }, async () => { | ||
| const run = await start(await e2e('sleepWinsRaceWorkflow'), []); | ||
| const returnValue = await run.returnValue; | ||
| expect(returnValue.winner).toBe('sleep'); |
There was a problem hiding this comment.
Blocker — every e2e CI job is failing because of this assertion.
This test fails on every framework's e2e run (express, hono, nitro, fastify, astro, vite, nuxt, sveltekit, tanstack-start, both Next.js variants — local-dev, local-prod, local-postgres, AND vercel-prod) because 99_e2e.ts is shared across all workbenches via symlink. As of the latest run, that's 56+ failed jobs all on expected 'step' to be 'sleep'.
For example: E2E Local Dev Tests (express - stable) — FAIL packages/core/e2e/e2e.test.ts > e2e > sleepWinsRaceWorkflow / AssertionError: expected 'step' to be 'sleep'.
The PR description acknowledges this is intentional and wants discussion before merge. That's reasonable as a discussion vehicle, but it can't merge in this state — the PR effectively makes the entire e2e matrix red for any other PR that bases on this one.
Until the underlying behavior question is settled, please pick one:
test.fails(...)— vitest will pass when the assertion fails, and flip to red if/when the runtime is fixed. Cleanly documents the known-broken behavior.test.skip(...)with a TODO comment linking to the discussion / issue tracking the race semantics decision.- Drop the failing test from this PR and land just
stepWinsRaceWorkflow(which is the meaningful regression-prevention test); open a follow-up issue or PR for the slow-step-vs-fast-sleep case once the semantics are decided.
For reference, the underlying cause is consistent with the V2 inline-execution model in packages/core/src/runtime.ts (workflowEntrypoint): the handler executes the step inline rather than enqueuing it, so a 10s step blocks the handler for 10s and prevents the 1s sleep timer from firing on time. That's a real semantics question worth resolving — but in a separate change.
There was a problem hiding this comment.
Resolved — merged latest main, which carried PR #1924's fix for the inline-execution behavior and added these same race tests directly. The duplicate set landed by this branch was dropped (commit 031e4b8), so 99_e2e.ts now matches main exactly and the race tests are no longer part of this PR's diff. Both sleepWinsRaceWorkflow and stepWinsRaceWorkflow now run from main's definitions and pass. This PR is dev-tooling only after the rebase.
| @@ -0,0 +1,125 @@ | |||
| --- | |||
| name: dev-tmux | |||
| description: Spin up a 3-pane tmux session for local Workflow SDK development — Next.js turbopack workbench, observability UI, and a scratchpad — all routed through portless so each worktree gets isolated `.localhost` URLs. Use when the user asks to "spin up the dev session", "start dev mode", "set up the workbench", or any time they want to test workflows interactively in a worktree alongside the observability UI. | |||
There was a problem hiding this comment.
Activation phrases here are very generic — "spin up the dev session", "start dev mode", "set up the workbench" all describe the general task most contributors do daily, not the specific portless+tmux setup this skill describes. As written, this skill will activate for any contributor asking how to start the dev server, then drive them toward installing portless and adopting a personal tmux layout that the rest of the team doesn't use.
A couple of options:
- Narrow the activation phrases to portless-specific terms: "portless dev session", "tmux dev layout", "worktree-isolated dev URLs", etc. That way the skill only fires for users who already know they want this specific tooling.
- Move it out of the shared
skills/directory. This appears to be one developer's personal workflow tooling rather than a project-wide convention —skills/migrating-to-workflow-sdk/is documenting a user-facing capability, whereas this is documenting an internal contributor's local-dev preference. A user-level agent skill (e.g.~/.agents/skills/dev-tmux/) would keep it personal without affecting other contributors.
(The skill itself is well-written and the troubleshooting section is genuinely useful — this is purely about scope/activation, not quality.)
There was a problem hiding this comment.
Tightened in 5a44360 (v1.1 of the skill). The description now opens with Spin up a portless + tmux dev session ... and explicitly lists the only triggers as portless dev session / tmux dev layout for workflow / worktree-isolated dev URLs / wanting to wire URLs into the Claude statusline. It also explicitly says not to activate for the generic start the dev server / run pnpm dev task. Should now stay out of any contributor's path who isn't already opting into this tooling.
Kept it under skills/ rather than moving to ~/.claude/ because v1.1 also adds a statusline.sh helper script that needs to live next to the skill — moving it out would split the two artifacts. Open to revisiting if option 2 still feels right.
Uh oh!
There was an error while loading. Please reload this page.
Worktrees get deleted, so wiring the statusline to a worktree path breaks the moment the worktree is removed. Update the skill and the script header to recommend pointing `statusLine.command` at the primary checkout (`$HOME/github/vercel/workflow/...`). The script itself is already worktree-aware via Claude's `workspace.current_dir` stdin JSON, so the same invocation surfaces routes for whichever worktree the session is in. Bump version to 1.2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
| author: Vercel Inc. | ||
| version: '1.2' |
There was a problem hiding this comment.
change the author to "Pranay Prakash" and set the version to 0.1 for this first release (since it's a new skill anyway)
There was a problem hiding this comment.
Done in 8b7225b — metadata.author: Pranay Prakash, metadata.version: '0.1'.
| @@ -0,0 +1,160 @@ | |||
| --- | |||
| name: dev-tmux | |||
There was a problem hiding this comment.
| name: dev-tmux | |
| name: internal-dev-workbench |
There was a problem hiding this comment.
Applied in 8b7225b — git mv skills/dev-tmux skills/internal-dev-workbench, plus the matching frontmatter name: change and updates to all internal references in SKILL.md and statusline.sh.
pranaygp
commented
May 5, 2026
making some changes to the skill and testing it a bit more. moving to draft for now |
- Statusline overlay now renders `[dev] · [obs] · tmux:<prefix>`, with the bracketed labels emitted as OSC 8 hyperlinks (clickable in any modern terminal) styled cyan + underline so they stand out. Replaces the old long-URL form that was hard to scan and click. - Add a tmux-session indicator: shown when a session named exactly the worktree prefix exists (uses `tmux has-session -t =<prefix>` for exact matching). - Change the skill's tmux session naming convention from the fixed `workflow-dev` to `<worktree-prefix>` (basename of the branch — same string portless uses as the subdomain prefix). This lets the statusline locate the session deterministically and lets multiple worktrees run dev sessions concurrently without manual disambiguation. - Bump skill to v1.3. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the abbreviated `tmux:<prefix>` indicator with the full copy-paste-ready `tmux attach -t <prefix>` invocation. Saves a step when grabbing the session from another shell. Bump skill to v1.4. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drop the dim styling that made the overlay hard to read; use bold bright cyan + underline for links and bold bright green for the tmux command. - Add Nerd Font glyphs: for dev, for obs, for the tmux copy-paste hint. Falls back to box-drawing if the font lacks Nerd Font ranges; layout is unaffected. - Visual differentiation: cyan + underline = clickable hyperlink; green = copy this command. Bump skill to v1.5. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The copy glyph in `emit_tmux` was a literal Nerd Font byte embedded in the printf string and got stripped during a prior rewrite. Promote all three icons (rocket / graph / copy) to top-level shell variables that use \uHHHH-equivalent UTF-8 escapes, so the source survives editor round-trips that don't preserve Private Use Area code points. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
pranaygp
commented
May 5, 2026
Updates since the previous round of review replies — all in Statusline overhaul (commits c714162 → 938c26e)
Tmux session naming convention (commit 431026c)
Skill version bumped to |
…set version - Rename `skills/dev-tmux/` → `skills/internal-dev-workbench/` to make the name self-explanatory about the skill's scope (an internal contributor's local dev workbench, not a generic tmux helper). - Author: Pranay Prakash. Version: 0.1 (first release of the skill). - Update internal references in SKILL.md and statusline.sh accordingly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
dev-tmuxskill documenting the 3-pane tmux + portless setup for local Workflow SDK development (turbopack workbench, observability UI scoped to that workbench, scratchpad shell — all worktree-isolated via portless's branch-prefixed.localhostURLs).allowedDevOrigins: ['turbopack.localhost', '*.turbopack.localhost']toworkbench/nextjs-turbopack/next.config.tsso portless-style URLs don't get blocked by Next's cross-origin protection in dev (currently floods the logs with HMR warnings).What changed since the original PR
The original PR also added two race workflows (
sleepWinsRaceWorkflow,stepWinsRaceWorkflow) and their e2e tests to surface a Promise.race-vs-replay semantics issue. Those landed independently in #1924 (with the underlying fix), so this branch was rebased onto main and the duplicate additions removed — the race tests now come from main. This PR is dev-tooling only.Test plan
pnpm vitest run packages/core/e2e/e2e.test.ts -t WinsRaceWorkflowagainst the turbopack workbench (passes after rebase, since [core] V2: skip inline step execution when suspension also has a wait #1924's fix is on main)skills/dev-tmux/SKILL.mdand confirm bothhttps://<branch>.turbopack.localhost(HMR works, no cross-origin warnings) andhttps://<branch>.workflow-obs.localhost(observability UI shows runs from the workbench)🤖 Generated with Claude Code