Inline step execution optimization on Vercel doesn't make use of full function timeout

[VaguelySerious]

After #1338, we inline-execute steps for higher performance, but the flow still suspends every ~2 minutes, even if I set my function timeouts to 13 min (800s).

This should be configurable or set to a better heuristic

[shtefcs]

Hey, just reporting from our setup since we're running into this same 120s boundary daily.

We're on the latest beta tarball (the one you said would become 5.0.0-beta.5) for a multi-agent chat product. Coding and browser agent sessions routinely run 5-15 minutes, and we're seeing the worker cycle every 2 minutes on the dot. Cross-invocation streaming itself works correctly via WorkflowChatTransport reconnecting on each cycle, so functionally we're OK, but every cycle is wasted overhead and an extra failure point for the client.

We saw WORKFLOW_V2_TIMEOUT_MS in the runtime source. Two questions before we set it:

1. Is bumping it to something like 700000 (just under our Vercel function maxDuration=800) safe to do today, or are there gotchas you've seen? Our function has 800s and is in the iad1 region same as the workflow backend.

2. Is there an ETA for the auto-derive heuristic? If it's days away we'd rather wait than carry an env var. If it's weeks, we'll set the env var now.

For context on what "works fine across cycles" looks like: one of our coding sessions yesterday spanned 4 worker invocations over 8 minutes, 53 messages persisted to DB, UI never froze. So the architecture is solid for our workload, it's just the cycle frequency that's higher than necessary for chats with a long maxDuration.

Happy to share a runId if useful.

[VaguelySerious]

@shtefcs You can override WORKFLOW_V2_TIMEOUT_MS if you want. The general concerns are as follows:

If your invocation runs for any time timeElapsed <= WORKFLOW_V2_TIMEOUT_MS and then starts to execute a step with expectedStepDuration >= maxFunctionDuration - timeElapsed, then your step will always time out and needs to be re-run in a fresh invocation.

This is to say, if your max function duration is 15 minutes, and you have a step which runs 10 minutes on average somewhere in the flow, then setting WORKFLOW_V2_TIMEOUT_MS to >= 5 minutes will likely lead to performance degradation, since the long-running step will get interrupted on average, but setting it to e.g. ~3 minutes will have a slight performance benefit. 2 minutes was chosen as a heuristic for the average use case. Feel free to play with the number.

[shtefcs]

Sharing some real-world data since we just hit this hard.

We run durable coding agents where a single run does 40 to 50 steps. Our function maxDuration is 800s. With WORKFLOW_V2_TIMEOUT_MS unset (so the 120s default), one build cycled through 17 separate /flow invocations on a roughly 2 minute heartbeat. We measured about 53 minutes of billed function time for around 48 steps of actual work, and the per-step latency itself was healthy (about 13s). So roughly 93 percent of the wall clock went to cross-invocation dispatch and idle, not real work.

Setting WORKFLOW_V2_TIMEOUT_MS closer to our function budget fixes most of it. We landed on 430000 using the rule of thumb from this thread (maxDuration minus the longest single step, which for us is a ~363s doStreamStep). That keeps a safe buffer while cutting the re-invocation count a lot.

Two things that would help:

1. Derive the default from the function maxDuration instead of a fixed 120s. For anyone running functions longer than 2 minutes, which is common for AI agent workloads on Fluid compute, the 120s default is silently close to the worst case. Even deriving it as maxDuration minus a conservative step estimate would remove this footgun for most people.

2. The current tradeoff is awkward. Raise it and long steps risk always timing out, lower it and you pay constant re-invocation overhead. An option to let a single in-progress step extend the invocation past the threshold, instead of forcing a re-run, would let us set the value aggressively without the timeout risk.

Is the auto-derive heuristic still on the roadmap, or is the current recommendation to just set this manually per deployment?