Skip to content

.NET: Fix off-thread RunStatus race where GetStatusAsync can return Running after ResumeAsync halts - #5412

Merged
Peter Ibekwe (peibekwe) merged 3 commits into
mainfrom
peibekwe/workflow-test-fix
Apr 23, 2026
Merged

.NET: Fix off-thread RunStatus race where GetStatusAsync can return Running after ResumeAsync halts#5412
Peter Ibekwe (peibekwe) merged 3 commits into
mainfrom
peibekwe/workflow-test-fix

Conversation

@peibekwe

Copy link
Copy Markdown
Contributor

Motivation and Context

In StreamingRunEventStream.RunLoopAsync, after every _inputWaiter.WaitForInputAsync wake-up, _runStatus was being flipped to Running unconditionally. This can lead stale signal to flip status back to running after consumer already observed halt signal.

Fixes#5411

Description

Only flip RunStatus to Running when there are messages to process.
LockstepRunEventStream is not affected — it runs supersteps synchronously on the consumer thread and has no comparable wake-up race.

This addresses one of the intermittent workflow test failures in the pipeline.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

CopilotAI review requested due to automatic review settings April 21, 2026 21:23
@moonbox3Evan Mattson (moonbox3) added .NET Usage: [Issues, PRs], Target: .Net workflows Usage: [Issues, PRs], Target: Workflows labels Apr 21, 2026
@github-actionsgithub-actionsBot changed the title Fix off-thread RunStatus race where GetStatusAsync can return Running after ResumeAsync halts.NET: Fix off-thread RunStatus race where GetStatusAsync can return Running after ResumeAsync haltsApr 21, 2026

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a race in the off-thread workflow run loop where RunStatus could transiently flip back to Running after a halt had already been observed, and adds a stress regression test to prevent reintroduction.

Changes:

  • Update StreamingRunEventStream.RunLoopAsync to set RunStatus.Running only when there are unprocessed messages to run.
  • Add an off-thread stress regression test that repeatedly runs the Step9 sample to catch the previously intermittent status race.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

FileDescription
dotnet/src/Microsoft.Agents.AI.Workflows/Execution/StreamingRunEventStream.csPrevents stale input signals from transiently setting RunStatus back to Running when there is no work to process.
dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/SampleSmokeTest.csAdds a stress regression test targeting the Step9 multi-response resume scenario in off-thread execution.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@peibekwe
Peter Ibekwe (peibekwe) added this pull request to the merge queueApr 22, 2026
@github-merge-queue
github-merge-queueBot removed this pull request from the merge queue due to failed status checks Apr 22, 2026
@peibekwe
Peter Ibekwe (peibekwe) added this pull request to the merge queueApr 22, 2026
@github-merge-queue
github-merge-queueBot removed this pull request from the merge queue due to failed status checks Apr 22, 2026
@peibekwe
Peter Ibekwe (peibekwe) added this pull request to the merge queueApr 23, 2026
@lokitoth

Copy link
Copy Markdown
Contributor

Ah, guess I missed it, but for information purposes:

I remember looking at this earlier in #2646; when I added tests for this, I noticed that Lockstep was semantically different in how it handled things, so there were some things to be done to get it to a happy state, but may be out of scope of this PR.

Might be worth bringing over the tests though?

I'll bring over tests + any changes that are needed for them separately.

Merged via the queue into main with commit 69adf6dApr 23, 2026
21 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

.NETUsage: [Issues, PRs], Target: .NetworkflowsUsage: [Issues, PRs], Target: Workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.NET: [Bug]: Workflows - Fix off-thread RunStatus race where GetStatusAsync can return Running after ResumeAsync halts

6 participants

@peibekwe@lokitoth@SergeyMenshykh@westey-m@moonbox3