Flaky device test: InstallAndRunTests.DotNetRun ("Activity should have started", CoreCLR, ~6 builds/4 days)

[simonrozsival]

Summary

InstallAndRunTests.DotNetRun is a recurring device-test flake on the public dotnet-android pipeline (dnceng-public def 333). In the window 2026‑06‑27 → 06‑30 it failed in 6 distinct builds across 5 different PRs/branches (including a main build), and 2 of those builds still ended succeeded — i.e. the test recovered on a later retry slice — confirming flakiness rather than a hard regression. Every hit is in an (Auto‑Retry) run (failed in the original run and again on retry).

It belongs to the same "Activity should have started" emulator launch/timing family as #11810 (DebuggingTest.ApplicationRunsWithoutDebugger).

Test: tests/MSBuildDeviceIntegration/Tests/InstallAndRunTests.cs:77
Assert site: InstallAndRunTests.cs:110

Representative failure (build 1485678, DotNetRun(True,"managed",CoreCLR)):

Activity should have started.
 Expected: True
 But was: False
 at Xamarin.Android.Build.Tests.InstallAndRunTests.DotNetRun(Boolean isRelease, String typemapImplementation, AndroidRuntime runtime) in .../tests/MSBuildDeviceIntegration/Tests/InstallAndRunTests.cs:line 110

Occurrences (dnceng-public def 333, 2026‑06‑27 → 06‑30)

Build	PR / branch	Date	Config (isRelease, typemap, runtime)	Lane	Symptom	Build result
1484332	main	06‑27	(True,"trimmable",CoreCLR)	macOS‑8 (Auto‑Retry)	dotnet run --no-build should succeed	failed
1484332	main	06‑27	(True,"managed",CoreCLR)	macOS‑6 (Auto‑Retry)	Activity should have started	failed
1484554	#11617	06‑28	(True,"managed",CoreCLR)	macOS‑3 (Auto‑Retry)	Activity should have started	failed
1485678	#11772	06‑29	(True,"managed",CoreCLR)	macOS‑3 (Auto‑Retry)	Activity should have started	failed
1485683	#11617	06‑29	(True,"managed",CoreCLR)	macOS‑3 (Auto‑Retry)	Activity should have started	failed
1485920	#11786	06‑29	(True,"llvm-ir",CoreCLR)	macOS‑1 (Auto‑Retry)	Activity should have started	succeeded
1486425	#11804	06‑30	(False,"llvm-ir",CoreCLR)	macOS‑10 (Auto‑Retry)	dotnet build should succeed	succeeded

Patterns

• Dominant symptom is an activity launch timeout — Activity should have started. Expected: True But was: False at InstallAndRunTests.cs:110: the app built+installed but the activity didn't appear within the wait budget. Two builds instead failed at an earlier dotnet run/dotnet build step (a flaky MSBuild/deploy failure — worth a separate look at the captured build.log).
• Not config-specific. Spans isRelease true/false and all CoreCLR typemap flavors (managed, trimmable, llvm-ir). All hits are CoreCLR (the NativeAOT parametrization is effectively not the runnable one here).
• Not machine-specific. Spread across emulator lanes macOS‑1/3/6/8/10 — systemic timing/environment flake, not a bad node.
• Recovers on retry — 2 of 6 builds ended succeeded.

Related lower-frequency flakes (same suite, same "didn't start in time" family)

These appear once each in the window (both in build 1485678, in the same macOS‑3 (Auto‑Retry) run where DotNetRun died — i.e. one bad emulator took several tests down together). Tracked here for context, not yet independently significant:

• InstallAndRunTests.ActivityAliasRuns(True,NativeAOT) (InstallAndRunTests.cs:678) — Activity MainActivityAlias should have started. Expected: True But was: False.
• InstallAndRunTests.DotNetInstallAndRunPreviewAPILevels(True,"net11.0-android37.0",True) (InstallAndRunTests.cs:1825) — Activity should have started. Expected: True But was: False.

Promote either to its own issue if frequency rises.

Suggested actions

1. Capture logcat.log (+ a screenshot) on the activity-start timeout and/or raise the WaitForActivityToStart budget; investigate emulator activity-launch latency.
2. For the two dotnet run/dotnet build step failures, pull the referenced build.log — likely a separate, lower-frequency root cause.

Related

• Flaky device test: DebuggingTest.ApplicationRunsWithoutDebugger ('Activity should have started' + flaky build failures, CoreCLR, ~12 hits/5 days) #11810 — DebuggingTest.ApplicationRunsWithoutDebugger, same "Activity should have started" family.
• Flaky test analysis: Xamarin.Android-PR pipeline, April 27 – May 11 2026 (follow-up to #11203) #11320 / [tests] test failures on dnceng-public #10832 — broader flaky-test analysis / dnceng-public test-failure trackers (cover DotNetRunWaitForExit, DotNetRunWithDeviceParameter, not the base DotNetRun).

[jonathanpeppers]

Closing because this flake has not reproduced recently. Azure DevOps test-result queries found no InstallAndRunTests.DotNetRun failures in the latest 100 completed dotnet-android builds (August 3–12, 2026) or the latest 22 completed main builds (July 16–August 12, 2026). Those queries include failed tests published from otherwise-green continueOnError lanes.

The activity-start timeout infrastructure has also been standardized at 150 seconds since #11811. If this recurs, please reopen or file a focused issue with the current build ID, parameterization, logcat, and captured build log.