Skip to content

Harden shared CI against transient failures - #437

Open
sjmiller609 wants to merge 33 commits into
mainfrom
sjmiller609/test-flakes
Open

Harden shared CI against transient failures#437
sjmiller609 wants to merge 33 commits into
mainfrom
sjmiller609/test-flakes

Conversation

@sjmiller609

@sjmiller609sjmiller609 commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

summary

  • retry checkout and SDK lint validation after transient network or dependency-download failures
  • derive test bridge names from uniquely leased subnets to prevent cross-process bridge collisions without reducing parallelism
  • recover and restage the host-provisioned Windows image and backing fixtures into /ci on every test run

tests

  • go test ./lib/instances -run ^TestBridgeNameForTestSubnet$ -count=1
  • git diff --check

Note

Medium Risk
CI-only and test-infra changes: retries and fixture copies are low-impact, but renaming Linux test bridges affects isolation of concurrent network tests on shared runners.

Overview
Hardens shared CI against transient checkout/lint failures, missing Windows VM images, and colliding test network bridges.

Checkout in test.yml now retries once on failure. SDK lint in stlc-generate.yml retries up to three times. Each Linux test run restages host-provisioned Windows qcow2/raw fixtures into /ci/windows when they are present.

Test Linux bridges are named from the leased subnet (hm + octets) instead of a seed/seq combo, so parallel processes no longer share a bridge name.

Reviewed by Cursor Bugbot for commit f33dca0. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
-->

✱ stlc build

gocode · compare

Your SDK build resulted in a merge conflict between your custom code and the newly generated changes. Run stlc build --continue after resolving conflicts.

generate ⚡bootstrap ✅format ✅

116 files generated

pythoncode · compare

Your SDK build was successful.

generate ✅bootstrap ✅format ✅

230 files generated at 681b411

typescriptcode · compare

Your SDK build resulted in a merge conflict between your custom code and the newly generated changes. Run stlc build --continue after resolving conflicts.

generate ⚡bootstrap ✅format ✅

138 files generated

Diagnostics: ❗ 1 error, 💡 5 note
LevelCodeMessageTargets
❗ errorPython/RequestParamShadowsDefaultRenamed param `timeout` because it shadowed a request option.
(resource) instances > (method) wait > (params) default > (param) timeout > (schema)
python
💡 notePagination/NotConfiguredConfirm your API does not return paginated results.
#/pagination
python, go, typescript
💡 noteModel/RecommendedWe recommend you use a model for `#/components/schemas/InstanceGPU`
#/components/schemas/InstanceGPU
python, go, typescript
💡 noteSchema/EnumHasOneMemberConfirm intentional use of `enum` with single member.
(resource) health > (model) HealthCheckResponse > (schema) > (property) status
go
💡 noteSchema/EnumHasOneMemberConfirm intentional use of `enum` with single member.
(resource) instances > (model) restart_status > (schema) > (property) last_reason
go
💡 noteMethod/BodyRootParamUnnamedConfirm `body` is an appropriate name for this method's request parameters.
#/paths//volumes/from-archive/post/requestBody/content/application/gzip/schema
go, python, typescript
Build metadata
Buildbd_76BdRHNh-elfin-snag
Timestamp2026-08-21T22:24:42.352Z
stlc8413509
Spec hash0f15511e31a6
Config hash55e15f6f4434

This comment is auto-generated by stlc and is kept up to date as you push.
If you push new commits, re-run this workflow to update this comment.
Last updated: 2026-08-21 22:25:04 UTC

@sjmiller609
sjmiller609 marked this pull request as ready for review August 21, 2026 12:56
Comment on lines +461 to +472
func TestBridgeNameForTestSubnet(t *testing.T) {
t.Parallel()

first, err := testBridgeNameForSubnet("10.200.1.0/24")
if err != nil {
t.Fatal(err)
}
second, err := testBridgeNameForSubnet("10.200.2.0/24")
if err != nil {
t.Fatal(err)
}
if first != "hmc801" || second != "hmc802" || first == second {

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks like it's just trying two options but that would only decrease but not resolve the issue

bridgeName, err = testBridgeNameForSubnet(subnet)
if err != nil {
return err
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orphan bridges can block subnets

Medium Severity

Bridge names are now a fixed function of the leased subnet, but subnet selection only skips leases and overlapping routes. An orphan hm* bridge with no lease and no route still maps to the same name on the next lease of that subnet, so createBridge can fail when the interface already exists without the expected address. The old seed-based names avoided reusing that interface name.

Additional Locations (2)
Fix in CursorFix in Web

Reviewed by Cursor Bugbot for commit 7e3924e. Configure here.

Comment thread.github/workflows/test.yml
Comment thread.github/workflows/test.yml Outdated
Comment thread.github/workflows/test.yml Outdated
Comment thread.github/workflows/test.yml Outdated

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6258e21. Configure here.

Comment thread.github/workflows/test.yml Outdated
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@sjmiller609