Skip to content

✨ Use a separate namespace for each e2e test - #1428

Merged
tmshort merged 1 commit into
operator-framework:mainfrom
tmshort:fix-e2e-flakes
Nov 7, 2024
Merged

✨ Use a separate namespace for each e2e test#1428
tmshort merged 1 commit into
operator-framework:mainfrom
tmshort:fix-e2e-flakes

Conversation

@tmshort

@tmshorttmshort commented Nov 6, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Fixes#1307

Create and use a new namespace for every e2e test. This means that extension resources are placed in their own namespace. The tests deletes the namespace, and then waits until completion. This ensures that most of an extension's resources are deleted.

It does guarantee that global resources (e.g. CRDs, CRs, CRBs) are deleted.

And would eventually allow the tests to be run in parallel (assuming the installed extensions allow for that).

Description

Reviewer Checklist

  • API Go Documentation
  • Tests: Unit Tests (and E2E Tests, if appropriate)
  • Comprehensive Commit Messages
  • Links to related GitHub Issue(s)

@tmshort
tmshort requested a review from a team as a code ownerNovember 6, 2024 18:02
@netlify

netlifyBot commented Nov 6, 2024•
edited
Loading

Copy link
Copy Markdown

✅ Deploy Preview for olmv1 ready!

NameLink
🔨 Latest commitc64e360
🔍 Latest deploy loghttps://app.netlify.com/sites/olmv1/deploys/672cef8276e9e800071036e4
😎 Deploy Previewhttps://deploy-preview-1428--olmv1.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@codecov

codecovBot commented Nov 6, 2024•
edited
Loading

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.26%. Comparing base (26904a2) to head (c64e360).
Report is 1 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #1428 +/- ##
=======================================
Coverage 73.26% 73.26% =======================================
Files 42 42 Lines 3063 3063 =======================================
Hits 2244 2244 Misses 644 644 Partials 175 175
FlagCoverage Δ
e2e54.91% <ø> (-0.10%)⬇️
unit52.59% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tmshort

tmshort commented Nov 6, 2024

Copy link
Copy Markdown
ContributorAuthor

I have encountered one failure over the course of several dozen; so this is not perfect, but it does reduce the likelyhood

@tmshort
tmshortforce-pushed the fix-e2e-flakes branch 3 times, most recently from d940463 to 7b79aceCompareNovember 6, 2024 20:55
@tmshort

tmshort commented Nov 6, 2024•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

Of course, it failed e2e... but it failed because the bundle's CRDs were still present... not much we can do here except chase a timeout, which is never a good idea.

This was the interesting condition on the CRDs:

{Terminating True 2024-11-06 21:00:31 +0000 UTC InstanceDeletionFailed could not list instances: storage is (re)initializing}

@tmshort
Use a separate namespace for each e2e test
c64e360
Fixesoperator-framework#1307
Create and use a new namespace for every e2e test. This means that
extension resources are placed in their own namespace. The tests
deletes the namespace, and then waits until completion. This ensures
that _most_ of an extension's resources are deleted.
It does guarantee that global resources (e.g. CRDs, CRs, CRBs) are
deleted.
And would eventually allow the tests to be run in parallel (assuming
the installed extensions allow for that).
Signed-off-by: Todd Short <tshort@redhat.com>
LalatenduMohanty
LalatenduMohanty approved these changes Nov 7, 2024
View reviewed changes

@LalatenduMohantyLalatenduMohanty left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ciopenshift-ciBot added the lgtm Indicates that a PR is ready to be merged. label Nov 7, 2024
@tmshort
tmshort added this pull request to the merge queue Nov 7, 2024
github-merge-queueBot pushed a commit that referenced this pull request Nov 7, 2024
@tmshort
Use a separate namespace for each e2e test (#1428)
10ab78d
Fixes#1307
Create and use a new namespace for every e2e test. This means that
extension resources are placed in their own namespace. The tests
deletes the namespace, and then waits until completion. This ensures
that _most_ of an extension's resources are deleted.
It does guarantee that global resources (e.g. CRDs, CRs, CRBs) are
deleted.
And would eventually allow the tests to be run in parallel (assuming
the installed extensions allow for that).
Signed-off-by: Todd Short <tshort@redhat.com>
@tmshort

tmshort commented Nov 7, 2024

Copy link
Copy Markdown
ContributorAuthor

Managed to get this to run 35 times in a row without a failure...

@tmshorttmshort mentioned this pull request Nov 7, 2024
Closed
4 tasks
Merged via the queue into operator-framework:main with commit 8bccc74Nov 7, 2024
@tmshort
tmshort deleted the fix-e2e-flakes branch November 7, 2024 18:26
@tmshorttmshort mentioned this pull request Nov 7, 2024
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@LalatenduMohantyLalatenduMohantyLalatenduMohanty approved these changes

Assignees

@LalatenduMohantyLalatenduMohanty

Labels

lgtmIndicates that a PR is ready to be merged.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

e2e error: already exists in namespace '' and cannot be managed by operator-controller

2 participants

@tmshort@LalatenduMohanty