Skip to content

Pass label selector predicate to the Secret Watch and handler - #443

Merged
perdasilva merged 3 commits into
operator-framework:mainfrom
kurlov:akurlov/pass-labelselector-predicate-to-secret-watch
Jun 30, 2025
Merged

Pass label selector predicate to the Secret Watch and handler#443
perdasilva merged 3 commits into
operator-framework:mainfrom
kurlov:akurlov/pass-labelselector-predicate-to-secret-watch

Conversation

@kurlov

@kurlovkurlov commented Mar 27, 2025

Copy link
Copy Markdown
Contributor

What

Updated the setupWatches method to pass predicate if labelSelector is set to the reconciler.

Why

If run two reconcilers with the same GVK but different labelSelectors then both reconcilers reconcile each other RCs ignoring configured LabelSelectors. This could be considered as not expected behavior if running two or more reconcilers with the same GVK but different labelSelectors is covered scenario. This PR fixes this issue.

Example logs of two reconcilers with the same GVK but different label selector.
There is only one CR with label app=nginx.
Reconciler1 has app=nginx label selector
Reconciler2 has app=foo label selector

Reconciler1:


2025-03-27T02:58:33+01:00 INFO controllers.Helm Watching resource {"group": "demo.example.com", "version": "v1alpha1", "kind": "Nginx"}
2025-03-27T02:58:33+01:00 INFO setup starting manager with label selector {"selector": "app=nginx"}
2025-03-27T02:58:33+01:00 INFO Starting EventSource {"controller": "nginx-controller", "source": "kind source: *unstructured.Unstructured"}
2025-03-27T02:58:33+01:00 INFO Starting EventSource {"controller": "nginx-controller", "source": "kind source: *v1.Secret"}
2025-03-27T02:58:33+01:00 INFO Starting Controller {"controller": "nginx-controller"}
2025-03-27T02:58:33+01:00 INFO Starting workers {"controller": "nginx-controller", "worker count": 1}
2025-03-27T02:58:33+01:00 DEBUG controllers.Helm Reconciliation triggered {"nginx": {"name":"nginx-sample","namespace":"default"}}
...

Reconciler2:

2025-03-27T03:00:39+01:00 INFO controllers.Helm Watching resource {"group": "demo.example.com", "version": "v1alpha1", "kind": "Nginx"}
2025-03-27T03:00:39+01:00 INFO setup starting manager with label selector {"selector": "app=foo"}
2025-03-27T03:00:39+01:00 INFO Starting EventSource {"controller": "nginx-controller", "source": "kind source: *unstructured.Unstructured"}
2025-03-27T03:00:39+01:00 INFO Starting EventSource {"controller": "nginx-controller", "source": "kind source: *v1.Secret"}
2025-03-27T03:00:39+01:00 INFO Starting Controller {"controller": "nginx-controller"}
2025-03-27T03:00:39+01:00 INFO Starting workers {"controller": "nginx-controller", "worker count": 1}
2025-03-27T03:00:39+01:00 DEBUG controllers.Helm Reconciliation triggered {"nginx": {"name":"nginx-sample","namespace":"default"}}

Manual testing

This small setup was used for running locally two reconcilers:

Running locally with the fix shows that only one reconciler reconcile the CR and another

Reconciler1:

2025-03-27T03:18:53+01:00 INFO controllers.Helm Watching resource {"group": "demo.example.com", "version": "v1alpha1", "kind": "Nginx"}
2025-03-27T03:18:53+01:00 INFO setup starting manager with label selector {"selector": "app=nginx"}
2025-03-27T03:18:53+01:00 INFO Starting EventSource {"controller": "nginx-controller", "source": "kind source: *unstructured.Unstructured"}
2025-03-27T03:18:53+01:00 INFO Starting EventSource {"controller": "nginx-controller", "source": "kind source: *v1.Secret"}
2025-03-27T03:18:53+01:00 INFO Starting Controller {"controller": "nginx-controller"}
2025-03-27T03:18:53+01:00 INFO Starting workers {"controller": "nginx-controller", "worker count": 1}
2025-03-27T03:18:53+01:00 DEBUG controllers.Helm Reconciliation triggered {"nginx": {"name":"nginx-sample","namespace":"default"}}

Reconciler2:

2025-03-27T03:18:37+01:00 INFO controllers.Helm Watching resource {"group": "demo.example.com", "version": "v1alpha1", "kind": "Nginx"}
2025-03-27T03:18:37+01:00 INFO setup starting manager with label selector {"selector": "app=foo"}
2025-03-27T03:18:37+01:00 INFO Starting EventSource {"controller": "nginx-controller", "source": "kind source: *unstructured.Unstructured"}
2025-03-27T03:18:37+01:00 INFO Starting EventSource {"controller": "nginx-controller", "source": "kind source: *v1.Secret"}
2025-03-27T03:18:37+01:00 INFO Starting Controller {"controller": "nginx-controller"}
2025-03-27T03:18:37+01:00 INFO Starting workers {"controller": "nginx-controller", "worker count": 1}

Comment threadpkg/reconciler/reconciler.go Outdated
var preds []predicate.Predicate
if r.selectorPredicate != nil {
preds = append(preds, r.selectorPredicate)
var secretPreds []predicate.TypedPredicate[*corev1.Secret]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get that you are harmonizing with the existing code, but do we need to use a slice at all here (for either preds or secretPreds)?

@perdasilvaperdasilvaMay 6, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, got it. It's to handle the nil case...

Comment threadpkg/reconciler/reconciler.go Outdated
obj,
handler.OnlyControllerOwner(),
),
secretPreds...,

@perdasilvaperdasilvaMay 6, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we get away with fewer changes if we used client.Object in the watch, like e.g.:

if err := c.Watch(
source.Kind(
mgr.GetCache(),
client.Object(secret),
handler.TypedEnqueueRequestForOwner[client.Object](
mgr.GetScheme(),
mgr.GetRESTMapper(),
obj,
handler.OnlyControllerOwner(),
),
preds...,
),
); err != nil {
return err
}

then we don't need to duplicate the predicate slice

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, this is better

})
})
})
When("label selector set", func() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<3

@perdasilvaperdasilva left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - just added a small suggestion that could obviate the need to refactor the option because of the requirement for a TypedPredicate. I don't know if there could be side-effects though.

@kurlov
kurlov requested a review from perdasilvaMay 22, 2025 15:03
@perdasilva
perdasilva enabled auto-merge June 13, 2025 11:20
auto-merge was automatically disabled June 16, 2025 09:10

Head branch was pushed to by a user without write access

@kurlov
kurlov requested a review from perdasilvaJune 23, 2025 09:32
@codecov-commenter

codecov-commenter commented Jun 30, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 45.45455% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.49%. Comparing base (08ab7fb) to head (c3f3347).
⚠️ Report is 139 commits behind head on main.

Files with missing linesPatch %Lines
pkg/reconciler/reconciler.go45.45%5 Missing and 1 partial ⚠️

❗ There is a different number of reports uploaded between BASE (08ab7fb) and HEAD (c3f3347). Click for more details.

HEAD has 1 upload less than BASE
FlagBASE (08ab7fb)HEAD (c3f3347)
21
Additional details and impacted files
@@ Coverage Diff @@## main #443 +/- ##
==========================================
- Coverage 85.06% 78.49% -6.58% 
==========================================
Files 19 31 +12 Lines 1346 2520 +1174 ==========================================
+ Hits 1145 1978 +833 - Misses 125 452 +327 - Partials 76 90 +14 

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@perdasilva
perdasilva added this pull request to the merge queueJun 30, 2025
Merged via the queue into operator-framework:main with commit 71298f1Jun 30, 2025
if r.selectorPredicate != nil {
preds = append(preds, r.selectorPredicate)

if r.labelSelector.MatchLabels != nil {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, so AFAICT WithSelector accepts a selector with match expressions (and no match labels), but then this check causes these expressions to be ignored? 🤔
This would be an unwelcome surprise for the user, no?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good catch!
I think it depends on how operator handles selectors. If operator uses built-in ParseToLabelSelector from k8s like in stackrox then this line will not be ignored and behaviour should be correct. But if some operator which uses this plugin creates selector manually with providing MatchExpressions only then there will be still a bug for multiple version for the same operator.

I also tested fir PR locally and my setup had such selector generation and it worked as expected:

...
selector := "app=" + appLabelValue
labelSelector, err := metav1.ParseToLabelSelector(selector)
...

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it depends on what you pass to ParseToLabelSelector. If you use any other operator than = or == then AFAICT MatchLabels will be empty and MatchExpressions will not.
I think the best way would be to change the if to do the emptiness check differently...

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@kurlov@codecov-commenter@porridge@perdasilva