Uh oh!
There was an error while loading. Please reload this page.
Add KubernetesPodOperatorAsync - #28523
Conversation
93cd391 to
d6e9eacCompareKubernetesPodOperatorAsyncUh oh!
There was an error while loading. Please reload this page.
af23ad3 to
5c88103CompareUh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
dimberman
left a comment
There was a problem hiding this comment.
Overall this looks good to me. My only major question is whether we need to asynchronously load the configuration. otherwise great job!
There was a problem hiding this comment.
Is this function really one that requires async? I imagine loading a kube config would be pretty quick?
There was a problem hiding this comment.
you're referring to async def _load_config(self) right?
well it does do file operations which are technically blocking (even though they might be fast)
so you might as well handle them asyncly ... just cus... why not?
but now that you mention it one thing that is a bit curious is ... how are we able to await await config.load_kube_config becaus it look like that is not an async method... i must be missing something
There was a problem hiding this comment.
yup, @phanikumv this config should come from from kubernetes_asyncio import config instead of from kubernetes import config like it is doing right now.
Please check https://github.com/astronomer/astronomer-providers/blob/main/astronomer/providers/cncf/kubernetes/hooks/kubernetes.py
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
May be place more descriptive comment here to describe what does actually in_cluster parameter mean?
There was a problem hiding this comment.
you're referring to async def _load_config(self) right?
well it does do file operations which are technically blocking (even though they might be fast)
so you might as well handle them asyncly ... just cus... why not?
but now that you mention it one thing that is a bit curious is ... how are we able to await await config.load_kube_config becaus it look like that is not an async method... i must be missing something
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
kaxil
commented
Dec 21, 2022
#28230 was created first, lets get that one merged and than rebase this on top of it to add more features |
bbd2254 to
30f12a2CompareUh oh!
There was an error while loading. Please reload this page.
36d7187 to
04e81e2Comparephanikumv
commented
Feb 1, 2023
I am working to rebase it and add more features |
373cb18 to
7cf6740Compare7cf6740 to
5ac8b9dComparephanikumv
commented
Feb 7, 2023
Closing as the additional feature on periodic fetching of logs has been implemented in #27758 |
This PR donates the
KubernetesPodOperatorAsyncfrom astronomer-providers repo to Airflow.Astronomer and its customers have been running this in production since last 6 months
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.