Uh oh!
There was an error while loading. Please reload this page.
Fix named parameters templating in Databricks operators - #40864
Merged
pankajkoti merged 2 commits intoJul 18, 2024
Conversation
potiuk
commented
Jul 18, 2024
Member
Those failing tests should be DB tests. |
pankajkoti
commented
Jul 18, 2024
Member
I have created a PR #40869 for marking them as db tests. Hope it passes and then we can rebase it here once CI passes and we merge it after review. |
pankajkoti
commented
Jul 18, 2024
Member
@boraberke could you please rebase as the PR to fix mark the required tests as db tests is merged now. |
potiukforce-pushed
the
fix/databricks-templated-params
branch
from
July 18, 2024 18:19
9b4fe65 to
3c6c175Comparepotiuk
commented
Jul 18, 2024
Member
I rebased it (@pankajkoti -> the "drop-down" on "update branch" has rebase option - and in most cases maintainers have the permission to push to the branch of the fork of Airflow :). |
pankajkoti
commented
Jul 18, 2024
Member
Thank you @potiuk :) Yes, I was just unsure if authors feel comfortable/uncomfortable if I do a rebase and whether it's an accepted policy. Glad to know it can be done, thank you :) |
boraberke
commented
Jul 18, 2024
ContributorAuthor
Thank you for the feedback @potiuk and the PR @pankajkoti, I wasn't really sure what to do with the failing tests! |
potiuk
commented
Jul 18, 2024
Member
I usually rebase and write a comment about it. Worst thing that can happen, the author will have to push their changes with |
potiuk
approved these changes
Jul 18, 2024
potiuk
commented
Jul 18, 2024
Member
LGTM. @pankajkoti ? |
pankajkoti
approved these changes
Jul 18, 2024
Member
There was a problem hiding this comment.
LGTM too.
@josh-fell I hope the issue you reported previously still is resolved together with this and the earlier PR #40471 from @boraberke .
@vatsrahul1001 I guess would be good if we could test this.
I will go ahead with merging the PR. But in case, you find something not working, please report here and we can try to resolve it asap.
Contributor
Verified named parameters issue is resolved now, however, I noticed Issue still persists. @josh-fell can you verify as well? |
Member
@boraberke would you have time to confirm if #35433 has resurfaced after this PR? We might have to reopen that issue in that case. |
romsharon98 pushed a commit
to romsharon98/airflow
that referenced
this pull request
Jul 26, 2024
This PR fixes the many named parameters that was templated and was broken with apache#40471. The following operators are affected: DatabricksCreateJobsOperator DatabricksSubmitRunOperator DatabricksRunNowOperator closes: apache#40788
pankajkoti added a commit
that referenced
this pull request
Jul 27, 2024
* Revert "Fix named parameters templating in Databricks operators (#40864)" This reverts commit cfe1d53. * Revert "Make Databricks operators' json parameter compatible with XComs, Jinja expression values (#40471)" This reverts commit 4fb2140. This reverts PR #40864 and PR #40471. Previously, PR #40471 was contributed to address issue #35433. However, that contribution gave rise to another issue #40788. Next #40788 was being attempted to be resolved in PR #40864. However, with the second PR, it appears that the previous old issue #35433 has [resurfaced](#40864 (comment)). So, at the moment, the case is that we have 2 PRs on top of the existing implementation eventually having nil effect and the previous issues persists. I believe it is better to revert those 2 PRs, reopen the earlier issue #35433 and peacefully address it by taking the needed time.
kosteev pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this pull request
Nov 9, 2024
* Revert "Fix named parameters templating in Databricks operators (#40864)" This reverts commit cfe1d53ed041ea903292e3789e1a5238db5b5031. * Revert "Make Databricks operators' json parameter compatible with XComs, Jinja expression values (#40471)" This reverts commit 4fb2140f393b6332903fb833151c2ce8a9c66fe2. This reverts PR #40864 and PR #40471. Previously, PR apache/airflow#40471 was contributed to address issue apache/airflow#35433. However, that contribution gave rise to another issue apache/airflow#40788. Next apache/airflow#40788 was being attempted to be resolved in PR #40864. However, with the second PR, it appears that the previous old issue #35433 has [resurfaced](apache/airflow#40864 (comment)). So, at the moment, the case is that we have 2 PRs on top of the existing implementation eventually having nil effect and the previous issues persists. I believe it is better to revert those 2 PRs, reopen the earlier issue #35433 and peacefully address it by taking the needed time. GitOrigin-RevId: 4535e08b862e2b7ff4f2a76de7124983d4efe9db
kosteev pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this pull request
May 6, 2025
* Revert "Fix named parameters templating in Databricks operators (#40864)" This reverts commit cfe1d53ed041ea903292e3789e1a5238db5b5031. * Revert "Make Databricks operators' json parameter compatible with XComs, Jinja expression values (#40471)" This reverts commit 4fb2140f393b6332903fb833151c2ce8a9c66fe2. This reverts PR #40864 and PR #40471. Previously, PR apache/airflow#40471 was contributed to address issue apache/airflow#35433. However, that contribution gave rise to another issue apache/airflow#40788. Next apache/airflow#40788 was being attempted to be resolved in PR #40864. However, with the second PR, it appears that the previous old issue #35433 has [resurfaced](apache/airflow#40864 (comment)). So, at the moment, the case is that we have 2 PRs on top of the existing implementation eventually having nil effect and the previous issues persists. I believe it is better to revert those 2 PRs, reopen the earlier issue #35433 and peacefully address it by taking the needed time. GitOrigin-RevId: 4535e08b862e2b7ff4f2a76de7124983d4efe9db
kosteev pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this pull request
May 27, 2025
* Revert "Fix named parameters templating in Databricks operators (#40864)" This reverts commit cfe1d53ed041ea903292e3789e1a5238db5b5031. * Revert "Make Databricks operators' json parameter compatible with XComs, Jinja expression values (#40471)" This reverts commit 4fb2140f393b6332903fb833151c2ce8a9c66fe2. This reverts PR #40864 and PR #40471. Previously, PR apache/airflow#40471 was contributed to address issue apache/airflow#35433. However, that contribution gave rise to another issue apache/airflow#40788. Next apache/airflow#40788 was being attempted to be resolved in PR #40864. However, with the second PR, it appears that the previous old issue #35433 has [resurfaced](apache/airflow#40864 (comment)). So, at the moment, the case is that we have 2 PRs on top of the existing implementation eventually having nil effect and the previous issues persists. I believe it is better to revert those 2 PRs, reopen the earlier issue #35433 and peacefully address it by taking the needed time. GitOrigin-RevId: 4535e08b862e2b7ff4f2a76de7124983d4efe9db
kosteev pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this pull request
Sep 22, 2025
* Revert "Fix named parameters templating in Databricks operators (#40864)" This reverts commit cfe1d53ed041ea903292e3789e1a5238db5b5031. * Revert "Make Databricks operators' json parameter compatible with XComs, Jinja expression values (#40471)" This reverts commit 4fb2140f393b6332903fb833151c2ce8a9c66fe2. This reverts PR #40864 and PR #40471. Previously, PR apache/airflow#40471 was contributed to address issue apache/airflow#35433. However, that contribution gave rise to another issue apache/airflow#40788. Next apache/airflow#40788 was being attempted to be resolved in PR #40864. However, with the second PR, it appears that the previous old issue #35433 has [resurfaced](apache/airflow#40864 (comment)). So, at the moment, the case is that we have 2 PRs on top of the existing implementation eventually having nil effect and the previous issues persists. I believe it is better to revert those 2 PRs, reopen the earlier issue #35433 and peacefully address it by taking the needed time. GitOrigin-RevId: 4535e08b862e2b7ff4f2a76de7124983d4efe9db
kosteev pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this pull request
Oct 20, 2025
* Revert "Fix named parameters templating in Databricks operators (#40864)" This reverts commit cfe1d53ed041ea903292e3789e1a5238db5b5031. * Revert "Make Databricks operators' json parameter compatible with XComs, Jinja expression values (#40471)" This reverts commit 4fb2140f393b6332903fb833151c2ce8a9c66fe2. This reverts PR #40864 and PR #40471. Previously, PR apache/airflow#40471 was contributed to address issue apache/airflow#35433. However, that contribution gave rise to another issue apache/airflow#40788. Next apache/airflow#40788 was being attempted to be resolved in PR #40864. However, with the second PR, it appears that the previous old issue #35433 has [resurfaced](apache/airflow#40864 (comment)). So, at the moment, the case is that we have 2 PRs on top of the existing implementation eventually having nil effect and the previous issues persists. I believe it is better to revert those 2 PRs, reopen the earlier issue #35433 and peacefully address it by taking the needed time. GitOrigin-RevId: 4535e08b862e2b7ff4f2a76de7124983d4efe9db
kosteev pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this pull request
Feb 25, 2026
* Revert "Fix named parameters templating in Databricks operators (#40864)" This reverts commit cfe1d53ed041ea903292e3789e1a5238db5b5031. * Revert "Make Databricks operators' json parameter compatible with XComs, Jinja expression values (#40471)" This reverts commit 4fb2140f393b6332903fb833151c2ce8a9c66fe2. This reverts PR #40864 and PR #40471. Previously, PR apache/airflow#40471 was contributed to address issue apache/airflow#35433. However, that contribution gave rise to another issue apache/airflow#40788. Next apache/airflow#40788 was being attempted to be resolved in PR #40864. However, with the second PR, it appears that the previous old issue #35433 has [resurfaced](apache/airflow#40864 (comment)). So, at the moment, the case is that we have 2 PRs on top of the existing implementation eventually having nil effect and the previous issues persists. I believe it is better to revert those 2 PRs, reopen the earlier issue #35433 and peacefully address it by taking the needed time. GitOrigin-RevId: 4535e08b862e2b7ff4f2a76de7124983d4efe9db
kosteev pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this pull request
May 11, 2026
* Revert "Fix named parameters templating in Databricks operators (#40864)" This reverts commit cfe1d53ed041ea903292e3789e1a5238db5b5031. * Revert "Make Databricks operators' json parameter compatible with XComs, Jinja expression values (#40471)" This reverts commit 4fb2140f393b6332903fb833151c2ce8a9c66fe2. This reverts PR #40864 and PR #40471. Previously, PR apache/airflow#40471 was contributed to address issue apache/airflow#35433. However, that contribution gave rise to another issue apache/airflow#40788. Next apache/airflow#40788 was being attempted to be resolved in PR #40864. However, with the second PR, it appears that the previous old issue #35433 has [resurfaced](apache/airflow#40864 (comment)). So, at the moment, the case is that we have 2 PRs on top of the existing implementation eventually having nil effect and the previous issues persists. I believe it is better to revert those 2 PRs, reopen the earlier issue #35433 and peacefully address it by taking the needed time. GitOrigin-RevId: 4535e08b862e2b7ff4f2a76de7124983d4efe9db
kosteev pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this pull request
Jul 26, 2026
* Revert "Fix named parameters templating in Databricks operators (#40864)" This reverts commit cfe1d53ed041ea903292e3789e1a5238db5b5031. * Revert "Make Databricks operators' json parameter compatible with XComs, Jinja expression values (#40471)" This reverts commit 4fb2140f393b6332903fb833151c2ce8a9c66fe2. This reverts PR #40864 and PR #40471. Previously, PR apache/airflow#40471 was contributed to address issue apache/airflow#35433. However, that contribution gave rise to another issue apache/airflow#40788. Next apache/airflow#40788 was being attempted to be resolved in PR #40864. However, with the second PR, it appears that the previous old issue #35433 has [resurfaced](apache/airflow#40864 (comment)). So, at the moment, the case is that we have 2 PRs on top of the existing implementation eventually having nil effect and the previous issues persists. I believe it is better to revert those 2 PRs, reopen the earlier issue #35433 and peacefully address it by taking the needed time. GitOrigin-RevId: 4535e08b862e2b7ff4f2a76de7124983d4efe9db
kosteev pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this pull request
Jul 28, 2026
* Revert "Fix named parameters templating in Databricks operators (#40864)" This reverts commit cfe1d53ed041ea903292e3789e1a5238db5b5031. * Revert "Make Databricks operators' json parameter compatible with XComs, Jinja expression values (#40471)" This reverts commit 4fb2140f393b6332903fb833151c2ce8a9c66fe2. This reverts PR #40864 and PR #40471. Previously, PR apache/airflow#40471 was contributed to address issue apache/airflow#35433. However, that contribution gave rise to another issue apache/airflow#40788. Next apache/airflow#40788 was being attempted to be resolved in PR #40864. However, with the second PR, it appears that the previous old issue #35433 has [resurfaced](apache/airflow#40864 (comment)). So, at the moment, the case is that we have 2 PRs on top of the existing implementation eventually having nil effect and the previous issues persists. I believe it is better to revert those 2 PRs, reopen the earlier issue #35433 and peacefully address it by taking the needed time. GitOrigin-RevId: 4535e08b862e2b7ff4f2a76de7124983d4efe9db
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes the many named parameters that was templated and was broken with #40471.
The following operators are affected:
closes: #40788
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.