AIP-84 Migrate patch a connection to FastAPI API by bugraoz93 · Pull Request #43102 · apache/airflow

bugraoz93 · 2024-10-17T00:08:17Z

closes: #42592.
Include, making fields nullable for patch endpoints for backward compatibility.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

pierrejeambrun

I think we should also make consistant the behavior for PATCH regarding None value.

Here when there is a mask you take them. And when there is no mask, you filter them out.

I did something different for Pool which I think can be extented. When there is a mask, same thing, we use values. When there is no mask, we validate that the payload is valide by it's own, i.e it's a valid db entity by its own, and we use everything. (none or not).

The idea is to allow someone to not specify mask and reset some fields to None if needed.

pierrejeambrun · 2024-10-28T11:27:16Z

I just pushed one commit to remove a remnent of old code _parse_comma_separated_query_params. PR rebased. Should be good to merge.

pierrejeambrun · 2024-10-28T13:48:48Z

I think we have the same issue here, we cannot update the password field. This should be possible, and the returned password should be redacted I believe. Otherwise we have no way for a user to update the password of a connection, or set it (event the POST create connection does not allow it in its current state.)

bugraoz93 · 2024-10-28T19:39:00Z

I just pushed one commit to remove a remnent of old code _parse_comma_separated_query_params. PR rebased. Should be good to merge.

I totally missed this one, thanks for the changes! :)

bugraoz93 · 2024-10-28T19:43:14Z

I think we should also make consistant the behavior for PATCH regarding None value.
Here when there is a mask you take them. And when there is no mask, you filter them out.
I did something different for Pool which I think can be extented. When there is a mask, same thing, we use values. When there is no mask, we validate that the payload is valide by it's own, i.e it's a valid db entity by its own, and we use everything. (none or not).
The idea is to allow someone to not specify mask and reset some fields to None if needed.

I think I looked from the lazy path when no parameter, no value to you :) Saw your changes. Looks good and makes sense! I agree, we should have a common approach for common methods because what we are doing at the end is pretty similar.

pierrejeambrun · 2024-10-29T08:39:03Z

For 'common' behavior of null values we can do that in another PR. That is not really specific to this one. I think we can just handle the 'password', then merge it, maybe you can take care of making all PATCH consistent regarding none values in a follow up, if that sounds good ?

bugraoz93 · 2024-10-29T16:16:49Z

For 'common' behavior of null values we can do that in another PR. That is not really specific to this one. I think we can just handle the 'password', then merge it, maybe you can take care of making all PATCH consistent regarding none values in a follow up, if that sounds good ?

Yeap, it sounds perfect! I am going to take care of that after finishing the password part. There is a small work left for password. I couldn't make redact work in the tests, which I will check it out today and push it to both PRs related to this

pierrejeambrun

We need to handle the password

bugraoz93 · 2024-11-04T20:17:04Z

Tests should be fixed in #43654.

pierrejeambrun

Looking good.

Need rebasing to fix conflicts, we can merge after.

…ameter Field default value for backward compatibility

…on for backward compat and fix for variables serialisation won't update val in setattr()

…date_mask breaking change

bugraoz93 · 2024-11-05T19:13:35Z

Looking good.
Need rebasing to fix conflicts, we can merge after.

Many thanks for rebasing and merging the other PR! You saved me from facing the conflict twice 🙏 Rebased.

* Migrate Patch Connection endpoint to FastAPI and Include Optional parameter Field default value for backward compatibility * Make all tests parametrized * Make UpdateMask Annotated for Patch endpoints, update the serialisation for backward compat and fix for variables serialisation won't update val in setattr() * Merge duplicate imports * Revert update_mask to explode true and include significant.rst for update_mask breaking change * Amend update_mask as array * generalize handling of list query parameters in newsfragment * Include password field * Include password to response and tests, rebase and rerun pre-commit * Fix copy&paste mistakes for method and naming * Convert redact_password to agreed approach, rebase and pre-commits * Include new exception doc generation * Run pre-commit after rebase

bugraoz93 requested review from bbovenzi, ephraimbuddy, jscheffl, pierrejeambrun and ryanahamilton as code owners October 17, 2024 00:08

boring-cyborgBot added area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. labels Oct 17, 2024

bugraoz93 commented Oct 17, 2024
View reviewed changes

Comment threadairflow/api_fastapi/core_api/serializers/variables.py Outdated

pierrejeambrun reviewed Oct 17, 2024
View reviewed changes

Comment threadairflow/api_fastapi/core_api/routes/public/connections.py Outdated
Comment threadairflow/api_fastapi/core_api/serializers/connections.py Outdated
Comment threadairflow/api_fastapi/core_api/serializers/variables.py Outdated

rawwar added the legacy api Whether legacy API changes should be allowed in PR label Oct 18, 2024

bugraoz93 force-pushed the feat/42592/patch-connections-fast-api branch 2 times, most recently from f33126e to e58dae7Compare October 24, 2024 18:49

pierrejeambrun reviewed Oct 25, 2024
View reviewed changes

Comment threadnewsfragments/43102.significant.rst Outdated

bugraoz93 force-pushed the feat/42592/patch-connections-fast-api branch 2 times, most recently from 94bfe26 to fa62ca2Compare October 26, 2024 11:06

bugraoz93 mentioned this pull request Oct 27, 2024
AIP-84 Migrate post a connection to FastAPI API #43396
Merged

bugraoz93 force-pushed the feat/42592/patch-connections-fast-api branch from fa62ca2 to 104d004Compare October 27, 2024 01:04

pierrejeambrun approved these changes Oct 28, 2024
View reviewed changes

pierrejeambrun force-pushed the feat/42592/patch-connections-fast-api branch from 104d004 to f4c90ddCompare October 28, 2024 11:28

pierrejeambrun requested changes Oct 29, 2024
View reviewed changes

pierrejeambrun reviewed Oct 30, 2024
View reviewed changes

Comment threadtests/api_fastapi/core_api/routes/public/test_connections.py Outdated

bugraoz93 force-pushed the feat/42592/patch-connections-fast-api branch from 2b85e78 to 0b67aaeCompare October 31, 2024 21:13

pierrejeambrun reviewed Nov 4, 2024
View reviewed changes

Comment threadairflow/api_fastapi/core_api/serializers/connections.py Outdated

bugraoz93 force-pushed the feat/42592/patch-connections-fast-api branch from 3ea9442 to 591657dCompare November 4, 2024 18:46

bugraoz93 mentioned this pull request Nov 4, 2024
AIP-84 Standardize PATCH Behavior Across Endpoints in FastAPI #43650
Closed
2 tasks

bugraoz93 force-pushed the feat/42592/patch-connections-fast-api branch from 591657d to d26497aCompare November 4, 2024 19:57

pierrejeambrun approved these changes Nov 5, 2024
View reviewed changes

bugraoz93 added 13 commits November 5, 2024 20:04

Migrate Patch Connection endpoint to FastAPI and Include Optional par…
8d18b4f
…ameter Field default value for backward compatibility

Make all tests parametrized
6de5eba

Make UpdateMask Annotated for Patch endpoints, update the serialisati…
c183f83
…on for backward compat and fix for variables serialisation won't update val in setattr()

Merge duplicate imports
33caa22

Revert update_mask to explode true and include significant.rst for up…
d44a189
…date_mask breaking change

Amend update_mask as array
5b99f03

generalize handling of list query parameters in newsfragment
84bb524

Include password field
1b7f72c

Include password to response and tests, rebase and rerun pre-commit
ded78ed

Fix copy&paste mistakes for method and naming
b40014b

Convert redact_password to agreed approach, rebase and pre-commits
8df443a

Include new exception doc generation
d2a6029

Run pre-commit after rebase
7c19e96

bugraoz93 force-pushed the feat/42592/patch-connections-fast-api branch from d26497a to 7c19e96Compare November 5, 2024 19:11

pierrejeambrun merged commit cd75707 into apache:mainNov 6, 2024

Lee-W mentioned this pull request Nov 18, 2024
Airflow 2 to 3 auto migration rules #41641
Closed
2 tasks

potiuk mentioned this pull request Jan 16, 2025
Update existing significant newsfragments with the later introduced template format #45678
Merged

Uh oh!

Conversation

bugraoz93 commented Oct 17, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun commented Oct 28, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pierrejeambrun commented Oct 28, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bugraoz93 commented Oct 28, 2024

Uh oh!

bugraoz93 commented Oct 28, 2024

Uh oh!

pierrejeambrun commented Oct 29, 2024

Uh oh!

bugraoz93 commented Oct 29, 2024

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bugraoz93 commented Nov 4, 2024

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

bugraoz93 commented Nov 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pierrejeambrun commented Oct 28, 2024•
edited
Loading

pierrejeambrun commented Oct 28, 2024•
edited
Loading