[v3-0-test] Switch pre-commit to prek (#54258) by potiuk · Pull Request #54585 · apache/airflow

potiuk · 2025-08-17T11:51:33Z

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of uv and the only way to use uv with pre-commit is to patch it by installing pre-comit-uv that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs.

Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses uv natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving prek to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits.

This PR switches our pre-commit support to use prek exclusively:

breeze static checks command is completely removed
custom auto-complete code in breeze as well
instructions are updated to setup prek instead of precommit
CI is updated to run prek instead of pre-commmit
documentation for static checks is reviewed and new features that prek enables are added

(cherry picked from commit 3442d81)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

potiuk · 2025-08-17T11:53:45Z

@jscheffl -> Was not that bad. I had to sync breeze with latest main (not diffocult) and bring back a few Python 3.9 construncts in that code, but other than that, it was rather painless. Let's see if the CI agrees as well.

potiuk · 2025-08-17T12:26:06Z

Few breeze unit tests, but should be fixed now.

aritra24

Mostly nits, can be merged other than them

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added (cherry picked from commit 3442d81)

potiuk · 2025-08-17T12:42:41Z

Thanks @aritra24 ! addressed all.

jscheffl

Did not find/see another typo, LGTM!

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added

potiuk · 2025-08-17T11:51:33Z

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of uv and the only way to use uv with pre-commit is to patch it by installing pre-comit-uv that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs.

Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses uv natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving prek to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits.

This PR switches our pre-commit support to use prek exclusively:

breeze static checks command is completely removed
custom auto-complete code in breeze as well
instructions are updated to setup prek instead of precommit
CI is updated to run prek instead of pre-commmit
documentation for static checks is reviewed and new features that prek enables are added

(cherry picked from commit 3442d81)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

potiuk · 2025-08-17T11:53:45Z

@jscheffl -> Was not that bad. I had to sync breeze with latest main (not diffocult) and bring back a few Python 3.9 construncts in that code, but other than that, it was rather painless. Let's see if the CI agrees as well.

potiuk · 2025-08-17T12:26:06Z

Few breeze unit tests, but should be fixed now.

aritra24

Mostly nits, can be merged other than them

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added (cherry picked from commit 3442d81)

potiuk · 2025-08-17T12:42:41Z

Thanks @aritra24 ! addressed all.

jscheffl

Did not find/see another typo, LGTM!

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added

potiuk · 2025-08-17T11:51:33Z

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of uv and the only way to use uv with pre-commit is to patch it by installing pre-comit-uv that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs.

Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses uv natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving prek to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits.

This PR switches our pre-commit support to use prek exclusively:

breeze static checks command is completely removed
custom auto-complete code in breeze as well
instructions are updated to setup prek instead of precommit
CI is updated to run prek instead of pre-commmit
documentation for static checks is reviewed and new features that prek enables are added

(cherry picked from commit 3442d81)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

potiuk · 2025-08-17T11:53:45Z

@jscheffl -> Was not that bad. I had to sync breeze with latest main (not diffocult) and bring back a few Python 3.9 construncts in that code, but other than that, it was rather painless. Let's see if the CI agrees as well.

potiuk · 2025-08-17T12:26:06Z

Few breeze unit tests, but should be fixed now.

aritra24

Mostly nits, can be merged other than them

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added (cherry picked from commit 3442d81)

potiuk · 2025-08-17T12:42:41Z

Thanks @aritra24 ! addressed all.

jscheffl

Did not find/see another typo, LGTM!

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added

potiuk · 2025-08-17T11:51:33Z

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of uv and the only way to use uv with pre-commit is to patch it by installing pre-comit-uv that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs.

Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses uv natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving prek to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits.

This PR switches our pre-commit support to use prek exclusively:

breeze static checks command is completely removed
custom auto-complete code in breeze as well
instructions are updated to setup prek instead of precommit
CI is updated to run prek instead of pre-commmit
documentation for static checks is reviewed and new features that prek enables are added

(cherry picked from commit 3442d81)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

potiuk · 2025-08-17T11:53:45Z

@jscheffl -> Was not that bad. I had to sync breeze with latest main (not diffocult) and bring back a few Python 3.9 construncts in that code, but other than that, it was rather painless. Let's see if the CI agrees as well.

potiuk · 2025-08-17T12:26:06Z

Few breeze unit tests, but should be fixed now.

aritra24

Mostly nits, can be merged other than them

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added (cherry picked from commit 3442d81)

potiuk · 2025-08-17T12:42:41Z

Thanks @aritra24 ! addressed all.

jscheffl

Did not find/see another typo, LGTM!

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added

potiuk · 2025-08-17T11:51:33Z

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of uv and the only way to use uv with pre-commit is to patch it by installing pre-comit-uv that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs.

Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses uv natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving prek to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits.

This PR switches our pre-commit support to use prek exclusively:

breeze static checks command is completely removed
custom auto-complete code in breeze as well
instructions are updated to setup prek instead of precommit
CI is updated to run prek instead of pre-commmit
documentation for static checks is reviewed and new features that prek enables are added

(cherry picked from commit 3442d81)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

potiuk · 2025-08-17T11:53:45Z

@jscheffl -> Was not that bad. I had to sync breeze with latest main (not diffocult) and bring back a few Python 3.9 construncts in that code, but other than that, it was rather painless. Let's see if the CI agrees as well.

potiuk · 2025-08-17T12:26:06Z

Few breeze unit tests, but should be fixed now.

aritra24

Mostly nits, can be merged other than them

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added (cherry picked from commit 3442d81)

potiuk · 2025-08-17T12:42:41Z

Thanks @aritra24 ! addressed all.

jscheffl

Did not find/see another typo, LGTM!

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added

potiuk · 2025-08-17T11:51:33Z

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of uv and the only way to use uv with pre-commit is to patch it by installing pre-comit-uv that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs.

Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses uv natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving prek to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits.

This PR switches our pre-commit support to use prek exclusively:

breeze static checks command is completely removed
custom auto-complete code in breeze as well
instructions are updated to setup prek instead of precommit
CI is updated to run prek instead of pre-commmit
documentation for static checks is reviewed and new features that prek enables are added

(cherry picked from commit 3442d81)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

potiuk · 2025-08-17T11:53:45Z

@jscheffl -> Was not that bad. I had to sync breeze with latest main (not diffocult) and bring back a few Python 3.9 construncts in that code, but other than that, it was rather painless. Let's see if the CI agrees as well.

potiuk · 2025-08-17T12:26:06Z

Few breeze unit tests, but should be fixed now.

aritra24

Mostly nits, can be merged other than them

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added (cherry picked from commit 3442d81)

potiuk · 2025-08-17T12:42:41Z

Thanks @aritra24 ! addressed all.

jscheffl

Did not find/see another typo, LGTM!

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added

potiuk · 2025-08-17T11:51:33Z

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of uv and the only way to use uv with pre-commit is to patch it by installing pre-comit-uv that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs.

Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses uv natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving prek to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits.

This PR switches our pre-commit support to use prek exclusively:

breeze static checks command is completely removed
custom auto-complete code in breeze as well
instructions are updated to setup prek instead of precommit
CI is updated to run prek instead of pre-commmit
documentation for static checks is reviewed and new features that prek enables are added

(cherry picked from commit 3442d81)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

potiuk · 2025-08-17T11:53:45Z

@jscheffl -> Was not that bad. I had to sync breeze with latest main (not diffocult) and bring back a few Python 3.9 construncts in that code, but other than that, it was rather painless. Let's see if the CI agrees as well.

potiuk · 2025-08-17T12:26:06Z

Few breeze unit tests, but should be fixed now.

aritra24

Mostly nits, can be merged other than them

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added (cherry picked from commit 3442d81)

potiuk · 2025-08-17T12:42:41Z

Thanks @aritra24 ! addressed all.

jscheffl

Did not find/see another typo, LGTM!

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added

potiuk · 2025-08-17T11:51:33Z

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of uv and the only way to use uv with pre-commit is to patch it by installing pre-comit-uv that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs.

Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses uv natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving prek to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits.

This PR switches our pre-commit support to use prek exclusively:

breeze static checks command is completely removed
custom auto-complete code in breeze as well
instructions are updated to setup prek instead of precommit
CI is updated to run prek instead of pre-commmit
documentation for static checks is reviewed and new features that prek enables are added

(cherry picked from commit 3442d81)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

potiuk · 2025-08-17T11:53:45Z

@jscheffl -> Was not that bad. I had to sync breeze with latest main (not diffocult) and bring back a few Python 3.9 construncts in that code, but other than that, it was rather painless. Let's see if the CI agrees as well.

potiuk · 2025-08-17T12:26:06Z

Few breeze unit tests, but should be fixed now.

aritra24

Mostly nits, can be merged other than them

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added (cherry picked from commit 3442d81)

potiuk · 2025-08-17T12:42:41Z

Thanks @aritra24 ! addressed all.

jscheffl

Did not find/see another typo, LGTM!

The pre-commit is a fantastic tool, and we heavily used it for years, but generally the tool stagnated and is not showing a sign of adapting to our needs. For years we tried to convince pre-commit maintainers that things like autocomplete are necessary - but it met with pretty much resistance (if not hostility) from the maintainer. Also there was no chance for them to accept expectations of bigger projects like ours, where we have a huge monorepo and not only multiple needs but also different parts of the repo needing different language support (golang, typescript soon) - and apparenty the maintainer of pre-commit does not think monorepo is a good thing at all. Similarly - they did not recognize the raise of `uv` and the only way to use `uv` with pre-commit is to patch it by installing `pre-comit-uv` that essentialy patches pre-commit with uv support. This is not really sustainable and the tool lags behind many of our needs. Luckily - we have new project in town - prek - which rewrites pre-commit that is 100% compatible (now), 10x faster (because rust), uses `uv` natively, supports auto-complete already and they have very friendly maintainer who is not only supporting us but also very happily works on improving `prek` to close all the gaps, and plans to implement (with our support of course and cooperation) monorepo support - that will allow us to modularise our pre-commits. This PR switches our pre-commit support to use prek exclusively: * breeze static checks command is completely removed * custom auto-complete code in breeze as well * instructions are updated to setup prek instead of precommit * CI is updated to run prek instead of pre-commmit * documentation for static checks is reviewed and new features that prek enables are added

potiuk requested review from Fokko, amoghrajesh, ashb, bolkedebruin, eladkal, gopidesupavan, hussein-awala, jason810496, jedcunningham, jscheffl, kaxil, mik-laj, mobuchowski, pierrejeambrun and vincbeck as code owners August 17, 2025 11:51

boring-cyborgBot added area:API Airflow's REST/HTTP API area:CLI labels Aug 17, 2025

potiuk requested a review from josh-fell as a code owner August 17, 2025 11:51

boring-cyborgBot added area:dev-tools area:production-image Production image improvements and fixes labels Aug 17, 2025

potiuk requested a review from o-nikolas as a code owner August 17, 2025 11:51

boring-cyborgBot added the backport-to-v3-0-test label Aug 17, 2025

potiuk requested a review from ephraimbuddy as a code owner August 17, 2025 11:51

boring-cyborgBot added the kind:documentation label Aug 17, 2025

potiuk requested review from bugraoz93, rawwar and shubhamraj-git as code owners August 17, 2025 11:51

potiuk removed request for Fokko, ashb and ephraimbuddy August 17, 2025 11:51

potiuk requested review from aritra24, gopidesupavan and shahar1 and removed request for eladkal, gopidesupavan, hussein-awala, jason810496, jedcunningham, josh-fell, kaxil, mik-laj, mobuchowski, o-nikolas, pierrejeambrun, rawwar, shubhamraj-git and vincbeck August 17, 2025 11:51

potiuk force-pushed the backport-3442d81-v3-0-test branch from 4035854 to a80f403Compare August 17, 2025 12:25

aritra24 approved these changes Aug 17, 2025
View reviewed changes

Comment threadAGENTS.md
Comment threadDockerfile Outdated
Comment threadDockerfile.ci Outdated
Comment threadDockerfile.ci
Comment threadcontributing-docs/06_development_environments.rst Outdated

potiuk force-pushed the backport-3442d81-v3-0-test branch from a80f403 to 4c99eccCompare August 17, 2025 12:42

jscheffl reviewed Aug 17, 2025
View reviewed changes

Comment threaddev/breeze/doc/05_test_commands.rst

jscheffl approved these changes Aug 17, 2025
View reviewed changes

potiuk merged commit 16fc308 into apache:v3-0-testAug 17, 2025
129 checks passed

potiuk deleted the backport-3442d81-v3-0-test branch August 17, 2025 14:00

kaxil mentioned this pull request Aug 22, 2025
Status of testing of Apache Airflow 3.0.6rc1 & Task SDK 1.0.6rc1#54835
Closed
37 tasks