Skip to content

build(prek-hook): Check whether new "raise AirflowException" is added - #55416

Merged
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except
Apr 9, 2026
Merged

build(prek-hook): Check whether new "raise AirflowException" is added#55416
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except

Conversation

@Lee-W

@Lee-WLee-W commented Sep 9, 2025

Copy link
Copy Markdown
Member

Why

https://lists.apache.org/thread/t8bnhyqy77kq4fk7fj3fmjd5wo9kv6w0

What

Add prek hook to detect raise AirflowException


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from f67ff02 to 23f571cCompareSeptember 13, 2025 06:45
@Lee-W
Lee-W marked this pull request as ready for review September 13, 2025 06:52

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W

Copy link
Copy Markdown
MemberAuthor

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Yep, we probably could exclude that file if needed. But this is the best I can think of now.

Comment thread.pre-commit-config.yaml Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 3 times, most recently from 506f630 to 9cd9e6dCompareSeptember 22, 2025 08:04
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9cd9e6d to 30615aeCompareSeptember 28, 2025 02:59
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 4 times, most recently from 288a6a3 to 73f587cCompareOctober 13, 2025 02:03
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 2 times, most recently from aaab427 to 621e7bcCompareOctober 13, 2025 13:02
@potiuk

potiuk commented Oct 13, 2025

Copy link
Copy Markdown
Member

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Maybe better solution:

  • dump (automatically) list of used Exceptions to a .txt file
  • fail if a found exception is not on the list
  • add a flag to the script to clean-up removed exceptions when --all-files are used
  • prek run check_no_new_airflow_exceptions_added (default - fails if not on list)
  • prek run check_no_new_airflow_exceptions_added --all-files (check all)
  • uv run ./scripts/ci/prek/check_no_new_airflow_exceptions_added.py --cleanup - removes all exceptions from .txt file when already removed in the code

Or similar.

@Lee-W
Lee-W marked this pull request as draft October 15, 2025 10:10
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 621e7bc to b109e01CompareOctober 22, 2025 08:41
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@Lee-W
Lee-W requested a review from jason810496 as a code ownerApril 7, 2026 08:05
@Lee-W

Lee-W commented Apr 7, 2026

Copy link
Copy Markdown
MemberAuthor

@potiuk@shahar1 I tried to wrap up this PR created long long long ago (with Claude). Please take a look when you have some time :)

@shahar1shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for creating it, and apologies that I couldn't finish it up on my own.
Few comments :)

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 5c26c0b to c3b6c00CompareApril 7, 2026 09:16
@Lee-W
Lee-W requested a review from shahar1April 7, 2026 11:30
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9ebcfb6 to 99ada8eCompareApril 9, 2026 03:27
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 99ada8e to 4724c32CompareApril 9, 2026 09:33
@potiuk

Copy link
Copy Markdown
Member

Oh wow. We have that many ?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I like this simple and smart way to keep track of all existing overly broad ‎AirflowException usages. It should help avoid adding too many broad ‎AirflowException usages and will serve as a final warning for contributors.

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, wow, really many. Let me fix the edge3 provider one :-D

@jscheffl
jscheffl merged commit 6f56aa1 into apache:mainApr 9, 2026
142 checks passed
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
(cherry picked from commit 6f56aa1)
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
Lee-W added a commit that referenced this pull request Apr 10, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@Lee-W@potiuk@amoghrajesh@shahar1@eladkal@jason810496@jscheffl@kaxil
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
build(prek-hook): Check whether new "raise AirflowException" is added by Lee-W · Pull Request #55416 · apache/airflow · GitHub
Skip to content

build(prek-hook): Check whether new "raise AirflowException" is added - #55416

Merged
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except
Apr 9, 2026
Merged

build(prek-hook): Check whether new "raise AirflowException" is added#55416
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except

Conversation

@Lee-W

@Lee-WLee-W commented Sep 9, 2025

Copy link
Copy Markdown
Member

Why

https://lists.apache.org/thread/t8bnhyqy77kq4fk7fj3fmjd5wo9kv6w0

What

Add prek hook to detect raise AirflowException


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from f67ff02 to 23f571cCompareSeptember 13, 2025 06:45
@Lee-W
Lee-W marked this pull request as ready for review September 13, 2025 06:52

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W

Copy link
Copy Markdown
MemberAuthor

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Yep, we probably could exclude that file if needed. But this is the best I can think of now.

Comment thread.pre-commit-config.yaml Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 3 times, most recently from 506f630 to 9cd9e6dCompareSeptember 22, 2025 08:04
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9cd9e6d to 30615aeCompareSeptember 28, 2025 02:59
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 4 times, most recently from 288a6a3 to 73f587cCompareOctober 13, 2025 02:03
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 2 times, most recently from aaab427 to 621e7bcCompareOctober 13, 2025 13:02
@potiuk

potiuk commented Oct 13, 2025

Copy link
Copy Markdown
Member

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Maybe better solution:

  • dump (automatically) list of used Exceptions to a .txt file
  • fail if a found exception is not on the list
  • add a flag to the script to clean-up removed exceptions when --all-files are used
  • prek run check_no_new_airflow_exceptions_added (default - fails if not on list)
  • prek run check_no_new_airflow_exceptions_added --all-files (check all)
  • uv run ./scripts/ci/prek/check_no_new_airflow_exceptions_added.py --cleanup - removes all exceptions from .txt file when already removed in the code

Or similar.

@Lee-W
Lee-W marked this pull request as draft October 15, 2025 10:10
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 621e7bc to b109e01CompareOctober 22, 2025 08:41
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@Lee-W
Lee-W requested a review from jason810496 as a code ownerApril 7, 2026 08:05
@Lee-W

Lee-W commented Apr 7, 2026

Copy link
Copy Markdown
MemberAuthor

@potiuk@shahar1 I tried to wrap up this PR created long long long ago (with Claude). Please take a look when you have some time :)

@shahar1shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for creating it, and apologies that I couldn't finish it up on my own.
Few comments :)

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 5c26c0b to c3b6c00CompareApril 7, 2026 09:16
@Lee-W
Lee-W requested a review from shahar1April 7, 2026 11:30
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9ebcfb6 to 99ada8eCompareApril 9, 2026 03:27
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 99ada8e to 4724c32CompareApril 9, 2026 09:33
@potiuk

Copy link
Copy Markdown
Member

Oh wow. We have that many ?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I like this simple and smart way to keep track of all existing overly broad ‎AirflowException usages. It should help avoid adding too many broad ‎AirflowException usages and will serve as a final warning for contributors.

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, wow, really many. Let me fix the edge3 provider one :-D

@jscheffl
jscheffl merged commit 6f56aa1 into apache:mainApr 9, 2026
142 checks passed
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
(cherry picked from commit 6f56aa1)
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
Lee-W added a commit that referenced this pull request Apr 10, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@Lee-W@potiuk@amoghrajesh@shahar1@eladkal@jason810496@jscheffl@kaxil
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' build(prek-hook): Check whether new "raise AirflowException" is added by Lee-W · Pull Request #55416 · apache/airflow · GitHub
Skip to content

build(prek-hook): Check whether new "raise AirflowException" is added - #55416

Merged
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except
Apr 9, 2026
Merged

build(prek-hook): Check whether new "raise AirflowException" is added#55416
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except

Conversation

@Lee-W

@Lee-WLee-W commented Sep 9, 2025

Copy link
Copy Markdown
Member

Why

https://lists.apache.org/thread/t8bnhyqy77kq4fk7fj3fmjd5wo9kv6w0

What

Add prek hook to detect raise AirflowException


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from f67ff02 to 23f571cCompareSeptember 13, 2025 06:45
@Lee-W
Lee-W marked this pull request as ready for review September 13, 2025 06:52

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W

Copy link
Copy Markdown
MemberAuthor

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Yep, we probably could exclude that file if needed. But this is the best I can think of now.

Comment thread.pre-commit-config.yaml Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 3 times, most recently from 506f630 to 9cd9e6dCompareSeptember 22, 2025 08:04
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9cd9e6d to 30615aeCompareSeptember 28, 2025 02:59
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 4 times, most recently from 288a6a3 to 73f587cCompareOctober 13, 2025 02:03
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 2 times, most recently from aaab427 to 621e7bcCompareOctober 13, 2025 13:02
@potiuk

potiuk commented Oct 13, 2025

Copy link
Copy Markdown
Member

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Maybe better solution:

  • dump (automatically) list of used Exceptions to a .txt file
  • fail if a found exception is not on the list
  • add a flag to the script to clean-up removed exceptions when --all-files are used
  • prek run check_no_new_airflow_exceptions_added (default - fails if not on list)
  • prek run check_no_new_airflow_exceptions_added --all-files (check all)
  • uv run ./scripts/ci/prek/check_no_new_airflow_exceptions_added.py --cleanup - removes all exceptions from .txt file when already removed in the code

Or similar.

@Lee-W
Lee-W marked this pull request as draft October 15, 2025 10:10
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 621e7bc to b109e01CompareOctober 22, 2025 08:41
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@Lee-W
Lee-W requested a review from jason810496 as a code ownerApril 7, 2026 08:05
@Lee-W

Lee-W commented Apr 7, 2026

Copy link
Copy Markdown
MemberAuthor

@potiuk@shahar1 I tried to wrap up this PR created long long long ago (with Claude). Please take a look when you have some time :)

@shahar1shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for creating it, and apologies that I couldn't finish it up on my own.
Few comments :)

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 5c26c0b to c3b6c00CompareApril 7, 2026 09:16
@Lee-W
Lee-W requested a review from shahar1April 7, 2026 11:30
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9ebcfb6 to 99ada8eCompareApril 9, 2026 03:27
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 99ada8e to 4724c32CompareApril 9, 2026 09:33
@potiuk

Copy link
Copy Markdown
Member

Oh wow. We have that many ?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I like this simple and smart way to keep track of all existing overly broad ‎AirflowException usages. It should help avoid adding too many broad ‎AirflowException usages and will serve as a final warning for contributors.

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, wow, really many. Let me fix the edge3 provider one :-D

@jscheffl
jscheffl merged commit 6f56aa1 into apache:mainApr 9, 2026
142 checks passed
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
(cherry picked from commit 6f56aa1)
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
Lee-W added a commit that referenced this pull request Apr 10, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@Lee-W@potiuk@amoghrajesh@shahar1@eladkal@jason810496@jscheffl@kaxil
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' build(prek-hook): Check whether new "raise AirflowException" is added by Lee-W · Pull Request #55416 · apache/airflow · GitHub
Skip to content

build(prek-hook): Check whether new "raise AirflowException" is added - #55416

Merged
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except
Apr 9, 2026
Merged

build(prek-hook): Check whether new "raise AirflowException" is added#55416
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except

Conversation

@Lee-W

@Lee-WLee-W commented Sep 9, 2025

Copy link
Copy Markdown
Member

Why

https://lists.apache.org/thread/t8bnhyqy77kq4fk7fj3fmjd5wo9kv6w0

What

Add prek hook to detect raise AirflowException


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from f67ff02 to 23f571cCompareSeptember 13, 2025 06:45
@Lee-W
Lee-W marked this pull request as ready for review September 13, 2025 06:52

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W

Copy link
Copy Markdown
MemberAuthor

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Yep, we probably could exclude that file if needed. But this is the best I can think of now.

Comment thread.pre-commit-config.yaml Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 3 times, most recently from 506f630 to 9cd9e6dCompareSeptember 22, 2025 08:04
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9cd9e6d to 30615aeCompareSeptember 28, 2025 02:59
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 4 times, most recently from 288a6a3 to 73f587cCompareOctober 13, 2025 02:03
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 2 times, most recently from aaab427 to 621e7bcCompareOctober 13, 2025 13:02
@potiuk

potiuk commented Oct 13, 2025

Copy link
Copy Markdown
Member

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Maybe better solution:

  • dump (automatically) list of used Exceptions to a .txt file
  • fail if a found exception is not on the list
  • add a flag to the script to clean-up removed exceptions when --all-files are used
  • prek run check_no_new_airflow_exceptions_added (default - fails if not on list)
  • prek run check_no_new_airflow_exceptions_added --all-files (check all)
  • uv run ./scripts/ci/prek/check_no_new_airflow_exceptions_added.py --cleanup - removes all exceptions from .txt file when already removed in the code

Or similar.

@Lee-W
Lee-W marked this pull request as draft October 15, 2025 10:10
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 621e7bc to b109e01CompareOctober 22, 2025 08:41
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@Lee-W
Lee-W requested a review from jason810496 as a code ownerApril 7, 2026 08:05
@Lee-W

Lee-W commented Apr 7, 2026

Copy link
Copy Markdown
MemberAuthor

@potiuk@shahar1 I tried to wrap up this PR created long long long ago (with Claude). Please take a look when you have some time :)

@shahar1shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for creating it, and apologies that I couldn't finish it up on my own.
Few comments :)

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 5c26c0b to c3b6c00CompareApril 7, 2026 09:16
@Lee-W
Lee-W requested a review from shahar1April 7, 2026 11:30
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9ebcfb6 to 99ada8eCompareApril 9, 2026 03:27
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 99ada8e to 4724c32CompareApril 9, 2026 09:33
@potiuk

Copy link
Copy Markdown
Member

Oh wow. We have that many ?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I like this simple and smart way to keep track of all existing overly broad ‎AirflowException usages. It should help avoid adding too many broad ‎AirflowException usages and will serve as a final warning for contributors.

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, wow, really many. Let me fix the edge3 provider one :-D

@jscheffl
jscheffl merged commit 6f56aa1 into apache:mainApr 9, 2026
142 checks passed
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
(cherry picked from commit 6f56aa1)
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
Lee-W added a commit that referenced this pull request Apr 10, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@Lee-W@potiuk@amoghrajesh@shahar1@eladkal@jason810496@jscheffl@kaxil
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' build(prek-hook): Check whether new "raise AirflowException" is added by Lee-W · Pull Request #55416 · apache/airflow · GitHub
Skip to content

build(prek-hook): Check whether new "raise AirflowException" is added - #55416

Merged
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except
Apr 9, 2026
Merged

build(prek-hook): Check whether new "raise AirflowException" is added#55416
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except

Conversation

@Lee-W

@Lee-WLee-W commented Sep 9, 2025

Copy link
Copy Markdown
Member

Why

https://lists.apache.org/thread/t8bnhyqy77kq4fk7fj3fmjd5wo9kv6w0

What

Add prek hook to detect raise AirflowException


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from f67ff02 to 23f571cCompareSeptember 13, 2025 06:45
@Lee-W
Lee-W marked this pull request as ready for review September 13, 2025 06:52

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W

Copy link
Copy Markdown
MemberAuthor

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Yep, we probably could exclude that file if needed. But this is the best I can think of now.

Comment thread.pre-commit-config.yaml Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 3 times, most recently from 506f630 to 9cd9e6dCompareSeptember 22, 2025 08:04
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9cd9e6d to 30615aeCompareSeptember 28, 2025 02:59
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 4 times, most recently from 288a6a3 to 73f587cCompareOctober 13, 2025 02:03
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 2 times, most recently from aaab427 to 621e7bcCompareOctober 13, 2025 13:02
@potiuk

potiuk commented Oct 13, 2025

Copy link
Copy Markdown
Member

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Maybe better solution:

  • dump (automatically) list of used Exceptions to a .txt file
  • fail if a found exception is not on the list
  • add a flag to the script to clean-up removed exceptions when --all-files are used
  • prek run check_no_new_airflow_exceptions_added (default - fails if not on list)
  • prek run check_no_new_airflow_exceptions_added --all-files (check all)
  • uv run ./scripts/ci/prek/check_no_new_airflow_exceptions_added.py --cleanup - removes all exceptions from .txt file when already removed in the code

Or similar.

@Lee-W
Lee-W marked this pull request as draft October 15, 2025 10:10
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 621e7bc to b109e01CompareOctober 22, 2025 08:41
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@Lee-W
Lee-W requested a review from jason810496 as a code ownerApril 7, 2026 08:05
@Lee-W

Lee-W commented Apr 7, 2026

Copy link
Copy Markdown
MemberAuthor

@potiuk@shahar1 I tried to wrap up this PR created long long long ago (with Claude). Please take a look when you have some time :)

@shahar1shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for creating it, and apologies that I couldn't finish it up on my own.
Few comments :)

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 5c26c0b to c3b6c00CompareApril 7, 2026 09:16
@Lee-W
Lee-W requested a review from shahar1April 7, 2026 11:30
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9ebcfb6 to 99ada8eCompareApril 9, 2026 03:27
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 99ada8e to 4724c32CompareApril 9, 2026 09:33
@potiuk

Copy link
Copy Markdown
Member

Oh wow. We have that many ?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I like this simple and smart way to keep track of all existing overly broad ‎AirflowException usages. It should help avoid adding too many broad ‎AirflowException usages and will serve as a final warning for contributors.

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, wow, really many. Let me fix the edge3 provider one :-D

@jscheffl
jscheffl merged commit 6f56aa1 into apache:mainApr 9, 2026
142 checks passed
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
(cherry picked from commit 6f56aa1)
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
Lee-W added a commit that referenced this pull request Apr 10, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@Lee-W@potiuk@amoghrajesh@shahar1@eladkal@jason810496@jscheffl@kaxil
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' build(prek-hook): Check whether new "raise AirflowException" is added by Lee-W · Pull Request #55416 · apache/airflow · GitHub
Skip to content

build(prek-hook): Check whether new "raise AirflowException" is added - #55416

Merged
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except
Apr 9, 2026
Merged

build(prek-hook): Check whether new "raise AirflowException" is added#55416
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except

Conversation

@Lee-W

@Lee-WLee-W commented Sep 9, 2025

Copy link
Copy Markdown
Member

Why

https://lists.apache.org/thread/t8bnhyqy77kq4fk7fj3fmjd5wo9kv6w0

What

Add prek hook to detect raise AirflowException


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from f67ff02 to 23f571cCompareSeptember 13, 2025 06:45
@Lee-W
Lee-W marked this pull request as ready for review September 13, 2025 06:52

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W

Copy link
Copy Markdown
MemberAuthor

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Yep, we probably could exclude that file if needed. But this is the best I can think of now.

Comment thread.pre-commit-config.yaml Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 3 times, most recently from 506f630 to 9cd9e6dCompareSeptember 22, 2025 08:04
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9cd9e6d to 30615aeCompareSeptember 28, 2025 02:59
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 4 times, most recently from 288a6a3 to 73f587cCompareOctober 13, 2025 02:03
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 2 times, most recently from aaab427 to 621e7bcCompareOctober 13, 2025 13:02
@potiuk

potiuk commented Oct 13, 2025

Copy link
Copy Markdown
Member

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Maybe better solution:

  • dump (automatically) list of used Exceptions to a .txt file
  • fail if a found exception is not on the list
  • add a flag to the script to clean-up removed exceptions when --all-files are used
  • prek run check_no_new_airflow_exceptions_added (default - fails if not on list)
  • prek run check_no_new_airflow_exceptions_added --all-files (check all)
  • uv run ./scripts/ci/prek/check_no_new_airflow_exceptions_added.py --cleanup - removes all exceptions from .txt file when already removed in the code

Or similar.

@Lee-W
Lee-W marked this pull request as draft October 15, 2025 10:10
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 621e7bc to b109e01CompareOctober 22, 2025 08:41
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@Lee-W
Lee-W requested a review from jason810496 as a code ownerApril 7, 2026 08:05
@Lee-W

Lee-W commented Apr 7, 2026

Copy link
Copy Markdown
MemberAuthor

@potiuk@shahar1 I tried to wrap up this PR created long long long ago (with Claude). Please take a look when you have some time :)

@shahar1shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for creating it, and apologies that I couldn't finish it up on my own.
Few comments :)

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 5c26c0b to c3b6c00CompareApril 7, 2026 09:16
@Lee-W
Lee-W requested a review from shahar1April 7, 2026 11:30
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9ebcfb6 to 99ada8eCompareApril 9, 2026 03:27
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 99ada8e to 4724c32CompareApril 9, 2026 09:33
@potiuk

Copy link
Copy Markdown
Member

Oh wow. We have that many ?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I like this simple and smart way to keep track of all existing overly broad ‎AirflowException usages. It should help avoid adding too many broad ‎AirflowException usages and will serve as a final warning for contributors.

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, wow, really many. Let me fix the edge3 provider one :-D

@jscheffl
jscheffl merged commit 6f56aa1 into apache:mainApr 9, 2026
142 checks passed
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
(cherry picked from commit 6f56aa1)
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
Lee-W added a commit that referenced this pull request Apr 10, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@Lee-W@potiuk@amoghrajesh@shahar1@eladkal@jason810496@jscheffl@kaxil
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' build(prek-hook): Check whether new "raise AirflowException" is added by Lee-W · Pull Request #55416 · apache/airflow · GitHub
Skip to content

build(prek-hook): Check whether new "raise AirflowException" is added - #55416

Merged
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except
Apr 9, 2026
Merged

build(prek-hook): Check whether new "raise AirflowException" is added#55416
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except

Conversation

@Lee-W

@Lee-WLee-W commented Sep 9, 2025

Copy link
Copy Markdown
Member

Why

https://lists.apache.org/thread/t8bnhyqy77kq4fk7fj3fmjd5wo9kv6w0

What

Add prek hook to detect raise AirflowException


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from f67ff02 to 23f571cCompareSeptember 13, 2025 06:45
@Lee-W
Lee-W marked this pull request as ready for review September 13, 2025 06:52

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W

Copy link
Copy Markdown
MemberAuthor

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Yep, we probably could exclude that file if needed. But this is the best I can think of now.

Comment thread.pre-commit-config.yaml Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 3 times, most recently from 506f630 to 9cd9e6dCompareSeptember 22, 2025 08:04
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9cd9e6d to 30615aeCompareSeptember 28, 2025 02:59
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 4 times, most recently from 288a6a3 to 73f587cCompareOctober 13, 2025 02:03
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 2 times, most recently from aaab427 to 621e7bcCompareOctober 13, 2025 13:02
@potiuk

potiuk commented Oct 13, 2025

Copy link
Copy Markdown
Member

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Maybe better solution:

  • dump (automatically) list of used Exceptions to a .txt file
  • fail if a found exception is not on the list
  • add a flag to the script to clean-up removed exceptions when --all-files are used
  • prek run check_no_new_airflow_exceptions_added (default - fails if not on list)
  • prek run check_no_new_airflow_exceptions_added --all-files (check all)
  • uv run ./scripts/ci/prek/check_no_new_airflow_exceptions_added.py --cleanup - removes all exceptions from .txt file when already removed in the code

Or similar.

@Lee-W
Lee-W marked this pull request as draft October 15, 2025 10:10
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 621e7bc to b109e01CompareOctober 22, 2025 08:41
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@Lee-W
Lee-W requested a review from jason810496 as a code ownerApril 7, 2026 08:05
@Lee-W

Lee-W commented Apr 7, 2026

Copy link
Copy Markdown
MemberAuthor

@potiuk@shahar1 I tried to wrap up this PR created long long long ago (with Claude). Please take a look when you have some time :)

@shahar1shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for creating it, and apologies that I couldn't finish it up on my own.
Few comments :)

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 5c26c0b to c3b6c00CompareApril 7, 2026 09:16
@Lee-W
Lee-W requested a review from shahar1April 7, 2026 11:30
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9ebcfb6 to 99ada8eCompareApril 9, 2026 03:27
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 99ada8e to 4724c32CompareApril 9, 2026 09:33
@potiuk

Copy link
Copy Markdown
Member

Oh wow. We have that many ?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I like this simple and smart way to keep track of all existing overly broad ‎AirflowException usages. It should help avoid adding too many broad ‎AirflowException usages and will serve as a final warning for contributors.

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, wow, really many. Let me fix the edge3 provider one :-D

@jscheffl
jscheffl merged commit 6f56aa1 into apache:mainApr 9, 2026
142 checks passed
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
(cherry picked from commit 6f56aa1)
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
Lee-W added a commit that referenced this pull request Apr 10, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@Lee-W@potiuk@amoghrajesh@shahar1@eladkal@jason810496@jscheffl@kaxil
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); build(prek-hook): Check whether new "raise AirflowException" is added by Lee-W · Pull Request #55416 · apache/airflow · GitHub
Skip to content

build(prek-hook): Check whether new "raise AirflowException" is added - #55416

Merged
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except
Apr 9, 2026
Merged

build(prek-hook): Check whether new "raise AirflowException" is added#55416
jscheffl merged 6 commits into
apache:mainfrom
astronomer:script-to-check-new-airflow-except

Conversation

@Lee-W

@Lee-WLee-W commented Sep 9, 2025

Copy link
Copy Markdown
Member

Why

https://lists.apache.org/thread/t8bnhyqy77kq4fk7fj3fmjd5wo9kv6w0

What

Add prek hook to detect raise AirflowException


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from f67ff02 to 23f571cCompareSeptember 13, 2025 06:45
@Lee-W
Lee-W marked this pull request as ready for review September 13, 2025 06:52

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W

Copy link
Copy Markdown
MemberAuthor

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Yep, we probably could exclude that file if needed. But this is the best I can think of now.

Comment thread.pre-commit-config.yaml Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 3 times, most recently from 506f630 to 9cd9e6dCompareSeptember 22, 2025 08:04
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9cd9e6d to 30615aeCompareSeptember 28, 2025 02:59
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 4 times, most recently from 288a6a3 to 73f587cCompareOctober 13, 2025 02:03
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch 2 times, most recently from aaab427 to 621e7bcCompareOctober 13, 2025 13:02
@potiuk

potiuk commented Oct 13, 2025

Copy link
Copy Markdown
Member

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Maybe better solution:

  • dump (automatically) list of used Exceptions to a .txt file
  • fail if a found exception is not on the list
  • add a flag to the script to clean-up removed exceptions when --all-files are used
  • prek run check_no_new_airflow_exceptions_added (default - fails if not on list)
  • prek run check_no_new_airflow_exceptions_added --all-files (check all)
  • uv run ./scripts/ci/prek/check_no_new_airflow_exceptions_added.py --cleanup - removes all exceptions from .txt file when already removed in the code

Or similar.

@Lee-W
Lee-W marked this pull request as draft October 15, 2025 10:10
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 621e7bc to b109e01CompareOctober 22, 2025 08:41
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@Lee-W
Lee-W requested a review from jason810496 as a code ownerApril 7, 2026 08:05
@Lee-W

Lee-W commented Apr 7, 2026

Copy link
Copy Markdown
MemberAuthor

@potiuk@shahar1 I tried to wrap up this PR created long long long ago (with Claude). Please take a look when you have some time :)

@shahar1shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for creating it, and apologies that I couldn't finish it up on my own.
Few comments :)

Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment threadscripts/ci/prek/check_new_airflow_exception_usage.py
Comment thread.pre-commit-config.yaml Outdated
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 5c26c0b to c3b6c00CompareApril 7, 2026 09:16
@Lee-W
Lee-W requested a review from shahar1April 7, 2026 11:30
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 9ebcfb6 to 99ada8eCompareApril 9, 2026 03:27
@Lee-W
Lee-Wforce-pushed the script-to-check-new-airflow-except branch from 99ada8e to 4724c32CompareApril 9, 2026 09:33
@potiuk

Copy link
Copy Markdown
Member

Oh wow. We have that many ?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I like this simple and smart way to keep track of all existing overly broad ‎AirflowException usages. It should help avoid adding too many broad ‎AirflowException usages and will serve as a final warning for contributors.

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, wow, really many. Let me fix the edge3 provider one :-D

@jscheffl
jscheffl merged commit 6f56aa1 into apache:mainApr 9, 2026
142 checks passed
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
(cherry picked from commit 6f56aa1)
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
…apache#55416)
* build(prek-hook): Check whether new "raise AirflowException" is added
* feat: add white list
* refactor: simplify the record to use count instead of raw content
* fixup! refactor: simplify the record to use count instead of raw content
* refactor: filter python files that matters
* fixup! refactor: filter python files that matters
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026
Lee-W added a commit that referenced this pull request Apr 10, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@Lee-W@potiuk@amoghrajesh@shahar1@eladkal@jason810496@jscheffl@kaxil