Add prek hook to ban caplog usage in test files

[amoghrajesh]

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Noticed recently in #65232 that caplog was used.

Wanted to add a prek hook to now allow this as per our coding standards: https://github.com/apache/airflow/blob/main/AGENTS.md#testing-standards.

Intent is to enforce the testing standard that tests should check logic rather than log output.

I am proposing to solve this by adding a pygrep hook (check-no-caplog-in-tests) to airflow-core, providers, and task-sdk pre commit config. The hook matches \bcaplog\b in test files and fails if any new usage is introduced.

Existing usages (35 core, 89 provider, 6 task-sdk files) are unaffected — prek only runs on files changed in a given PR, so cleanup can happen incrementally.

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[jason810496]

Nice improvement, I agree that we should to add the prek hook to prevent more caplog usage in the first place.

[bugraoz93]

Nice, thanks for including these, Amogh! I think we were checking in the root beforehand

[potiuk]

Rebased to run with "full tests"

[bugraoz93]

We should use the label for this now as it fails for all and maybe for accepting some if needed

[bugraoz93]

Since we call directly pygrep maybe we need to add skip option to prek but I don't know how or even possible

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

Oh, just see the list of static checks, seems like a call for helping hands is needed!

[amoghrajesh]

Oh haha, seems I need to do that. Let me take a stab at that

[Lee-W]

I thought we're to use ast 🤔 but pygrep should be good enough

[jason810496]

Would it be better to follow the convention of

• #55416
• #67150

by using a .txt to track the current violations. Then we could defer the fix for replacing all the caplog usage (or make it as good first issue contribution opportunity for the community)

Based on the CI error logs, there're hundreds violations now, refactoring in single PR might cause even bigger test regression IMHO.

[amoghrajesh]

Yes, thanks @jason810496. I will get to this next week when I have some time.

[amoghrajesh]

Finally got back to this one.

[jason810496]

LGTM after moving the generated known usage, thanks!

[ashb]

-1. Now we are using strictlog the caplog fixture is better than mocking.

[amoghrajesh]

Closing in favour of: #69803

[amoghrajesh]

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Noticed recently in #65232 that caplog was used.

Wanted to add a prek hook to now allow this as per our coding standards: https://github.com/apache/airflow/blob/main/AGENTS.md#testing-standards.

Intent is to enforce the testing standard that tests should check logic rather than log output.

I am proposing to solve this by adding a pygrep hook (check-no-caplog-in-tests) to airflow-core, providers, and task-sdk pre commit config. The hook matches \bcaplog\b in test files and fails if any new usage is introduced.

Existing usages (35 core, 89 provider, 6 task-sdk files) are unaffected — prek only runs on files changed in a given PR, so cleanup can happen incrementally.

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[jason810496]

Nice improvement, I agree that we should to add the prek hook to prevent more caplog usage in the first place.

[bugraoz93]

Nice, thanks for including these, Amogh! I think we were checking in the root beforehand

[potiuk]

Rebased to run with "full tests"

[bugraoz93]

We should use the label for this now as it fails for all and maybe for accepting some if needed

[bugraoz93]

Since we call directly pygrep maybe we need to add skip option to prek but I don't know how or even possible

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

Oh, just see the list of static checks, seems like a call for helping hands is needed!

[amoghrajesh]

Oh haha, seems I need to do that. Let me take a stab at that

[Lee-W]

I thought we're to use ast 🤔 but pygrep should be good enough

[jason810496]

Would it be better to follow the convention of

• #55416
• #67150

by using a .txt to track the current violations. Then we could defer the fix for replacing all the caplog usage (or make it as good first issue contribution opportunity for the community)

Based on the CI error logs, there're hundreds violations now, refactoring in single PR might cause even bigger test regression IMHO.

[amoghrajesh]

Yes, thanks @jason810496. I will get to this next week when I have some time.

[amoghrajesh]

Finally got back to this one.

[jason810496]

LGTM after moving the generated known usage, thanks!

[ashb]

-1. Now we are using strictlog the caplog fixture is better than mocking.

[amoghrajesh]

Closing in favour of: #69803

[amoghrajesh]

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Noticed recently in #65232 that caplog was used.

Wanted to add a prek hook to now allow this as per our coding standards: https://github.com/apache/airflow/blob/main/AGENTS.md#testing-standards.

Intent is to enforce the testing standard that tests should check logic rather than log output.

I am proposing to solve this by adding a pygrep hook (check-no-caplog-in-tests) to airflow-core, providers, and task-sdk pre commit config. The hook matches \bcaplog\b in test files and fails if any new usage is introduced.

Existing usages (35 core, 89 provider, 6 task-sdk files) are unaffected — prek only runs on files changed in a given PR, so cleanup can happen incrementally.

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[jason810496]

Nice improvement, I agree that we should to add the prek hook to prevent more caplog usage in the first place.

[bugraoz93]

Nice, thanks for including these, Amogh! I think we were checking in the root beforehand

[potiuk]

Rebased to run with "full tests"

[bugraoz93]

We should use the label for this now as it fails for all and maybe for accepting some if needed

[bugraoz93]

Since we call directly pygrep maybe we need to add skip option to prek but I don't know how or even possible

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

Oh, just see the list of static checks, seems like a call for helping hands is needed!

[amoghrajesh]

Oh haha, seems I need to do that. Let me take a stab at that

[Lee-W]

I thought we're to use ast 🤔 but pygrep should be good enough

[jason810496]

Would it be better to follow the convention of

• #55416
• #67150

by using a .txt to track the current violations. Then we could defer the fix for replacing all the caplog usage (or make it as good first issue contribution opportunity for the community)

Based on the CI error logs, there're hundreds violations now, refactoring in single PR might cause even bigger test regression IMHO.

[amoghrajesh]

Yes, thanks @jason810496. I will get to this next week when I have some time.

[amoghrajesh]

Finally got back to this one.

[jason810496]

LGTM after moving the generated known usage, thanks!

[ashb]

-1. Now we are using strictlog the caplog fixture is better than mocking.

[amoghrajesh]

Closing in favour of: #69803

[amoghrajesh]

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Noticed recently in #65232 that caplog was used.

Wanted to add a prek hook to now allow this as per our coding standards: https://github.com/apache/airflow/blob/main/AGENTS.md#testing-standards.

Intent is to enforce the testing standard that tests should check logic rather than log output.

I am proposing to solve this by adding a pygrep hook (check-no-caplog-in-tests) to airflow-core, providers, and task-sdk pre commit config. The hook matches \bcaplog\b in test files and fails if any new usage is introduced.

Existing usages (35 core, 89 provider, 6 task-sdk files) are unaffected — prek only runs on files changed in a given PR, so cleanup can happen incrementally.

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[jason810496]

Nice improvement, I agree that we should to add the prek hook to prevent more caplog usage in the first place.

[bugraoz93]

Nice, thanks for including these, Amogh! I think we were checking in the root beforehand

[potiuk]

Rebased to run with "full tests"

[bugraoz93]

We should use the label for this now as it fails for all and maybe for accepting some if needed

[bugraoz93]

Since we call directly pygrep maybe we need to add skip option to prek but I don't know how or even possible

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

Oh, just see the list of static checks, seems like a call for helping hands is needed!

[amoghrajesh]

Oh haha, seems I need to do that. Let me take a stab at that

[Lee-W]

I thought we're to use ast 🤔 but pygrep should be good enough

[jason810496]

Would it be better to follow the convention of

• #55416
• #67150

by using a .txt to track the current violations. Then we could defer the fix for replacing all the caplog usage (or make it as good first issue contribution opportunity for the community)

Based on the CI error logs, there're hundreds violations now, refactoring in single PR might cause even bigger test regression IMHO.

[amoghrajesh]

Yes, thanks @jason810496. I will get to this next week when I have some time.

[amoghrajesh]

Finally got back to this one.

[jason810496]

LGTM after moving the generated known usage, thanks!

[ashb]

-1. Now we are using strictlog the caplog fixture is better than mocking.

[amoghrajesh]

Closing in favour of: #69803

[amoghrajesh]

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Noticed recently in #65232 that caplog was used.

Wanted to add a prek hook to now allow this as per our coding standards: https://github.com/apache/airflow/blob/main/AGENTS.md#testing-standards.

Intent is to enforce the testing standard that tests should check logic rather than log output.

I am proposing to solve this by adding a pygrep hook (check-no-caplog-in-tests) to airflow-core, providers, and task-sdk pre commit config. The hook matches \bcaplog\b in test files and fails if any new usage is introduced.

Existing usages (35 core, 89 provider, 6 task-sdk files) are unaffected — prek only runs on files changed in a given PR, so cleanup can happen incrementally.

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[jason810496]

Nice improvement, I agree that we should to add the prek hook to prevent more caplog usage in the first place.

[bugraoz93]

Nice, thanks for including these, Amogh! I think we were checking in the root beforehand

[potiuk]

Rebased to run with "full tests"

[bugraoz93]

We should use the label for this now as it fails for all and maybe for accepting some if needed

[bugraoz93]

Since we call directly pygrep maybe we need to add skip option to prek but I don't know how or even possible

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

Oh, just see the list of static checks, seems like a call for helping hands is needed!

[amoghrajesh]

Oh haha, seems I need to do that. Let me take a stab at that

[Lee-W]

I thought we're to use ast 🤔 but pygrep should be good enough

[jason810496]

Would it be better to follow the convention of

• #55416
• #67150

by using a .txt to track the current violations. Then we could defer the fix for replacing all the caplog usage (or make it as good first issue contribution opportunity for the community)

Based on the CI error logs, there're hundreds violations now, refactoring in single PR might cause even bigger test regression IMHO.

[amoghrajesh]

Yes, thanks @jason810496. I will get to this next week when I have some time.

[amoghrajesh]

Finally got back to this one.

[jason810496]

LGTM after moving the generated known usage, thanks!

[ashb]

-1. Now we are using strictlog the caplog fixture is better than mocking.

[amoghrajesh]

Closing in favour of: #69803

[amoghrajesh]

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Noticed recently in #65232 that caplog was used.

Wanted to add a prek hook to now allow this as per our coding standards: https://github.com/apache/airflow/blob/main/AGENTS.md#testing-standards.

Intent is to enforce the testing standard that tests should check logic rather than log output.

I am proposing to solve this by adding a pygrep hook (check-no-caplog-in-tests) to airflow-core, providers, and task-sdk pre commit config. The hook matches \bcaplog\b in test files and fails if any new usage is introduced.

Existing usages (35 core, 89 provider, 6 task-sdk files) are unaffected — prek only runs on files changed in a given PR, so cleanup can happen incrementally.

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[jason810496]

Nice improvement, I agree that we should to add the prek hook to prevent more caplog usage in the first place.

[bugraoz93]

Nice, thanks for including these, Amogh! I think we were checking in the root beforehand

[potiuk]

Rebased to run with "full tests"

[bugraoz93]

We should use the label for this now as it fails for all and maybe for accepting some if needed

[bugraoz93]

Since we call directly pygrep maybe we need to add skip option to prek but I don't know how or even possible

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

Oh, just see the list of static checks, seems like a call for helping hands is needed!

[amoghrajesh]

Oh haha, seems I need to do that. Let me take a stab at that

[Lee-W]

I thought we're to use ast 🤔 but pygrep should be good enough

[jason810496]

Would it be better to follow the convention of

• #55416
• #67150

by using a .txt to track the current violations. Then we could defer the fix for replacing all the caplog usage (or make it as good first issue contribution opportunity for the community)

Based on the CI error logs, there're hundreds violations now, refactoring in single PR might cause even bigger test regression IMHO.

[amoghrajesh]

Yes, thanks @jason810496. I will get to this next week when I have some time.

[amoghrajesh]

Finally got back to this one.

[jason810496]

LGTM after moving the generated known usage, thanks!

[ashb]

-1. Now we are using strictlog the caplog fixture is better than mocking.

[amoghrajesh]

Closing in favour of: #69803

[amoghrajesh]

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Noticed recently in #65232 that caplog was used.

Wanted to add a prek hook to now allow this as per our coding standards: https://github.com/apache/airflow/blob/main/AGENTS.md#testing-standards.

Intent is to enforce the testing standard that tests should check logic rather than log output.

I am proposing to solve this by adding a pygrep hook (check-no-caplog-in-tests) to airflow-core, providers, and task-sdk pre commit config. The hook matches \bcaplog\b in test files and fails if any new usage is introduced.

Existing usages (35 core, 89 provider, 6 task-sdk files) are unaffected — prek only runs on files changed in a given PR, so cleanup can happen incrementally.

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[jason810496]

Nice improvement, I agree that we should to add the prek hook to prevent more caplog usage in the first place.

[bugraoz93]

Nice, thanks for including these, Amogh! I think we were checking in the root beforehand

[potiuk]

Rebased to run with "full tests"

[bugraoz93]

We should use the label for this now as it fails for all and maybe for accepting some if needed

[bugraoz93]

Since we call directly pygrep maybe we need to add skip option to prek but I don't know how or even possible

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

Oh, just see the list of static checks, seems like a call for helping hands is needed!

[amoghrajesh]

Oh haha, seems I need to do that. Let me take a stab at that

[Lee-W]

I thought we're to use ast 🤔 but pygrep should be good enough

[jason810496]

Would it be better to follow the convention of

• #55416
• #67150

by using a .txt to track the current violations. Then we could defer the fix for replacing all the caplog usage (or make it as good first issue contribution opportunity for the community)

Based on the CI error logs, there're hundreds violations now, refactoring in single PR might cause even bigger test regression IMHO.

[amoghrajesh]

Yes, thanks @jason810496. I will get to this next week when I have some time.

[amoghrajesh]

Finally got back to this one.

[jason810496]

LGTM after moving the generated known usage, thanks!

[ashb]

-1. Now we are using strictlog the caplog fixture is better than mocking.

[amoghrajesh]

Closing in favour of: #69803

[amoghrajesh]

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Noticed recently in #65232 that caplog was used.

Wanted to add a prek hook to now allow this as per our coding standards: https://github.com/apache/airflow/blob/main/AGENTS.md#testing-standards.

Intent is to enforce the testing standard that tests should check logic rather than log output.

I am proposing to solve this by adding a pygrep hook (check-no-caplog-in-tests) to airflow-core, providers, and task-sdk pre commit config. The hook matches \bcaplog\b in test files and fails if any new usage is introduced.

Existing usages (35 core, 89 provider, 6 task-sdk files) are unaffected — prek only runs on files changed in a given PR, so cleanup can happen incrementally.

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[jason810496]

Nice improvement, I agree that we should to add the prek hook to prevent more caplog usage in the first place.

[bugraoz93]

Nice, thanks for including these, Amogh! I think we were checking in the root beforehand

[potiuk]

Rebased to run with "full tests"

[bugraoz93]

We should use the label for this now as it fails for all and maybe for accepting some if needed

[bugraoz93]

Since we call directly pygrep maybe we need to add skip option to prek but I don't know how or even possible

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

[jscheffl]

Cool!

But probably you need to add an exclusion list for all cases still in the repo or are we (still) clean?

Oh, just see the list of static checks, seems like a call for helping hands is needed!

[amoghrajesh]

Oh haha, seems I need to do that. Let me take a stab at that

[Lee-W]

I thought we're to use ast 🤔 but pygrep should be good enough

[jason810496]

Would it be better to follow the convention of

• #55416
• #67150

by using a .txt to track the current violations. Then we could defer the fix for replacing all the caplog usage (or make it as good first issue contribution opportunity for the community)

Based on the CI error logs, there're hundreds violations now, refactoring in single PR might cause even bigger test regression IMHO.

[amoghrajesh]

Yes, thanks @jason810496. I will get to this next week when I have some time.

[amoghrajesh]

Finally got back to this one.

[jason810496]

LGTM after moving the generated known usage, thanks!

[ashb]

-1. Now we are using strictlog the caplog fixture is better than mocking.

[amoghrajesh]

Closing in favour of: #69803