Fix: DAGs are not marked as stale if the dags folder change by utkarsharma2 · Pull Request #41433 · apache/airflow

utkarsharma2 · 2024-08-13T13:59:01Z

Closes: #41432

…hanges

uranusjr · 2024-08-22T03:18:26Z

Is a test possible?

utkarsharma2 · 2024-08-22T05:14:49Z

Is a test possible?

Yes, I'll add them.

utkarsharma2 · 2024-08-26T12:34:01Z

Is a test possible?

@uranusjr, @Lee-W I have added a test, PTAL.

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

ephraimbuddy · 2024-08-28T09:36:21Z

@utkarsharma2 , can you backport this to v2-10-test

utkarsharma2 · 2024-08-28T10:03:46Z

Thanks @ephraimbuddy I'll do that.

…1433) * Fix: DAGs are not marked as stale if the AIRFLOW__CORE__DAGS_FOLDER changes * Update airflow/dag_processing/manager.py * Add testcase * Add code comment * Update code comment * Update the logic for checking the current dag_directory * Update testcases * Remove unwanted code * Uncomment code * Add processor_subdir when creating processor_subdir * Fix test_retry_still_in_executor test * Remove config from test * Update airflow/dag_processing/manager.py Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> * Update if condition for readability --------- Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> (cherry picked from commit 9f30a41)

utkarsharma2 · 2024-08-28T11:15:43Z

Cherry picked - #41829

…41829) * Fix: DAGs are not marked as stale if the AIRFLOW__CORE__DAGS_FOLDER changes * Update airflow/dag_processing/manager.py * Add testcase * Add code comment * Update code comment * Update the logic for checking the current dag_directory * Update testcases * Remove unwanted code * Uncomment code * Add processor_subdir when creating processor_subdir * Fix test_retry_still_in_executor test * Remove config from test * Update airflow/dag_processing/manager.py Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> * Update if condition for readability --------- Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> (cherry picked from commit 9f30a41)

…pache#41433)" This reverts commit 9f30a41.

…41829) * Fix: DAGs are not marked as stale if the AIRFLOW__CORE__DAGS_FOLDER changes * Update airflow/dag_processing/manager.py * Add testcase * Add code comment * Update code comment * Update the logic for checking the current dag_directory * Update testcases * Remove unwanted code * Uncomment code * Add processor_subdir when creating processor_subdir * Fix test_retry_still_in_executor test * Remove config from test * Update airflow/dag_processing/manager.py Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> * Update if condition for readability --------- Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> (cherry picked from commit 9f30a41)

…pache#41433)" This reverts commit 9f30a41.

…41433)" (#42197) This reverts commit 9f30a41.

…pache#41433) (apache#41829)" This reverts commit 996af78.

…(41433) (#41829) (#41893) 41829 (#42220) * Revert "Skip test_scan_stale_dags_when_dag_folder_change in DB isolation mode (#41893)" This reverts commit 07af14a. * Revert "Fix: DAGs are not marked as stale if the dags folder change (#41433) (#41829)" This reverts commit 996af78.

…pache#41433)" (apache#42197) This reverts commit 9f30a41.

Fix: DAGs are not marked as stale if the AIRFLOW__CORE__DAGS_FOLDER c…
74ce043
…hanges

utkarsharma2 requested review from ephraimbuddy and jedcunningham as code owners August 13, 2024 13:59

boring-cyborgBot added the area:Scheduler including HA (high availability) scheduler label Aug 13, 2024

Merge branch 'main' into FixDagParsing
b881678

utkarsharma2 commented Aug 13, 2024
View reviewed changes

Comment threadairflow/dag_processing/manager.py Outdated

utkarsharma2 added 3 commits August 13, 2024 19:31

Update airflow/dag_processing/manager.py
a41d411

Merge branch 'main' into FixDagParsing
00f2b0a

Merge branch 'main' into FixDagParsing
40c6cb0

Lee-W reviewed Aug 21, 2024
View reviewed changes

Comment threadairflow/dag_processing/manager.py Outdated

utkarsharma2 added 3 commits August 26, 2024 18:01

Add testcase
5e11d81

Add code comment
252e473

Update code comment
df185ba

utkarsharma2 requested review from Lee-W and uranusjr August 26, 2024 12:34

utkarsharma2 added 2 commits August 26, 2024 18:04

Merge branch 'main' into FixDagParsing
9898b39

Update the logic for checking the current dag_directory
75c9d93

ephraimbuddy reviewed Aug 26, 2024
View reviewed changes

Comment threadtests/dag_processing/test_job_runner.py Outdated
Comment threadtests/dag_processing/test_job_runner.py Outdated
Comment threadtests/dag_processing/test_job_runner.py Outdated

utkarsharma2 added 4 commits August 26, 2024 20:24

Update testcases
4b745b8

Remove unwanted code
79cd22e

Merge branch 'main' into FixDagParsing
da5ae39

Uncomment code
055192a

utkarsharma2 force-pushed the FixDagParsing branch from bb61051 to 055192aCompare August 26, 2024 15:08

Merge branch 'main' into FixDagParsing
245bef8

utkarsharma2 requested a review from ephraimbuddy August 26, 2024 15:10

ephraimbuddy reviewed Aug 26, 2024
View reviewed changes

Comment threadtests/dag_processing/test_job_runner.py

utkarsharma2 requested a review from ephraimbuddy August 26, 2024 16:15

ephraimbuddy approved these changes Aug 27, 2024
View reviewed changes

jedcunningham reviewed Aug 28, 2024
View reviewed changes

Comment threadairflow/dag_processing/manager.py Outdated
Comment threadairflow/dag_processing/manager.py Outdated

utkarsharma2and others added 3 commits August 28, 2024 11:54

Remove config from test
3545a7a

Update airflow/dag_processing/manager.py
3917819
Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

Merge branch 'main' into FixDagParsing
da42bab

utkarsharma2 requested a review from jedcunningham August 28, 2024 06:36

Update if condition for readability
fa17bc6

utkarsharma2 merged commit 9f30a41 into apache:mainAug 28, 2024

utkarsharma2 deleted the FixDagParsing branch August 28, 2024 09:07

ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Aug 29, 2024
Revert "Fix: DAGs are not marked as stale if the dags folder change (a…
60c2fa2
…pache#41433)" This reverts commit 9f30a41.

utkarsharma2 added the type:bug-fix Changelog: Bug Fixes label Aug 30, 2024

utkarsharma2 added this to the Airflow 2.10.1 milestone Aug 30, 2024

jcrobak mentioned this pull request Sep 9, 2024
DAGs are missing and deleted in Airflow 2.10.1 #42111
Closed
2 tasks

utkarsharma2 added a commit to astronomer/airflow that referenced this pull request Sep 12, 2024
Revert "Fix: DAGs are not marked as stale if the dags folder change (a…
3d432dd
…pache#41433)" This reverts commit 9f30a41.

utkarsharma2 mentioned this pull request Sep 12, 2024
Revert "Fix: DAGs are not marked as stale if the dags folder change #42197
Merged

ephraimbuddy pushed a commit that referenced this pull request Sep 12, 2024
Revert "Fix: DAGs are not marked as stale if the dags folder change (#…
3bc2d3a
…41433)" (#42197) This reverts commit 9f30a41.

ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Sep 13, 2024
Revert "Fix: DAGs are not marked as stale if the dags folder change (a…
2bda0a2
…pache#41433) (apache#41829)" This reverts commit 996af78.

joaopamaral pushed a commit to joaopamaral/airflow that referenced this pull request Oct 21, 2024
Revert "Fix: DAGs are not marked as stale if the dags folder change (a…
c942342
…pache#41433)" (apache#42197) This reverts commit 9f30a41.

jedcunningham mentioned this pull request Feb 18, 2025
[v2-10-test] Don't resolve path for DAGs folder #46877
Merged

kaxil mentioned this pull request May 15, 2025
Status of testing of Apache Airflow 2.11.0rc1 #50672
Closed
39 tasks

Navigation Menu

Uh oh!

Fix: DAGs are not marked as stale if the dags folder change - #41433

Fix: DAGs are not marked as stale if the dags folder change#41433
utkarsharma2 merged 23 commits into
apache:mainfrom
astronomer:FixDagParsing

utkarsharma2 commented Aug 13, 2024•
edited
Loading

Uh oh!

Uh oh!

Uh oh!

uranusjr commented Aug 22, 2024

Uh oh!

utkarsharma2 commented Aug 22, 2024

Uh oh!

utkarsharma2 commented Aug 26, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ephraimbuddy commented Aug 28, 2024

Uh oh!

utkarsharma2 commented Aug 28, 2024

Uh oh!

utkarsharma2 commented Aug 28, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

utkarsharma2 commented Aug 13, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

uranusjr commented Aug 22, 2024

Uh oh!

utkarsharma2 commented Aug 22, 2024

Uh oh!

utkarsharma2 commented Aug 26, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ephraimbuddy commented Aug 28, 2024

Uh oh!

utkarsharma2 commented Aug 28, 2024

Uh oh!

utkarsharma2 commented Aug 28, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

utkarsharma2 commented Aug 13, 2024•
edited
Loading