Skip to content

Fail dag test if defer without triggerer - #34619

Merged
dstandish merged 9 commits into
apache:mainfrom
astronomer:fail-dag-test-if-defer-without-triggeer
Sep 27, 2023
Merged

Fail dag test if defer without triggerer#34619
dstandish merged 9 commits into
apache:mainfrom
astronomer:fail-dag-test-if-defer-without-triggeer

Conversation

@dstandish

Copy link
Copy Markdown
Contributor

If user runs dag.test and task defers and no triggerer is running, we should fail so user does not sit there waiting forever.

@dstandishdstandish changed the title Fail dag test if defer without triggeerFail dag test if defer without triggererSep 26, 2023
Comment threadairflow/exceptions.py Outdated
Comment threadairflow/models/dag.py Outdated
Comment threadairflow/models/dag.py Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make sure. we do have a plan to integrate triggerer functionality into test directly right? Because this would only be good as a temporary measure; if we want to keep this function long term I’d want some refactoring instead of importing api from models.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But in either case this really should call TriggererJobRunner.most_recent_job() directly instead of going through the additional abstraction in api.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make sure. we do have a plan to integrate triggerer functionality into test directly right? Because this would only be good as a temporary measure; if we want to keep this function long term I’d want some refactoring instead of importing api from models

Plan might be overstating it. It's an idea. But, there's no guarantee it happens. I'm simply being pragmatic, doing the easy thing to make user's life nicer, by failing when otherwise they might just wait forever.

But in either case this really should call TriggererJobRunner.most_recent_job() directly instead of going through the additional abstraction in api.

Yeah that sounds good. I wasn't sure exactly what was best.

@dstandish
dstandishforce-pushed the fail-dag-test-if-defer-without-triggeer branch from 5dad289 to ef577e7CompareSeptember 27, 2023 06:12
@dstandish

Copy link
Copy Markdown
ContributorAuthor

alright, sir, PTALWYGAC

Comment threadairflow/models/dag.py
@dstandish
dstandish merged commit e81bb48 into apache:mainSep 27, 2023
@dstandish
dstandish deleted the fail-dag-test-if-defer-without-triggeer branch September 27, 2023 16:25
@eladkaleladkal added this to the Airflow 2.7.2 milestone Sep 29, 2023
@ephraimbuddyephraimbuddy added the type:misc/internal Changelog: Misc changes that should appear in change log label Oct 3, 2023
ephraimbuddy pushed a commit that referenced this pull request Oct 5, 2023
If user runs dag.test and task defers and no triggerer is running, we should fail so user does not sit there waiting forever.
---------
Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
(cherry picked from commit e81bb48)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:CLItype:misc/internalChangelog: Misc changes that should appear in change log

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@dstandish@uranusjr@ephraimbuddy@eladkal