Use separate method to decide if task is to be scheduled by kacpermuda · Pull Request #56039 · apache/airflow

kacpermuda · 2025-09-24T10:06:04Z

The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method.

If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

kacpermuda · 2025-10-28T12:43:31Z

@ashb @uranusjr would you guys mind taking another look at this?

…straight as success (apache#56039) The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method. If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

kacpermuda · 2025-09-24T10:06:04Z

The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method.

If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

kacpermuda · 2025-10-28T12:43:31Z

@ashb @uranusjr would you guys mind taking another look at this?

…straight as success (apache#56039) The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method. If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

kacpermuda · 2025-09-24T10:06:04Z

The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method.

If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

kacpermuda · 2025-10-28T12:43:31Z

@ashb @uranusjr would you guys mind taking another look at this?

…straight as success (apache#56039) The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method. If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

kacpermuda · 2025-09-24T10:06:04Z

The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method.

If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

kacpermuda · 2025-10-28T12:43:31Z

@ashb @uranusjr would you guys mind taking another look at this?

…straight as success (apache#56039) The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method. If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

kacpermuda · 2025-09-24T10:06:04Z

The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method.

If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

kacpermuda · 2025-10-28T12:43:31Z

@ashb @uranusjr would you guys mind taking another look at this?

…straight as success (apache#56039) The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method. If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

kacpermuda · 2025-09-24T10:06:04Z

The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method.

If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

kacpermuda · 2025-10-28T12:43:31Z

@ashb @uranusjr would you guys mind taking another look at this?

…straight as success (apache#56039) The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method. If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

kacpermuda · 2025-09-24T10:06:04Z

The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method.

If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

kacpermuda · 2025-10-28T12:43:31Z

@ashb @uranusjr would you guys mind taking another look at this?

…straight as success (apache#56039) The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method. If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

kacpermuda · 2025-09-24T10:06:04Z

The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method.

If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

kacpermuda · 2025-10-28T12:43:31Z

@ashb @uranusjr would you guys mind taking another look at this?

…straight as success (apache#56039) The check whether or not to schedule a task is now done within schedule_tis method, I think it would be beneficial to extract that logic into a separate public method. If task is not being scheduled, the listeners are not called for it - and some listeners would like to know that ahead of time, which would be easier if there was a separate method for that decision. Right now the OL listener was re-creating the logic from the Airflow core, but it became complex and hard to keep up with. I'll have another PR for the OpenLineage provider to check for existence of this new is_schedulable_task method, and call it instead.

kacpermuda requested review from XD-DENG and ashb as code owners September 24, 2025 10:06

ashb reviewed Sep 24, 2025
View reviewed changes

Comment threadairflow-core/src/airflow/models/dagrun.py Outdated

kacpermuda mentioned this pull request Sep 24, 2025
fix: Adjust OL utils for AF3.1 changes #56040
Merged

ashb changed the title ~~nit: Use separate method to decide if task is to be scheduled~~Use separate method to decide if task is to be scheduledOct 21, 2025

feat: Use separate method to decide if task is to be scheduled
a9d76ea

kacpermuda force-pushed the nit-is-schedulable-task branch from 7690a51 to a9d76eaCompare October 21, 2025 14:31

ashb approved these changes Oct 28, 2025
View reviewed changes

ashb merged commit e506470 into apache:mainOct 28, 2025
112 checks passed

kacpermuda deleted the nit-is-schedulable-task branch October 28, 2025 13:58

kacpermuda mentioned this pull request Oct 28, 2025
nit: Use new taskinstance method to determine if task will emit OL event #57446
Merged

potiuk mentioned this pull request Nov 14, 2025
Status of testing Providers that were prepared on November 14, 2025 #58315
Closed