Fix grid view topological sort for cross-group dependencies by alex-stiff · Pull Request #67964 · apache/airflow

alex-stiff · 2026-06-03T14:40:37Z

When specifying group dependencies like first_stage.end >> second_stage.start, the ordering within the Airflow UI does not respect the default topological sorting.

Minimal DAG to reproduce:

fromdatetimeimportdatetimefromairflow.sdkimportDAG, task, TaskGroup@taskdefnoop_task():
passwithDAG("grid_sort_bug_repro", start_date=datetime(2024, 1, 1), schedule=None, catchup=False):
withTaskGroup("stage_b"):
b_start=noop_task()
b_end=noop_task()
b_start>>b_endwithTaskGroup("stage_a"):
a_start=noop_task()
a_end=noop_task()
a_start>>a_endb_end>>a_start# task-level dep, bypasses group-level relationship

This shows in the grid view UI as:

stage_a
stage_b

Which is alphabetical, rather than the desired stage_b coming before stage_a.

This is happening because the _project_child_deps function only checks child.upstream_task_ids populated explicitly. We need to also scan the upstream_task_ids of the TaskGroup's root tasks so that the tasks within the group are also visible to this function.

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Important

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-06-17 14:51 UTC

Helpful heads-up from the maintainers — please address before this PR can be reviewed:

Static / docs checks failing (CI image checks / Static checks). Run them locally with prek run --all-files (or pre-commit run --all-files) and push the fixes.
Failing test jobs: Low dep tests:core / All-core:LowestDeps:14:3.10:Always...Core, MySQL tests: core / DB-core:MySQL:8.0:3.10:Always...Core, Postgres tests: core / DB-core:Postgres:14:3.10:Always...Core, Sqlite tests: core / DB-core:Sqlite:3.10:Always...Core. Reproduce and fix locally, then push.
See the Pull Request quality criteria.

The ball is in your court — you've been assigned to this PR. Fix the above, then mark it Ready for review.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

boring-cyborg · 2026-06-03T14:40:43Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

ashb

Please add unit tests covering it.

alex-stiff · 2026-06-18T12:34:45Z

Please add unit tests covering it.

Thank you, added.

potiuk · 2026-07-08T16:05:50Z

Note

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-07-08 16:05 UTC

This draft has been inactive for ~20 days, so I'm closing it to keep the review queue tidy — no judgment on the work itself.

No rush — reopen it (or open a fresh PR) whenever you're ready to continue; nothing is lost.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

alex-stiff · 2026-07-08T16:14:26Z

@potiuk would you mind reopening this for me please, I don't seem to have perms! Thank you.

…deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite).

…dependencies (#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

…dependencies (#69933) (#70591) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com> (cherry picked from commit d7aa929) # Conflicts: # airflow-core/src/airflow/api_fastapi/core_api/services/ui/task_group.py Co-authored-by: Hemkumar Chheda <95332229+hkc-8010@users.noreply.github.com>

…dependencies (apache#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

alex-stiff · 2026-06-03T14:40:37Z

When specifying group dependencies like first_stage.end >> second_stage.start, the ordering within the Airflow UI does not respect the default topological sorting.

Minimal DAG to reproduce:

fromdatetimeimportdatetimefromairflow.sdkimportDAG, task, TaskGroup@taskdefnoop_task():
passwithDAG("grid_sort_bug_repro", start_date=datetime(2024, 1, 1), schedule=None, catchup=False):
withTaskGroup("stage_b"):
b_start=noop_task()
b_end=noop_task()
b_start>>b_endwithTaskGroup("stage_a"):
a_start=noop_task()
a_end=noop_task()
a_start>>a_endb_end>>a_start# task-level dep, bypasses group-level relationship

This shows in the grid view UI as:

stage_a
stage_b

Which is alphabetical, rather than the desired stage_b coming before stage_a.

This is happening because the _project_child_deps function only checks child.upstream_task_ids populated explicitly. We need to also scan the upstream_task_ids of the TaskGroup's root tasks so that the tasks within the group are also visible to this function.

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Important

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-06-17 14:51 UTC

Helpful heads-up from the maintainers — please address before this PR can be reviewed:

Static / docs checks failing (CI image checks / Static checks). Run them locally with prek run --all-files (or pre-commit run --all-files) and push the fixes.
Failing test jobs: Low dep tests:core / All-core:LowestDeps:14:3.10:Always...Core, MySQL tests: core / DB-core:MySQL:8.0:3.10:Always...Core, Postgres tests: core / DB-core:Postgres:14:3.10:Always...Core, Sqlite tests: core / DB-core:Sqlite:3.10:Always...Core. Reproduce and fix locally, then push.
See the Pull Request quality criteria.

The ball is in your court — you've been assigned to this PR. Fix the above, then mark it Ready for review.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

boring-cyborg · 2026-06-03T14:40:43Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

ashb

Please add unit tests covering it.

alex-stiff · 2026-06-18T12:34:45Z

Please add unit tests covering it.

Thank you, added.

potiuk · 2026-07-08T16:05:50Z

Note

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-07-08 16:05 UTC

This draft has been inactive for ~20 days, so I'm closing it to keep the review queue tidy — no judgment on the work itself.

No rush — reopen it (or open a fresh PR) whenever you're ready to continue; nothing is lost.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

alex-stiff · 2026-07-08T16:14:26Z

@potiuk would you mind reopening this for me please, I don't seem to have perms! Thank you.

…deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite).

…dependencies (#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

…dependencies (#69933) (#70591) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com> (cherry picked from commit d7aa929) # Conflicts: # airflow-core/src/airflow/api_fastapi/core_api/services/ui/task_group.py Co-authored-by: Hemkumar Chheda <95332229+hkc-8010@users.noreply.github.com>

…dependencies (apache#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

alex-stiff · 2026-06-03T14:40:37Z

When specifying group dependencies like first_stage.end >> second_stage.start, the ordering within the Airflow UI does not respect the default topological sorting.

Minimal DAG to reproduce:

fromdatetimeimportdatetimefromairflow.sdkimportDAG, task, TaskGroup@taskdefnoop_task():
passwithDAG("grid_sort_bug_repro", start_date=datetime(2024, 1, 1), schedule=None, catchup=False):
withTaskGroup("stage_b"):
b_start=noop_task()
b_end=noop_task()
b_start>>b_endwithTaskGroup("stage_a"):
a_start=noop_task()
a_end=noop_task()
a_start>>a_endb_end>>a_start# task-level dep, bypasses group-level relationship

This shows in the grid view UI as:

stage_a
stage_b

Which is alphabetical, rather than the desired stage_b coming before stage_a.

This is happening because the _project_child_deps function only checks child.upstream_task_ids populated explicitly. We need to also scan the upstream_task_ids of the TaskGroup's root tasks so that the tasks within the group are also visible to this function.

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Important

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-06-17 14:51 UTC

Helpful heads-up from the maintainers — please address before this PR can be reviewed:

Static / docs checks failing (CI image checks / Static checks). Run them locally with prek run --all-files (or pre-commit run --all-files) and push the fixes.
Failing test jobs: Low dep tests:core / All-core:LowestDeps:14:3.10:Always...Core, MySQL tests: core / DB-core:MySQL:8.0:3.10:Always...Core, Postgres tests: core / DB-core:Postgres:14:3.10:Always...Core, Sqlite tests: core / DB-core:Sqlite:3.10:Always...Core. Reproduce and fix locally, then push.
See the Pull Request quality criteria.

The ball is in your court — you've been assigned to this PR. Fix the above, then mark it Ready for review.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

boring-cyborg · 2026-06-03T14:40:43Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

ashb

Please add unit tests covering it.

alex-stiff · 2026-06-18T12:34:45Z

Please add unit tests covering it.

Thank you, added.

potiuk · 2026-07-08T16:05:50Z

Note

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-07-08 16:05 UTC

This draft has been inactive for ~20 days, so I'm closing it to keep the review queue tidy — no judgment on the work itself.

No rush — reopen it (or open a fresh PR) whenever you're ready to continue; nothing is lost.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

alex-stiff · 2026-07-08T16:14:26Z

@potiuk would you mind reopening this for me please, I don't seem to have perms! Thank you.

…deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite).

…dependencies (#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

…dependencies (#69933) (#70591) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com> (cherry picked from commit d7aa929) # Conflicts: # airflow-core/src/airflow/api_fastapi/core_api/services/ui/task_group.py Co-authored-by: Hemkumar Chheda <95332229+hkc-8010@users.noreply.github.com>

…dependencies (apache#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

alex-stiff · 2026-06-03T14:40:37Z

When specifying group dependencies like first_stage.end >> second_stage.start, the ordering within the Airflow UI does not respect the default topological sorting.

Minimal DAG to reproduce:

fromdatetimeimportdatetimefromairflow.sdkimportDAG, task, TaskGroup@taskdefnoop_task():
passwithDAG("grid_sort_bug_repro", start_date=datetime(2024, 1, 1), schedule=None, catchup=False):
withTaskGroup("stage_b"):
b_start=noop_task()
b_end=noop_task()
b_start>>b_endwithTaskGroup("stage_a"):
a_start=noop_task()
a_end=noop_task()
a_start>>a_endb_end>>a_start# task-level dep, bypasses group-level relationship

This shows in the grid view UI as:

stage_a
stage_b

Which is alphabetical, rather than the desired stage_b coming before stage_a.

This is happening because the _project_child_deps function only checks child.upstream_task_ids populated explicitly. We need to also scan the upstream_task_ids of the TaskGroup's root tasks so that the tasks within the group are also visible to this function.

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Important

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-06-17 14:51 UTC

Helpful heads-up from the maintainers — please address before this PR can be reviewed:

Static / docs checks failing (CI image checks / Static checks). Run them locally with prek run --all-files (or pre-commit run --all-files) and push the fixes.
Failing test jobs: Low dep tests:core / All-core:LowestDeps:14:3.10:Always...Core, MySQL tests: core / DB-core:MySQL:8.0:3.10:Always...Core, Postgres tests: core / DB-core:Postgres:14:3.10:Always...Core, Sqlite tests: core / DB-core:Sqlite:3.10:Always...Core. Reproduce and fix locally, then push.
See the Pull Request quality criteria.

The ball is in your court — you've been assigned to this PR. Fix the above, then mark it Ready for review.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

boring-cyborg · 2026-06-03T14:40:43Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

ashb

Please add unit tests covering it.

alex-stiff · 2026-06-18T12:34:45Z

Please add unit tests covering it.

Thank you, added.

potiuk · 2026-07-08T16:05:50Z

Note

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-07-08 16:05 UTC

This draft has been inactive for ~20 days, so I'm closing it to keep the review queue tidy — no judgment on the work itself.

No rush — reopen it (or open a fresh PR) whenever you're ready to continue; nothing is lost.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

alex-stiff · 2026-07-08T16:14:26Z

@potiuk would you mind reopening this for me please, I don't seem to have perms! Thank you.

…deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite).

…dependencies (#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

…dependencies (#69933) (#70591) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com> (cherry picked from commit d7aa929) # Conflicts: # airflow-core/src/airflow/api_fastapi/core_api/services/ui/task_group.py Co-authored-by: Hemkumar Chheda <95332229+hkc-8010@users.noreply.github.com>

…dependencies (apache#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

alex-stiff · 2026-06-03T14:40:37Z

When specifying group dependencies like first_stage.end >> second_stage.start, the ordering within the Airflow UI does not respect the default topological sorting.

Minimal DAG to reproduce:

fromdatetimeimportdatetimefromairflow.sdkimportDAG, task, TaskGroup@taskdefnoop_task():
passwithDAG("grid_sort_bug_repro", start_date=datetime(2024, 1, 1), schedule=None, catchup=False):
withTaskGroup("stage_b"):
b_start=noop_task()
b_end=noop_task()
b_start>>b_endwithTaskGroup("stage_a"):
a_start=noop_task()
a_end=noop_task()
a_start>>a_endb_end>>a_start# task-level dep, bypasses group-level relationship

This shows in the grid view UI as:

stage_a
stage_b

Which is alphabetical, rather than the desired stage_b coming before stage_a.

This is happening because the _project_child_deps function only checks child.upstream_task_ids populated explicitly. We need to also scan the upstream_task_ids of the TaskGroup's root tasks so that the tasks within the group are also visible to this function.

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Important

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-06-17 14:51 UTC

Helpful heads-up from the maintainers — please address before this PR can be reviewed:

Static / docs checks failing (CI image checks / Static checks). Run them locally with prek run --all-files (or pre-commit run --all-files) and push the fixes.
Failing test jobs: Low dep tests:core / All-core:LowestDeps:14:3.10:Always...Core, MySQL tests: core / DB-core:MySQL:8.0:3.10:Always...Core, Postgres tests: core / DB-core:Postgres:14:3.10:Always...Core, Sqlite tests: core / DB-core:Sqlite:3.10:Always...Core. Reproduce and fix locally, then push.
See the Pull Request quality criteria.

The ball is in your court — you've been assigned to this PR. Fix the above, then mark it Ready for review.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

boring-cyborg · 2026-06-03T14:40:43Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

ashb

Please add unit tests covering it.

alex-stiff · 2026-06-18T12:34:45Z

Please add unit tests covering it.

Thank you, added.

potiuk · 2026-07-08T16:05:50Z

Note

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-07-08 16:05 UTC

This draft has been inactive for ~20 days, so I'm closing it to keep the review queue tidy — no judgment on the work itself.

No rush — reopen it (or open a fresh PR) whenever you're ready to continue; nothing is lost.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

alex-stiff · 2026-07-08T16:14:26Z

@potiuk would you mind reopening this for me please, I don't seem to have perms! Thank you.

…deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite).

…dependencies (#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

…dependencies (#69933) (#70591) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com> (cherry picked from commit d7aa929) # Conflicts: # airflow-core/src/airflow/api_fastapi/core_api/services/ui/task_group.py Co-authored-by: Hemkumar Chheda <95332229+hkc-8010@users.noreply.github.com>

…dependencies (apache#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

alex-stiff · 2026-06-03T14:40:37Z

When specifying group dependencies like first_stage.end >> second_stage.start, the ordering within the Airflow UI does not respect the default topological sorting.

Minimal DAG to reproduce:

fromdatetimeimportdatetimefromairflow.sdkimportDAG, task, TaskGroup@taskdefnoop_task():
passwithDAG("grid_sort_bug_repro", start_date=datetime(2024, 1, 1), schedule=None, catchup=False):
withTaskGroup("stage_b"):
b_start=noop_task()
b_end=noop_task()
b_start>>b_endwithTaskGroup("stage_a"):
a_start=noop_task()
a_end=noop_task()
a_start>>a_endb_end>>a_start# task-level dep, bypasses group-level relationship

This shows in the grid view UI as:

stage_a
stage_b

Which is alphabetical, rather than the desired stage_b coming before stage_a.

This is happening because the _project_child_deps function only checks child.upstream_task_ids populated explicitly. We need to also scan the upstream_task_ids of the TaskGroup's root tasks so that the tasks within the group are also visible to this function.

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Important

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-06-17 14:51 UTC

Helpful heads-up from the maintainers — please address before this PR can be reviewed:

Static / docs checks failing (CI image checks / Static checks). Run them locally with prek run --all-files (or pre-commit run --all-files) and push the fixes.
Failing test jobs: Low dep tests:core / All-core:LowestDeps:14:3.10:Always...Core, MySQL tests: core / DB-core:MySQL:8.0:3.10:Always...Core, Postgres tests: core / DB-core:Postgres:14:3.10:Always...Core, Sqlite tests: core / DB-core:Sqlite:3.10:Always...Core. Reproduce and fix locally, then push.
See the Pull Request quality criteria.

The ball is in your court — you've been assigned to this PR. Fix the above, then mark it Ready for review.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

boring-cyborg · 2026-06-03T14:40:43Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

ashb

Please add unit tests covering it.

alex-stiff · 2026-06-18T12:34:45Z

Please add unit tests covering it.

Thank you, added.

potiuk · 2026-07-08T16:05:50Z

Note

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-07-08 16:05 UTC

This draft has been inactive for ~20 days, so I'm closing it to keep the review queue tidy — no judgment on the work itself.

No rush — reopen it (or open a fresh PR) whenever you're ready to continue; nothing is lost.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

alex-stiff · 2026-07-08T16:14:26Z

@potiuk would you mind reopening this for me please, I don't seem to have perms! Thank you.

…deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite).

…dependencies (#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

…dependencies (#69933) (#70591) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com> (cherry picked from commit d7aa929) # Conflicts: # airflow-core/src/airflow/api_fastapi/core_api/services/ui/task_group.py Co-authored-by: Hemkumar Chheda <95332229+hkc-8010@users.noreply.github.com>

…dependencies (apache#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

alex-stiff · 2026-06-03T14:40:37Z

When specifying group dependencies like first_stage.end >> second_stage.start, the ordering within the Airflow UI does not respect the default topological sorting.

Minimal DAG to reproduce:

fromdatetimeimportdatetimefromairflow.sdkimportDAG, task, TaskGroup@taskdefnoop_task():
passwithDAG("grid_sort_bug_repro", start_date=datetime(2024, 1, 1), schedule=None, catchup=False):
withTaskGroup("stage_b"):
b_start=noop_task()
b_end=noop_task()
b_start>>b_endwithTaskGroup("stage_a"):
a_start=noop_task()
a_end=noop_task()
a_start>>a_endb_end>>a_start# task-level dep, bypasses group-level relationship

This shows in the grid view UI as:

stage_a
stage_b

Which is alphabetical, rather than the desired stage_b coming before stage_a.

This is happening because the _project_child_deps function only checks child.upstream_task_ids populated explicitly. We need to also scan the upstream_task_ids of the TaskGroup's root tasks so that the tasks within the group are also visible to this function.

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Important

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-06-17 14:51 UTC

Helpful heads-up from the maintainers — please address before this PR can be reviewed:

Static / docs checks failing (CI image checks / Static checks). Run them locally with prek run --all-files (or pre-commit run --all-files) and push the fixes.
Failing test jobs: Low dep tests:core / All-core:LowestDeps:14:3.10:Always...Core, MySQL tests: core / DB-core:MySQL:8.0:3.10:Always...Core, Postgres tests: core / DB-core:Postgres:14:3.10:Always...Core, Sqlite tests: core / DB-core:Sqlite:3.10:Always...Core. Reproduce and fix locally, then push.
See the Pull Request quality criteria.

The ball is in your court — you've been assigned to this PR. Fix the above, then mark it Ready for review.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

boring-cyborg · 2026-06-03T14:40:43Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

ashb

Please add unit tests covering it.

alex-stiff · 2026-06-18T12:34:45Z

Please add unit tests covering it.

Thank you, added.

potiuk · 2026-07-08T16:05:50Z

Note

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-07-08 16:05 UTC

This draft has been inactive for ~20 days, so I'm closing it to keep the review queue tidy — no judgment on the work itself.

No rush — reopen it (or open a fresh PR) whenever you're ready to continue; nothing is lost.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

alex-stiff · 2026-07-08T16:14:26Z

@potiuk would you mind reopening this for me please, I don't seem to have perms! Thank you.

…deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite).

…dependencies (#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

…dependencies (#69933) (#70591) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com> (cherry picked from commit d7aa929) # Conflicts: # airflow-core/src/airflow/api_fastapi/core_api/services/ui/task_group.py Co-authored-by: Hemkumar Chheda <95332229+hkc-8010@users.noreply.github.com>

…dependencies (apache#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

alex-stiff · 2026-06-03T14:40:37Z

When specifying group dependencies like first_stage.end >> second_stage.start, the ordering within the Airflow UI does not respect the default topological sorting.

Minimal DAG to reproduce:

fromdatetimeimportdatetimefromairflow.sdkimportDAG, task, TaskGroup@taskdefnoop_task():
passwithDAG("grid_sort_bug_repro", start_date=datetime(2024, 1, 1), schedule=None, catchup=False):
withTaskGroup("stage_b"):
b_start=noop_task()
b_end=noop_task()
b_start>>b_endwithTaskGroup("stage_a"):
a_start=noop_task()
a_end=noop_task()
a_start>>a_endb_end>>a_start# task-level dep, bypasses group-level relationship

This shows in the grid view UI as:

stage_a
stage_b

Which is alphabetical, rather than the desired stage_b coming before stage_a.

This is happening because the _project_child_deps function only checks child.upstream_task_ids populated explicitly. We need to also scan the upstream_task_ids of the TaskGroup's root tasks so that the tasks within the group are also visible to this function.

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Important

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-06-17 14:51 UTC

Helpful heads-up from the maintainers — please address before this PR can be reviewed:

Static / docs checks failing (CI image checks / Static checks). Run them locally with prek run --all-files (or pre-commit run --all-files) and push the fixes.
Failing test jobs: Low dep tests:core / All-core:LowestDeps:14:3.10:Always...Core, MySQL tests: core / DB-core:MySQL:8.0:3.10:Always...Core, Postgres tests: core / DB-core:Postgres:14:3.10:Always...Core, Sqlite tests: core / DB-core:Sqlite:3.10:Always...Core. Reproduce and fix locally, then push.
See the Pull Request quality criteria.

The ball is in your court — you've been assigned to this PR. Fix the above, then mark it Ready for review.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

boring-cyborg · 2026-06-03T14:40:43Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

ashb

Please add unit tests covering it.

alex-stiff · 2026-06-18T12:34:45Z

Please add unit tests covering it.

Thank you, added.

potiuk · 2026-07-08T16:05:50Z

Note

🛠️ Maintainer triage note for @alex-stiff · by @potiuk · 2026-07-08 16:05 UTC

This draft has been inactive for ~20 days, so I'm closing it to keep the review queue tidy — no judgment on the work itself.

No rush — reopen it (or open a fresh PR) whenever you're ready to continue; nothing is lost.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

alex-stiff · 2026-07-08T16:14:26Z

@potiuk would you mind reopening this for me please, I don't seem to have perms! Thank you.

…deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite).

…dependencies (#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

…dependencies (#69933) (#70591) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: #65291 Related: #67964 (closed for inactivity, written against the topological_sort implementation before PR #67288/#67688 rewrote it) and #65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com> (cherry picked from commit d7aa929) # Conflicts: # airflow-core/src/airflow/api_fastapi/core_api/services/ui/task_group.py Co-authored-by: Hemkumar Chheda <95332229+hkc-8010@users.noreply.github.com>

…dependencies (apache#69933) * Fix grid/graph view topological sort for group-level and cross-group deps TaskGroup._project_child_deps only looked at a group's own upstream_task_ids, which stays empty for a direct group-to-group dependency (list or individual `>>`) and for a task-level dependency that crosses into another group's entry task. Both cases sorted the group as if it had no upstream at all. Now also pulls in the group's upstream_group_ids and its root tasks' upstream task ids before projecting sibling dependencies. Applied to both the serialization-layer sort and the mirrored design-time sort in task-sdk. closes: apache#65291 Related: apache#67964 (closed for inactivity, written against the topological_sort implementation before PR apache#67288/apache#67688 rewrote it) and apache#65639 (draft, same issue, also predates the rewrite). * Address review feedback: cache get_task_group_dict, describe test intent not issue numbers viiccwen pointed out that fetching the group map inside topological_sort() rebuilds the whole DAG's group tree on every nested group's own call, turning a render with G groups into an O(G^2) cost. get_task_group_dict() is now memoized per DAG instance (kept behind a small private helper since methodtools.lru_cache has no type stubs and would otherwise widen the public method's return type to Any for every caller). Also reworded test comments/docstrings that cited issue numbers to describe what's actually being verified instead. * Hoist common logic into shared lib * Remove caching on get_task_group_dict * Remove stale get_task_group_dict cache tests The cache these tests asserted was removed in the previous commit, so the identity check and the _get_task_group_dict_cached.cache_info() assertions no longer apply. * Add call-level task group memo to reduce calc * Tidy Typy * Add test for task group memoing --------- Co-authored-by: TP <uranusjr@apache.org> Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>

Fix grid view topological sort for cross-group dependencies
68d63f4

alex-stiff requested review from ashb and bolkedebruin as code owners June 3, 2026 14:40

boring-cyborgBot added the area:DAG-processing label Jun 3, 2026

alex-stiff added 2 commits June 3, 2026 15:44

Merge branch 'main' into patch-1
fe474ef

Merge branch 'main' into patch-1
75d8e96

potiuk assigned alex-stiff Jun 12, 2026

Merge branch 'main' into patch-1
ab9f83e

ashb reviewed Jun 18, 2026
View reviewed changes

alex-stiff added 2 commits June 18, 2026 13:25

Update test_task_group.py
cd8af58

Merge branch 'main' into patch-1
8835298

potiuk marked this pull request as draft June 18, 2026 21:47

potiuk closed this Jul 8, 2026

hkc-8010 mentioned this pull request Jul 15, 2026
Fix grid/graph view topological sort for group-level and cross-group dependencies #69933
Merged
1 task

jason810496 mentioned this pull request Jul 28, 2026
[v3-3-test] Fix grid/graph view topological sort for group-level and cross-group dependencies (#69933) #70591
Merged

Fix grid view topological sort for cross-group dependencies
68d63f4

alex-stiff requested review from ashb and bolkedebruin as code owners June 3, 2026 14:40

boring-cyborgBot added the area:DAG-processing label Jun 3, 2026

alex-stiff added 2 commits June 3, 2026 15:44

Merge branch 'main' into patch-1
fe474ef

Merge branch 'main' into patch-1
75d8e96

potiuk assigned alex-stiff Jun 12, 2026

Merge branch 'main' into patch-1
ab9f83e

ashb reviewed Jun 18, 2026
View reviewed changes

alex-stiff added 2 commits June 18, 2026 13:25

Update test_task_group.py
cd8af58

Merge branch 'main' into patch-1
8835298

potiuk marked this pull request as draft June 18, 2026 21:47

potiuk closed this Jul 8, 2026

hkc-8010 mentioned this pull request Jul 15, 2026
Fix grid/graph view topological sort for group-level and cross-group dependencies #69933
Merged
1 task

jason810496 mentioned this pull request Jul 28, 2026
[v3-3-test] Fix grid/graph view topological sort for group-level and cross-group dependencies (#69933) #70591
Merged

Fix grid view topological sort for cross-group dependencies
68d63f4

alex-stiff requested review from ashb and bolkedebruin as code owners June 3, 2026 14:40

boring-cyborgBot added the area:DAG-processing label Jun 3, 2026

alex-stiff added 2 commits June 3, 2026 15:44

Merge branch 'main' into patch-1
fe474ef

Merge branch 'main' into patch-1
75d8e96

potiuk assigned alex-stiff Jun 12, 2026

Merge branch 'main' into patch-1
ab9f83e

ashb reviewed Jun 18, 2026
View reviewed changes

alex-stiff added 2 commits June 18, 2026 13:25

Update test_task_group.py
cd8af58

Merge branch 'main' into patch-1
8835298

potiuk marked this pull request as draft June 18, 2026 21:47

potiuk closed this Jul 8, 2026

hkc-8010 mentioned this pull request Jul 15, 2026
Fix grid/graph view topological sort for group-level and cross-group dependencies #69933
Merged
1 task

jason810496 mentioned this pull request Jul 28, 2026
[v3-3-test] Fix grid/graph view topological sort for group-level and cross-group dependencies (#69933) #70591
Merged

Fix grid view topological sort for cross-group dependencies
68d63f4

alex-stiff requested review from ashb and bolkedebruin as code owners June 3, 2026 14:40

boring-cyborgBot added the area:DAG-processing label Jun 3, 2026

alex-stiff added 2 commits June 3, 2026 15:44

Merge branch 'main' into patch-1
fe474ef

Merge branch 'main' into patch-1
75d8e96

potiuk assigned alex-stiff Jun 12, 2026

Merge branch 'main' into patch-1
ab9f83e

ashb reviewed Jun 18, 2026
View reviewed changes

alex-stiff added 2 commits June 18, 2026 13:25

Update test_task_group.py
cd8af58

Merge branch 'main' into patch-1
8835298

potiuk marked this pull request as draft June 18, 2026 21:47

potiuk closed this Jul 8, 2026

hkc-8010 mentioned this pull request Jul 15, 2026
Fix grid/graph view topological sort for group-level and cross-group dependencies #69933
Merged
1 task

jason810496 mentioned this pull request Jul 28, 2026
[v3-3-test] Fix grid/graph view topological sort for group-level and cross-group dependencies (#69933) #70591
Merged

Fix grid view topological sort for cross-group dependencies
68d63f4

alex-stiff requested review from ashb and bolkedebruin as code owners June 3, 2026 14:40

boring-cyborgBot added the area:DAG-processing label Jun 3, 2026

alex-stiff added 2 commits June 3, 2026 15:44

Merge branch 'main' into patch-1
fe474ef

Merge branch 'main' into patch-1
75d8e96

potiuk assigned alex-stiff Jun 12, 2026

Merge branch 'main' into patch-1
ab9f83e

ashb reviewed Jun 18, 2026
View reviewed changes

alex-stiff added 2 commits June 18, 2026 13:25

Update test_task_group.py
cd8af58

Merge branch 'main' into patch-1
8835298

potiuk marked this pull request as draft June 18, 2026 21:47

potiuk closed this Jul 8, 2026

hkc-8010 mentioned this pull request Jul 15, 2026
Fix grid/graph view topological sort for group-level and cross-group dependencies #69933
Merged
1 task

jason810496 mentioned this pull request Jul 28, 2026
[v3-3-test] Fix grid/graph view topological sort for group-level and cross-group dependencies (#69933) #70591
Merged

Fix grid view topological sort for cross-group dependencies
68d63f4

alex-stiff requested review from ashb and bolkedebruin as code owners June 3, 2026 14:40

boring-cyborgBot added the area:DAG-processing label Jun 3, 2026

alex-stiff added 2 commits June 3, 2026 15:44

Merge branch 'main' into patch-1
fe474ef

Merge branch 'main' into patch-1
75d8e96

potiuk assigned alex-stiff Jun 12, 2026

Merge branch 'main' into patch-1
ab9f83e

ashb reviewed Jun 18, 2026
View reviewed changes

alex-stiff added 2 commits June 18, 2026 13:25

Update test_task_group.py
cd8af58

Merge branch 'main' into patch-1
8835298

potiuk marked this pull request as draft June 18, 2026 21:47

potiuk closed this Jul 8, 2026

hkc-8010 mentioned this pull request Jul 15, 2026
Fix grid/graph view topological sort for group-level and cross-group dependencies #69933
Merged
1 task

jason810496 mentioned this pull request Jul 28, 2026
[v3-3-test] Fix grid/graph view topological sort for group-level and cross-group dependencies (#69933) #70591
Merged

Fix grid view topological sort for cross-group dependencies
68d63f4

alex-stiff requested review from ashb and bolkedebruin as code owners June 3, 2026 14:40

boring-cyborgBot added the area:DAG-processing label Jun 3, 2026

alex-stiff added 2 commits June 3, 2026 15:44

Merge branch 'main' into patch-1
fe474ef

Merge branch 'main' into patch-1
75d8e96

potiuk assigned alex-stiff Jun 12, 2026

Merge branch 'main' into patch-1
ab9f83e

ashb reviewed Jun 18, 2026
View reviewed changes

alex-stiff added 2 commits June 18, 2026 13:25

Update test_task_group.py
cd8af58

Merge branch 'main' into patch-1
8835298

potiuk marked this pull request as draft June 18, 2026 21:47

potiuk closed this Jul 8, 2026

hkc-8010 mentioned this pull request Jul 15, 2026
Fix grid/graph view topological sort for group-level and cross-group dependencies #69933
Merged
1 task

jason810496 mentioned this pull request Jul 28, 2026
[v3-3-test] Fix grid/graph view topological sort for group-level and cross-group dependencies (#69933) #70591
Merged

Fix grid view topological sort for cross-group dependencies
68d63f4

alex-stiff requested review from ashb and bolkedebruin as code owners June 3, 2026 14:40

boring-cyborgBot added the area:DAG-processing label Jun 3, 2026

alex-stiff added 2 commits June 3, 2026 15:44

Merge branch 'main' into patch-1
fe474ef

Merge branch 'main' into patch-1
75d8e96

potiuk assigned alex-stiff Jun 12, 2026

Merge branch 'main' into patch-1
ab9f83e

ashb reviewed Jun 18, 2026
View reviewed changes

alex-stiff added 2 commits June 18, 2026 13:25

Update test_task_group.py
cd8af58

Merge branch 'main' into patch-1
8835298

potiuk marked this pull request as draft June 18, 2026 21:47

potiuk closed this Jul 8, 2026

hkc-8010 mentioned this pull request Jul 15, 2026
Fix grid/graph view topological sort for group-level and cross-group dependencies #69933
Merged
1 task

jason810496 mentioned this pull request Jul 28, 2026
[v3-3-test] Fix grid/graph view topological sort for group-level and cross-group dependencies (#69933) #70591
Merged

Uh oh!

Conversation

alex-stiff commented Jun 3, 2026• edited by potiuk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Was generative AI tooling used to co-author this PR?

Uh oh!

boring-cyborgBot commented Jun 3, 2026

Uh oh!

ashb left a comment

Choose a reason for hiding this comment

Uh oh!

alex-stiff commented Jun 18, 2026

Uh oh!

potiuk commented Jul 8, 2026

Uh oh!

alex-stiff commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

alex-stiff commented Jun 3, 2026• edited by potiuk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Was generative AI tooling used to co-author this PR?

Uh oh!

boring-cyborgBot commented Jun 3, 2026

Uh oh!

ashb left a comment

Choose a reason for hiding this comment

Uh oh!

alex-stiff commented Jun 18, 2026

Uh oh!

potiuk commented Jul 8, 2026

Uh oh!

alex-stiff commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

alex-stiff commented Jun 3, 2026• edited by potiuk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Was generative AI tooling used to co-author this PR?

Uh oh!

boring-cyborgBot commented Jun 3, 2026

Uh oh!

ashb left a comment

Choose a reason for hiding this comment

Uh oh!

alex-stiff commented Jun 18, 2026

Uh oh!

potiuk commented Jul 8, 2026

Uh oh!

alex-stiff commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

alex-stiff commented Jun 3, 2026• edited by potiuk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Was generative AI tooling used to co-author this PR?

alex-stiff commented Jun 3, 2026•
edited by potiuk
Loading

alex-stiff commented Jun 3, 2026•
edited by potiuk
Loading

alex-stiff commented Jun 3, 2026•
edited by potiuk
Loading

alex-stiff commented Jun 3, 2026•
edited by potiuk
Loading

alex-stiff commented Jun 3, 2026•
edited by potiuk
Loading