feat: adding new feature to mark task group as success or failed by itayweb · Pull Request #58102 · apache/airflow

itayweb · 2025-11-09T15:39:22Z

closes: #56103
related: #50443

Adding new feature for Apache Airflow 3.x to mark TaskGroup tasks as Success/Failed, using existing bulk api

boring-cyborg · 2025-11-09T15:39:30Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

guan404ming · 2025-11-12T04:12:09Z

Could you help resolve static check error and attatch some screenshot or demo for your changes? Thanks!

jason810496

Thanks for the PR!

The unit tests are also required in airflow-core/tests/unit/api_fastapi/core_api/services/public/test_task_instances.py for the corresponding changes, thanks!

itayweb · 2025-11-12T21:40:06Z

Could you help resolve static check error and attatch some screenshot or demo for your changes? Thanks!

What do you mean by static check error? Regarding the screenshots I'll add

potiuk · 2025-11-12T23:24:16Z

What do you mean by static check error?

See the errors in the CI outputs - you need to have prek installed and fix all the issues (prek will fix them mostly automatically) - make sure also to rebase to latest main

uranusjr · 2025-11-13T06:52:17Z

 """Bulk update, and delete task instances."""
 return BulkTaskInstanceService(
- session=session, request=request, dag_id=dag_id, dag_run_id=dag_run_id, dag_bag=dag_bag, user=user
+ session=session, request=request, dag_id=dag_id, dag_run_id=dag_run_id, dag_bag=dag_bag, user=user, commit=True,


Why does this need to change?

I've removed the duplicate dry run function and used the new parameter passed (commit) to reuse the existing function

itayweb · 2025-11-22T16:09:50Z

Thanks for the PR!
The unit tests are also required in airflow-core/tests/unit/api_fastapi/core_api/services/public/test_task_instances.py for the corresponding changes, thanks!

I've took a look into the unit tests that are already exist and noticed that there is already for bulk task instances but in the routes instead of in the services, so is it suitable for this? looks good to me, also there is not for the new added dry run, should I added another for the dry run?

pierrejeambrun

Cool PR.

A few suggestions before we can merge 🎉

itayweb · 2025-12-06T20:40:37Z

I assume this is some uups in rebase :-D

Yes 😅

pierrejeambrun

I gave it some thoughts and it appears that the bulk api doesn't really fit well here.

Maybe we should instead use the "patch/update" TI endpoint and add support for TaskGroup there.

There already is a dry run endpoint for this too. And in addition it returns a TI collection already because it handles the "multiple mapped indexes" use case.

pierrejeambrun · 2026-01-09T10:22:03Z

Feel free to resolve irrelevant/addressed comments. Let us know when the PR is ready for another round of review, it looks like there are unrelated change added that need to be cleaned.

itayweb · 2026-01-09T10:37:16Z

Feel free to resolve irrelevant/addressed comments. Let us know when the PR is ready for another round of review, it looks like there are unrelated change added that need to be cleaned.

Currently working on your last comment of using the existing patch endpoint

itayweb · 2026-01-09T14:53:57Z

I've created a new endpoint for patching task and task group to not interfere with existing routes/tests, let me know what do you all think about my new approach 😄

…n clear multiple tasks

pierrejeambrun

#60161 is closer to being mergeable. There is a bunch here to address and I would suggest closing it in favor of #60161, sorry for the inconvenience.

pierrejeambrun · 2026-01-30T13:53:50Z

+ ],
+ operation_id="post_patch_task_instances",
+)
+def post_patch_task_instances(


Some cleaning like that post_patch

pierrejeambrun · 2026-01-30T15:06:19Z

 application/json:
 schema:
 $ref: '#/components/schemas/HTTPValidationError'
+ /api/v2/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/dry_run:


We shouldn't use the bulk task instance endpoint.

itayweb · 2026-01-30T21:44:26Z

#60161 is closer to being mergeable. There is a bunch here to address and I would suggest closing it in favor of #60161, sorry for the inconvenience.

I see.. so should I fix your comments anyway or should I wait for another maintainers?

guan404ming · 2026-01-31T03:39:05Z

I see.. so should I fix your comments anyway or should I wait for another maintainers?

Sorry about that. I think you could take your time on finding another issue and trying to fix it. Meanwhile, you could still read the review from reviewer here and learn how to improve your next pr. Thanks for your contribution and looking forward to your next try!

pierrejeambrun · 2026-02-02T08:45:02Z

@itayweb I don't think it's worth continuing the effort at this point.

I'll close this PR in favor of #60161, feel free to re-open if you still want to work on it but it's unlikely it will get merged.

boring-cyborgBot added area:API Airflow's REST/HTTP API area:serialization labels Nov 9, 2025

boring-cyborgBot added the area:UI Related to UI/UX. For Frontend Developers. label Nov 9, 2025

itayweb marked this pull request as ready for review November 9, 2025 18:25

itayweb requested review from bbovenzi, bugraoz93, ephraimbuddy, guan404ming, jason810496, pierrejeambrun, rawwar, ryanahamilton and shubhamraj-git as code owners November 9, 2025 18:25

jason810496 reviewed Nov 12, 2025
View reviewed changes

Comment threadairflow-core/src/airflow/api_fastapi/core_api/services/public/task_instances.py
Comment threadairflow-core/src/airflow/api_fastapi/core_api/services/public/task_instances.py Outdated

uranusjr reviewed Nov 13, 2025
View reviewed changes

Comment threadairflow-core/src/airflow/api_fastapi/core_api/routes/public/task_instances.py

itayweb force-pushed the mark_as_task_group branch from 5228041 to 0f9013fCompare November 14, 2025 10:44

itayweb requested review from jason810496 and uranusjr November 22, 2025 16:10

pierrejeambrun mentioned this pull request Dec 1, 2025
Add task group ID filtering support to task instance query #58092
Merged

pierrejeambrun reviewed Dec 1, 2025
View reviewed changes

itayweb requested review from amoghrajesh, ashb, kaxil and potiuk as code owners December 6, 2025 20:31

itayweb force-pushed the mark_as_task_group branch from ddf30db to 80172f6Compare December 6, 2025 20:39

pierrejeambrun reviewed Dec 8, 2025
View reviewed changes

itayweb force-pushed the mark_as_task_group branch 2 times, most recently from 10b0b1a to ce03f38Compare December 26, 2025 11:42

itayweb force-pushed the mark_as_task_group branch from ce03f38 to c491700Compare January 9, 2026 14:52

itayweb requested a review from choo121600 as a code owner January 9, 2026 14:52

itayweb requested a review from pierrejeambrun January 9, 2026 19:28

itayweb force-pushed the mark_as_task_group branch from c491700 to 777b701Compare January 22, 2026 13:19

itayweb added 11 commits January 25, 2026 21:01

feat: adding new feature to mark task group as success or failed
4208cc4

fix: fixing dry run for mark task group
35850a7

fix: removed unnecessary code
e4a82e5

fix: removing the dry run function
284ad7b

feat: adding new feature to mark task group as success or failed
fe8ea9c

fix: removing the dry run function
7fb143b

reverting to TI due to unnecessary changes
5a84212

fix: rebasing and fixing Button wrong import
d654895

feat: creating new endpoint to enable patching multiple tasks based o…
72d52c6
…n clear multiple tasks

fix: rebasing, resolving conflicts and fixing some of the checks errors
ff1e05c

fix: fixing tests errors
461e109

itayweb force-pushed the mark_as_task_group branch from 777b701 to 461e109Compare January 25, 2026 19:01

pierrejeambrun reviewed Jan 30, 2026
View reviewed changes

pierrejeambrun closed this Feb 2, 2026

Uh oh!

Conversation

itayweb commented Nov 9, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boring-cyborgBot commented Nov 9, 2025

Uh oh!

guan404ming commented Nov 12, 2025

Uh oh!

jason810496 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

itayweb commented Nov 12, 2025

Uh oh!

potiuk commented Nov 12, 2025

Uh oh!

uranusjrNov 13, 2025

Choose a reason for hiding this comment

Uh oh!

itaywebNov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

itayweb commented Nov 22, 2025

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

itayweb commented Dec 6, 2025

Uh oh!

pierrejeambrun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun commented Jan 9, 2026

Uh oh!

itayweb commented Jan 9, 2026

Uh oh!

itayweb commented Jan 9, 2026

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

pierrejeambrunJan 30, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pierrejeambrunJan 30, 2026

Choose a reason for hiding this comment

Uh oh!

itayweb commented Jan 30, 2026

Uh oh!

guan404ming commented Jan 31, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pierrejeambrun commented Feb 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

itayweb commented Nov 9, 2025•
edited
Loading

pierrejeambrun left a comment •
edited
Loading

pierrejeambrunJan 30, 2026•
edited
Loading

guan404ming commented Jan 31, 2026•
edited
Loading