Show task state counts for the latest Dag run on the Dags list page

[yuseok89]

closes: #66947

Each Dag card and table row on the Dag list page now shows the task-instance state breakdown of that Dag's most recent run, as clickable badges. Clicking a badge opens that run's task list filtered by the state, restoring the scan-then-click "Recent Tasks" workflow from Airflow 2.

What changed

• New endpointGET /api/v2/ui/dags/latest_run_task_instance_state_counts. For the requested Dag IDs it resolves each Dag's latest run and counts its task instances grouped by state. dag_ids is bounded to the API max page limit. A run's task instances are bounded by the Dag's task structure, so the per-state counts are exact.
• Card view shows the state badges below the recent-runs bar chart.
• Table view adds compact badges in a new "Latest run tasks" column.
• Dynamic badges show only the states present in the run, ordered by priority (most urgent first). Task instances with no state yet appear as no_status.

Unlike the run-state badges from #67242 (which count a Dag's runs by state), these count the task instances of its single latest run.

Demo

Screen.Recording.2026-07-08.at.11.08.18.PM.mov

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)
  • Opus 4.8

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[bbovenzi]

Thanks for opening this.

Let's make sure dags list performance issues are resolved first. Like in this PR: #69806. Only after that should we add any more complexity to the dags list

[yuseok89]

Thanks for opening this.

Let's make sure dags list performance issues are resolved first. Like in this PR: #69806. Only after that should we add any more complexity to the dags list

Makes sense, thanks. I'll take a look at the dags list performance work too and see if there's anything I can help with there.

[yuseok89]

Hi @bbovenzi.
The dags list performance work is now resolved (#69806 merged, #69609 closed).
So would this be a good time to pick it back up?

[pierrejeambrun]

Can you benchmark this when there are millions of TI and hundred of thousand of runs in the DB?

I would like to double check performance, my feeling is that it will not scale way because of the subqueries

[yuseok89]

Good call, you were right. I seeded 600 Dags, 850k runs, 3M task instances (Postgres 14) and measured a 50-Dag page.

The bottleneck is the per-Dag ORDER BY run_after DESC LIMIT 1. There is no (dag_id, run_after) index, so a Dag whose latest run is old makes the planner walk idx_dag_run_run_after backwards across most of the table. For a page of such Dags that comes out around 1050 ms.

before, one branch of the per-Dag lookup
Index Scan Backward using idx_dag_run_run_after on dag_run dag_run_11
(actual time=152.613..152.614 rows=1 loops=1)
Filter: ((dag_id)::text = 'stale_dag_0011'::text)
Rows Removed by Filter: 800043
Buffers: shared hit=801672
Execution Time: 1049.016 ms

The fix drops that lookup. The Dags list already carries each Dag's latest run, so the endpoint now takes dag_run_ids instead of dag_ids and counts them directly, around 0.5 ms for the same page. As a bonus the list and the counts can no longer resolve different latest runs.

after
Index Scan using dag_run_pkey on dag_run (actual time=0.034..0.465 rows=50 loops=1)
Execution Time: 0.541 ms

Pushed.