[v3-2-test] UI: Serve grid TI summaries from shared cached DagBag (#65775) by pierrejeambrun · Pull Request #65966 · apache/airflow

pierrejeambrun · 2026-04-27T16:20:28Z

Backport of #65775 to v3-2-test.

Cherry-picked from be41cb4.

Conflict in airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py:

v3-2-test still uses the simpler _build_ti_summaries shape (collections.defaultdict(list) + _get_aggs_for_node, returning dict) — kept that, just switched the DAG lookup to the shared DBDagBag.
v3-2-test's DBDagBag does not yet expose a public get_dag(version_id, session) (added on main alongside the LRU/TTL cache). Used _get_dag(...) instead, matching how airflow.api_fastapi.common.dagbag.get_dag_for_run_or_latest_version already calls it on this branch.
Narrowed the dag_version_id parameter to UUID | None in _get_serdag to match v3-2-test's DBDagBag._get_dag signature (mypy caught this).
v3-2-test's _find_aggregates yields plain dicts (not (node, _) tuples), so kept for node in _find_aggregates(...).
Dropped the per-request serdag_cache: dict = {} in get_grid_ti_summaries_stream — the shared DBDagBag instance attached to app.state already deduplicates across runs and across requests.

Generated files (_private_ui.yaml, the openapi-gen *.ts) auto-merged cleanly.

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

The grid TI summaries streaming endpoint used a per-request dict to dedupe serialized Dag loads within a single response. Switch to the app-wide DBDagBag so the serialized Dag is cached across requests (via the LRU+TTL configured by [api] dag_cache_size / dag_cache_ttl) and the in-memory representation is the lightweight SerializedDAG instead of the SerializedDagModel ORM row holding the full JSON blob. (cherry picked from commit be41cb4)

pierrejeambrun · 2026-04-27T16:20:28Z

Backport of #65775 to v3-2-test.

Cherry-picked from be41cb4.

Conflict in airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py:

v3-2-test still uses the simpler _build_ti_summaries shape (collections.defaultdict(list) + _get_aggs_for_node, returning dict) — kept that, just switched the DAG lookup to the shared DBDagBag.
v3-2-test's DBDagBag does not yet expose a public get_dag(version_id, session) (added on main alongside the LRU/TTL cache). Used _get_dag(...) instead, matching how airflow.api_fastapi.common.dagbag.get_dag_for_run_or_latest_version already calls it on this branch.
Narrowed the dag_version_id parameter to UUID | None in _get_serdag to match v3-2-test's DBDagBag._get_dag signature (mypy caught this).
v3-2-test's _find_aggregates yields plain dicts (not (node, _) tuples), so kept for node in _find_aggregates(...).
Dropped the per-request serdag_cache: dict = {} in get_grid_ti_summaries_stream — the shared DBDagBag instance attached to app.state already deduplicates across runs and across requests.

Generated files (_private_ui.yaml, the openapi-gen *.ts) auto-merged cleanly.

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

The grid TI summaries streaming endpoint used a per-request dict to dedupe serialized Dag loads within a single response. Switch to the app-wide DBDagBag so the serialized Dag is cached across requests (via the LRU+TTL configured by [api] dag_cache_size / dag_cache_ttl) and the in-memory representation is the lightweight SerializedDAG instead of the SerializedDagModel ORM row holding the full JSON blob. (cherry picked from commit be41cb4)

pierrejeambrun · 2026-04-27T16:20:28Z

Backport of #65775 to v3-2-test.

Cherry-picked from be41cb4.

Conflict in airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py:

v3-2-test still uses the simpler _build_ti_summaries shape (collections.defaultdict(list) + _get_aggs_for_node, returning dict) — kept that, just switched the DAG lookup to the shared DBDagBag.
v3-2-test's DBDagBag does not yet expose a public get_dag(version_id, session) (added on main alongside the LRU/TTL cache). Used _get_dag(...) instead, matching how airflow.api_fastapi.common.dagbag.get_dag_for_run_or_latest_version already calls it on this branch.
Narrowed the dag_version_id parameter to UUID | None in _get_serdag to match v3-2-test's DBDagBag._get_dag signature (mypy caught this).
v3-2-test's _find_aggregates yields plain dicts (not (node, _) tuples), so kept for node in _find_aggregates(...).
Dropped the per-request serdag_cache: dict = {} in get_grid_ti_summaries_stream — the shared DBDagBag instance attached to app.state already deduplicates across runs and across requests.

Generated files (_private_ui.yaml, the openapi-gen *.ts) auto-merged cleanly.

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

The grid TI summaries streaming endpoint used a per-request dict to dedupe serialized Dag loads within a single response. Switch to the app-wide DBDagBag so the serialized Dag is cached across requests (via the LRU+TTL configured by [api] dag_cache_size / dag_cache_ttl) and the in-memory representation is the lightweight SerializedDAG instead of the SerializedDagModel ORM row holding the full JSON blob. (cherry picked from commit be41cb4)

pierrejeambrun · 2026-04-27T16:20:28Z

Backport of #65775 to v3-2-test.

Cherry-picked from be41cb4.

Conflict in airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py:

v3-2-test still uses the simpler _build_ti_summaries shape (collections.defaultdict(list) + _get_aggs_for_node, returning dict) — kept that, just switched the DAG lookup to the shared DBDagBag.
v3-2-test's DBDagBag does not yet expose a public get_dag(version_id, session) (added on main alongside the LRU/TTL cache). Used _get_dag(...) instead, matching how airflow.api_fastapi.common.dagbag.get_dag_for_run_or_latest_version already calls it on this branch.
Narrowed the dag_version_id parameter to UUID | None in _get_serdag to match v3-2-test's DBDagBag._get_dag signature (mypy caught this).
v3-2-test's _find_aggregates yields plain dicts (not (node, _) tuples), so kept for node in _find_aggregates(...).
Dropped the per-request serdag_cache: dict = {} in get_grid_ti_summaries_stream — the shared DBDagBag instance attached to app.state already deduplicates across runs and across requests.

Generated files (_private_ui.yaml, the openapi-gen *.ts) auto-merged cleanly.

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

The grid TI summaries streaming endpoint used a per-request dict to dedupe serialized Dag loads within a single response. Switch to the app-wide DBDagBag so the serialized Dag is cached across requests (via the LRU+TTL configured by [api] dag_cache_size / dag_cache_ttl) and the in-memory representation is the lightweight SerializedDAG instead of the SerializedDagModel ORM row holding the full JSON blob. (cherry picked from commit be41cb4)

pierrejeambrun · 2026-04-27T16:20:28Z

Backport of #65775 to v3-2-test.

Cherry-picked from be41cb4.

Conflict in airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py:

v3-2-test still uses the simpler _build_ti_summaries shape (collections.defaultdict(list) + _get_aggs_for_node, returning dict) — kept that, just switched the DAG lookup to the shared DBDagBag.
v3-2-test's DBDagBag does not yet expose a public get_dag(version_id, session) (added on main alongside the LRU/TTL cache). Used _get_dag(...) instead, matching how airflow.api_fastapi.common.dagbag.get_dag_for_run_or_latest_version already calls it on this branch.
Narrowed the dag_version_id parameter to UUID | None in _get_serdag to match v3-2-test's DBDagBag._get_dag signature (mypy caught this).
v3-2-test's _find_aggregates yields plain dicts (not (node, _) tuples), so kept for node in _find_aggregates(...).
Dropped the per-request serdag_cache: dict = {} in get_grid_ti_summaries_stream — the shared DBDagBag instance attached to app.state already deduplicates across runs and across requests.

Generated files (_private_ui.yaml, the openapi-gen *.ts) auto-merged cleanly.

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

The grid TI summaries streaming endpoint used a per-request dict to dedupe serialized Dag loads within a single response. Switch to the app-wide DBDagBag so the serialized Dag is cached across requests (via the LRU+TTL configured by [api] dag_cache_size / dag_cache_ttl) and the in-memory representation is the lightweight SerializedDAG instead of the SerializedDagModel ORM row holding the full JSON blob. (cherry picked from commit be41cb4)

pierrejeambrun · 2026-04-27T16:20:28Z

Backport of #65775 to v3-2-test.

Cherry-picked from be41cb4.

Conflict in airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py:

v3-2-test still uses the simpler _build_ti_summaries shape (collections.defaultdict(list) + _get_aggs_for_node, returning dict) — kept that, just switched the DAG lookup to the shared DBDagBag.
v3-2-test's DBDagBag does not yet expose a public get_dag(version_id, session) (added on main alongside the LRU/TTL cache). Used _get_dag(...) instead, matching how airflow.api_fastapi.common.dagbag.get_dag_for_run_or_latest_version already calls it on this branch.
Narrowed the dag_version_id parameter to UUID | None in _get_serdag to match v3-2-test's DBDagBag._get_dag signature (mypy caught this).
v3-2-test's _find_aggregates yields plain dicts (not (node, _) tuples), so kept for node in _find_aggregates(...).
Dropped the per-request serdag_cache: dict = {} in get_grid_ti_summaries_stream — the shared DBDagBag instance attached to app.state already deduplicates across runs and across requests.

Generated files (_private_ui.yaml, the openapi-gen *.ts) auto-merged cleanly.

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

The grid TI summaries streaming endpoint used a per-request dict to dedupe serialized Dag loads within a single response. Switch to the app-wide DBDagBag so the serialized Dag is cached across requests (via the LRU+TTL configured by [api] dag_cache_size / dag_cache_ttl) and the in-memory representation is the lightweight SerializedDAG instead of the SerializedDagModel ORM row holding the full JSON blob. (cherry picked from commit be41cb4)

pierrejeambrun · 2026-04-27T16:20:28Z

Backport of #65775 to v3-2-test.

Cherry-picked from be41cb4.

Conflict in airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py:

v3-2-test still uses the simpler _build_ti_summaries shape (collections.defaultdict(list) + _get_aggs_for_node, returning dict) — kept that, just switched the DAG lookup to the shared DBDagBag.
v3-2-test's DBDagBag does not yet expose a public get_dag(version_id, session) (added on main alongside the LRU/TTL cache). Used _get_dag(...) instead, matching how airflow.api_fastapi.common.dagbag.get_dag_for_run_or_latest_version already calls it on this branch.
Narrowed the dag_version_id parameter to UUID | None in _get_serdag to match v3-2-test's DBDagBag._get_dag signature (mypy caught this).
v3-2-test's _find_aggregates yields plain dicts (not (node, _) tuples), so kept for node in _find_aggregates(...).
Dropped the per-request serdag_cache: dict = {} in get_grid_ti_summaries_stream — the shared DBDagBag instance attached to app.state already deduplicates across runs and across requests.

Generated files (_private_ui.yaml, the openapi-gen *.ts) auto-merged cleanly.

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

The grid TI summaries streaming endpoint used a per-request dict to dedupe serialized Dag loads within a single response. Switch to the app-wide DBDagBag so the serialized Dag is cached across requests (via the LRU+TTL configured by [api] dag_cache_size / dag_cache_ttl) and the in-memory representation is the lightweight SerializedDAG instead of the SerializedDagModel ORM row holding the full JSON blob. (cherry picked from commit be41cb4)

pierrejeambrun · 2026-04-27T16:20:28Z

Backport of #65775 to v3-2-test.

Cherry-picked from be41cb4.

Conflict in airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py:

v3-2-test still uses the simpler _build_ti_summaries shape (collections.defaultdict(list) + _get_aggs_for_node, returning dict) — kept that, just switched the DAG lookup to the shared DBDagBag.
v3-2-test's DBDagBag does not yet expose a public get_dag(version_id, session) (added on main alongside the LRU/TTL cache). Used _get_dag(...) instead, matching how airflow.api_fastapi.common.dagbag.get_dag_for_run_or_latest_version already calls it on this branch.
Narrowed the dag_version_id parameter to UUID | None in _get_serdag to match v3-2-test's DBDagBag._get_dag signature (mypy caught this).
v3-2-test's _find_aggregates yields plain dicts (not (node, _) tuples), so kept for node in _find_aggregates(...).
Dropped the per-request serdag_cache: dict = {} in get_grid_ti_summaries_stream — the shared DBDagBag instance attached to app.state already deduplicates across runs and across requests.

Generated files (_private_ui.yaml, the openapi-gen *.ts) auto-merged cleanly.

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

The grid TI summaries streaming endpoint used a per-request dict to dedupe serialized Dag loads within a single response. Switch to the app-wide DBDagBag so the serialized Dag is cached across requests (via the LRU+TTL configured by [api] dag_cache_size / dag_cache_ttl) and the in-memory representation is the lightweight SerializedDAG instead of the SerializedDagModel ORM row holding the full JSON blob. (cherry picked from commit be41cb4)

pierrejeambrun requested review from bbovenzi, bugraoz93, choo121600, ephraimbuddy, guan404ming, jason810496, rawwar, ryanahamilton, shubhamraj-git and vatsrahul1001 as code owners April 27, 2026 16:20

boring-cyborgBot added area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. labels Apr 27, 2026

eladkal approved these changes Apr 27, 2026
View reviewed changes

pierrejeambrun merged commit 7249b6c into apache:v3-2-testApr 28, 2026
110 checks passed

pierrejeambrun deleted the backport-65775 branch April 28, 2026 07:58

vatsrahul1001 mentioned this pull request May 15, 2026
[v3-2-test] improve grid/ti_summaries and grid/runs (#64034) #66982
Closed
1 task

vatsrahul1001 added this to the Airflow 3.2.2 milestone May 18, 2026

vatsrahul1001 added the type:misc/internal Changelog: Misc changes that should appear in change log label May 18, 2026

vatsrahul1001 mentioned this pull request May 21, 2026
Status of testing of Apache Airflow 3.2.2rc3 #67282
Closed