feat(ui): add aggregate Dag schedule overview view (#68532) - #68547

Closed
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532
Closed

feat(ui): add aggregate Dag schedule overview view (#68532)#68547
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532

Conversation

@anmolxlight

Copy link
Copy Markdown
Contributor

Fixes#68532

What

Adds a new /dags/schedule page that visualizes the typical (mean / median) start and end time of each Dag over a 24-hour timeline, aggregated from recent successful Dag runs. This complements the existing per-Dag Gantt view, which only shows a single run at a time. The new view operates at a different level: it aggregates across all Dags to answer "across my whole deployment, when during the day does each Dag usually run?"

Why

Operators running many Dags currently have no at-a-glance way to see how scheduled load is distributed across the day. The new view makes the whole schedule visible at once, which helps to:

  • spot overlapping heavy windows where many Dags pile up at the same hour,
  • find quiet windows for maintenance / deploys / resource-intensive jobs,
  • notice drift when a Dag's typical run time shifts over recent runs,
  • plan capacity and SLAs across the fleet.

How

Backend

  • New /ui/dag_schedule_overview endpoint (in api_fastapi/core_api/routes/ui/dag_schedule_overview.py) gated by requires_access_dag(method="GET", access_entity=DagAccessEntity.RUN).
  • New DagScheduleOverviewService (api_fastapi/core_api/services/ui/dag_schedule_overview.py) that:
    • fetches all DagModel rows (so dags with no runs still appear as zero-statistic rows),
    • pulls up to 200 most-recent successful DagRuns per dag, optionally scoped by a run_after window,
    • computes mean/median of start_date and end_date mapped to seconds-of-day in UTC, plus mean/median duration,
    • supports dag_id_pattern and dag_display_name_pattern filters.
  • New datamodels DagScheduleOverviewEntry + DagScheduleOverviewCollectionResponse (api_fastapi/core_api/datamodels/ui/dag_schedule_overview.py).
  • 11 backend tests in tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py: 200/401/403, all-three-dag fixture, morning/evening/no-runs statistics, FAILED-run exclusion, pattern filters, run_after window edges.
  • OpenAPI _private_ui.yaml updated; pnpm codegen regenerated.

Frontend

  • New page src/pages/DagsList/ScheduleOverview/ mounted at /dags/schedule with:
    • one row per Dag (linkable to the Dag detail page),
    • a 24h tick rail at 0/3/6/9/12/15/18/21/24 hours,
    • a horizontal bar showing the mean start→end window,
    • a hover tooltip with mean and median start / end, mean and median duration, and the recent-runs date range,
    • a client-side dag-id filter input.
  • New sidebar nav button Schedule Overview (FiBarChart2) on the main nav, alongside Dags.
  • New i18n keys under dags.scheduleOverview.* and common.nav.scheduleOverview (English locale).

Verification

# Backend
uv run --project airflow-core pytest airflow-core/tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py --with-db-init
# 11 passed
uv run --project airflow-core ruff check + ruff format --check + mypy
# All clean
# Frontend
pnpm lint # eslint --quiet && tsc -p tsconfig.app.json — passes
pnpm prettier --check public/i18n/locales/en/dags.json public/i18n/locales/en/common.json
# Passes

The two pre-existing vitest failures in useTagFilter.test.tsx and Logs.test.tsx reproduce on main (verified with git stash), so they are unrelated to this PR.

@boring-cyborgboring-cyborgBot added area:API Airflow's REST/HTTP API area:translations area:UI Related to UI/UX. For Frontend Developers. translation:default labels Jun 14, 2026
@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from 64767cb to a33d4bdCompareJune 14, 2026 22:49
anmolxlight added a commit to anmolxlight/airflow that referenced this pull request Jun 15, 2026
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
@choo121600

Copy link
Copy Markdown
Member

Could you please attach a short recording of this feature following the PR template? It would be helpful for the review.

Adds a 24h Gantt-style view that answers the question `across my whole
deployment, when during the day does each Dag usually run?` — a
complement to the per-Dag Gantt view, which only shows a single run.
Backend
* New `/ui/dag_schedule_overview` endpoint that aggregates the
most-recent successful Dag runs per Dag and returns mean/median
start/end times-of-day in UTC, plus duration statistics.
* Optional `run_after_gte` / `run_after_lte` window and
`dag_id_pattern` / `dag_display_name_pattern` filters.
* 11 backend tests covering 200/401/403, all-three-dag fixture,
morning/evening/no-runs statistics, FAILED-run exclusion, and the
filter / window edge cases.
* OpenAPI YAML updated, `pnpm codegen` regenerated.
Frontend
* New `/dags/schedule` page (one row per Dag, mean start→end bar
across a 24h tick rail, hover tooltip with mean+median start, end,
duration, and recent-runs range).
* New `Schedule Overview` sidebar nav link.
* i18n keys for English locale.
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
The OpenAPI spec regen in the previous commit changed the DagScheduleOverviewEntry
schema: start_*_seconds, end_*_seconds, duration_*_seconds, and their median
counterparts were moved out of the 'required' list in Pydantic v2. The auto-generated
TS type now marks these fields as 'number | null | undefined'.
The hand-written formatTooltipBody in scheduleOverviewUtils.ts only accepted
'number | null', causing:
error TS2345 in ScheduleOverviewRow.tsx:44
Argument of type 'DagScheduleOverviewEntry' is not assignable to parameter
of type '{ readonly end_mean_seconds: number | null; ... }'
Types of property 'end_mean_seconds' are incompatible.
Type 'number | null | undefined' is not assignable to type 'number | null'.
Widen the formatTooltipBody parameter to allow 'undefined' for the now-optional
duration/start/end fields. The downstream helpers (formatSecondsAsClock,
formatDurationSeconds) already accept undefined and render it as em-dash.
Local verification:
- prek run ts-compile-lint-ui --files <changed file>: Passed
- prek run --files <changed file>: Passed
…nd params
The previous commit added '| undefined' to the duration/start/end types in
formatTooltipBody's parameter, but TypeScript's structural typing is stricter
about optional vs required properties. Since DagScheduleOverviewEntry marks
end_mean_seconds / end_median_seconds / start_mean_seconds / start_median_seconds
as '?: number | null' (truly optional, undefined comes from optionality, not
the value type), the parameter must mirror that with '?:' on the same fields.
'?: number | null' and ': number | null | undefined' are NOT structurally
assignable in either direction under TypeScript's strictness here.
Use '?: number | null' to match the auto-generated type shape. The
downstream helpers (formatSecondsAsClock, formatDurationSeconds) still
handle the undefined case and render em-dash.
Local verification:
- pnpm tsc --noEmit -p tsconfig.app.json: clean
- prek run --files <changed file>: Passed
- prek run ts-compile-lint-ui --files <changed file>: Passed
@anmolxlight

anmolxlight commented Jun 15, 2026

Copy link
Copy Markdown
ContributorAuthor
VID_20260615_232836_381.mp4

Screen recording of the new Schedule Overview page:

Shows the /dags/schedule page with the 24h UTC timeline bars, hover tooltips, and the filter working.

@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from a60f1d5 to b3353f0CompareJune 15, 2026 17:54

@choo121600choo121600 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the screen recording you attached, this appears to be somewhat inconsistent with the existing UI patterns.

I think we need more discussion in the issue before moving forward with an implementation. In particular, we should first align on whether this feature should be included at all, and if so, how it should be integrated into the UI (tab placement, design, UX, and overall approach).

Could you please close this PR for now? I'd prefer to continue the discussion in the issue first and revisit the implementation once we've reached agreement on the design.

Once we've aligned on the direction in the issue, I'd be happy to see this PR reopened.

@potiukpotiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jun 17, 2026
@jroachgolf84

Copy link
Copy Markdown
Collaborator

I don't think that this is a feature that should be implemented. See my comment from the issue:

Not quite sure how this is useful:

typical (mean / median) start and end time of each Dag over a 24-hour timeline

If I run a DAG hourly, that would make the average noon. That's pretty useless IMO. Plus, I don't think we could make this robust enough to handle all the scheduling use-cases that Airflow offers.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP APIarea:translationsarea:UIRelated to UI/UX. For Frontend Developers.ready for maintainer reviewSet after triaging when all criteria pass.translation:default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an aggregate Dag schedule view: typical daily run times across all Dags

4 participants

@anmolxlight@choo121600@jroachgolf84@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

feat(ui): add aggregate Dag schedule overview view (#68532) - #68547

Closed
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532
Closed

feat(ui): add aggregate Dag schedule overview view (#68532)#68547
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532

Conversation

@anmolxlight

Copy link
Copy Markdown
Contributor

Fixes#68532

What

Adds a new /dags/schedule page that visualizes the typical (mean / median) start and end time of each Dag over a 24-hour timeline, aggregated from recent successful Dag runs. This complements the existing per-Dag Gantt view, which only shows a single run at a time. The new view operates at a different level: it aggregates across all Dags to answer "across my whole deployment, when during the day does each Dag usually run?"

Why

Operators running many Dags currently have no at-a-glance way to see how scheduled load is distributed across the day. The new view makes the whole schedule visible at once, which helps to:

  • spot overlapping heavy windows where many Dags pile up at the same hour,
  • find quiet windows for maintenance / deploys / resource-intensive jobs,
  • notice drift when a Dag's typical run time shifts over recent runs,
  • plan capacity and SLAs across the fleet.

How

Backend

  • New /ui/dag_schedule_overview endpoint (in api_fastapi/core_api/routes/ui/dag_schedule_overview.py) gated by requires_access_dag(method="GET", access_entity=DagAccessEntity.RUN).
  • New DagScheduleOverviewService (api_fastapi/core_api/services/ui/dag_schedule_overview.py) that:
    • fetches all DagModel rows (so dags with no runs still appear as zero-statistic rows),
    • pulls up to 200 most-recent successful DagRuns per dag, optionally scoped by a run_after window,
    • computes mean/median of start_date and end_date mapped to seconds-of-day in UTC, plus mean/median duration,
    • supports dag_id_pattern and dag_display_name_pattern filters.
  • New datamodels DagScheduleOverviewEntry + DagScheduleOverviewCollectionResponse (api_fastapi/core_api/datamodels/ui/dag_schedule_overview.py).
  • 11 backend tests in tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py: 200/401/403, all-three-dag fixture, morning/evening/no-runs statistics, FAILED-run exclusion, pattern filters, run_after window edges.
  • OpenAPI _private_ui.yaml updated; pnpm codegen regenerated.

Frontend

  • New page src/pages/DagsList/ScheduleOverview/ mounted at /dags/schedule with:
    • one row per Dag (linkable to the Dag detail page),
    • a 24h tick rail at 0/3/6/9/12/15/18/21/24 hours,
    • a horizontal bar showing the mean start→end window,
    • a hover tooltip with mean and median start / end, mean and median duration, and the recent-runs date range,
    • a client-side dag-id filter input.
  • New sidebar nav button Schedule Overview (FiBarChart2) on the main nav, alongside Dags.
  • New i18n keys under dags.scheduleOverview.* and common.nav.scheduleOverview (English locale).

Verification

# Backend
uv run --project airflow-core pytest airflow-core/tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py --with-db-init
# 11 passed
uv run --project airflow-core ruff check + ruff format --check + mypy
# All clean
# Frontend
pnpm lint # eslint --quiet && tsc -p tsconfig.app.json — passes
pnpm prettier --check public/i18n/locales/en/dags.json public/i18n/locales/en/common.json
# Passes

The two pre-existing vitest failures in useTagFilter.test.tsx and Logs.test.tsx reproduce on main (verified with git stash), so they are unrelated to this PR.

@boring-cyborgboring-cyborgBot added area:API Airflow's REST/HTTP API area:translations area:UI Related to UI/UX. For Frontend Developers. translation:default labels Jun 14, 2026
@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from 64767cb to a33d4bdCompareJune 14, 2026 22:49
anmolxlight added a commit to anmolxlight/airflow that referenced this pull request Jun 15, 2026
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
@choo121600

Copy link
Copy Markdown
Member

Could you please attach a short recording of this feature following the PR template? It would be helpful for the review.

Adds a 24h Gantt-style view that answers the question `across my whole
deployment, when during the day does each Dag usually run?` — a
complement to the per-Dag Gantt view, which only shows a single run.
Backend
* New `/ui/dag_schedule_overview` endpoint that aggregates the
most-recent successful Dag runs per Dag and returns mean/median
start/end times-of-day in UTC, plus duration statistics.
* Optional `run_after_gte` / `run_after_lte` window and
`dag_id_pattern` / `dag_display_name_pattern` filters.
* 11 backend tests covering 200/401/403, all-three-dag fixture,
morning/evening/no-runs statistics, FAILED-run exclusion, and the
filter / window edge cases.
* OpenAPI YAML updated, `pnpm codegen` regenerated.
Frontend
* New `/dags/schedule` page (one row per Dag, mean start→end bar
across a 24h tick rail, hover tooltip with mean+median start, end,
duration, and recent-runs range).
* New `Schedule Overview` sidebar nav link.
* i18n keys for English locale.
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
The OpenAPI spec regen in the previous commit changed the DagScheduleOverviewEntry
schema: start_*_seconds, end_*_seconds, duration_*_seconds, and their median
counterparts were moved out of the 'required' list in Pydantic v2. The auto-generated
TS type now marks these fields as 'number | null | undefined'.
The hand-written formatTooltipBody in scheduleOverviewUtils.ts only accepted
'number | null', causing:
error TS2345 in ScheduleOverviewRow.tsx:44
Argument of type 'DagScheduleOverviewEntry' is not assignable to parameter
of type '{ readonly end_mean_seconds: number | null; ... }'
Types of property 'end_mean_seconds' are incompatible.
Type 'number | null | undefined' is not assignable to type 'number | null'.
Widen the formatTooltipBody parameter to allow 'undefined' for the now-optional
duration/start/end fields. The downstream helpers (formatSecondsAsClock,
formatDurationSeconds) already accept undefined and render it as em-dash.
Local verification:
- prek run ts-compile-lint-ui --files <changed file>: Passed
- prek run --files <changed file>: Passed
…nd params
The previous commit added '| undefined' to the duration/start/end types in
formatTooltipBody's parameter, but TypeScript's structural typing is stricter
about optional vs required properties. Since DagScheduleOverviewEntry marks
end_mean_seconds / end_median_seconds / start_mean_seconds / start_median_seconds
as '?: number | null' (truly optional, undefined comes from optionality, not
the value type), the parameter must mirror that with '?:' on the same fields.
'?: number | null' and ': number | null | undefined' are NOT structurally
assignable in either direction under TypeScript's strictness here.
Use '?: number | null' to match the auto-generated type shape. The
downstream helpers (formatSecondsAsClock, formatDurationSeconds) still
handle the undefined case and render em-dash.
Local verification:
- pnpm tsc --noEmit -p tsconfig.app.json: clean
- prek run --files <changed file>: Passed
- prek run ts-compile-lint-ui --files <changed file>: Passed
@anmolxlight

anmolxlight commented Jun 15, 2026

Copy link
Copy Markdown
ContributorAuthor
VID_20260615_232836_381.mp4

Screen recording of the new Schedule Overview page:

Shows the /dags/schedule page with the 24h UTC timeline bars, hover tooltips, and the filter working.

@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from a60f1d5 to b3353f0CompareJune 15, 2026 17:54

@choo121600choo121600 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the screen recording you attached, this appears to be somewhat inconsistent with the existing UI patterns.

I think we need more discussion in the issue before moving forward with an implementation. In particular, we should first align on whether this feature should be included at all, and if so, how it should be integrated into the UI (tab placement, design, UX, and overall approach).

Could you please close this PR for now? I'd prefer to continue the discussion in the issue first and revisit the implementation once we've reached agreement on the design.

Once we've aligned on the direction in the issue, I'd be happy to see this PR reopened.

@potiukpotiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jun 17, 2026
@jroachgolf84

Copy link
Copy Markdown
Collaborator

I don't think that this is a feature that should be implemented. See my comment from the issue:

Not quite sure how this is useful:

typical (mean / median) start and end time of each Dag over a 24-hour timeline

If I run a DAG hourly, that would make the average noon. That's pretty useless IMO. Plus, I don't think we could make this robust enough to handle all the scheduling use-cases that Airflow offers.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP APIarea:translationsarea:UIRelated to UI/UX. For Frontend Developers.ready for maintainer reviewSet after triaging when all criteria pass.translation:default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an aggregate Dag schedule view: typical daily run times across all Dags

4 participants

@anmolxlight@choo121600@jroachgolf84@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(ui): add aggregate Dag schedule overview view (#68532) - #68547

Closed
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532
Closed

feat(ui): add aggregate Dag schedule overview view (#68532)#68547
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532

Conversation

@anmolxlight

Copy link
Copy Markdown
Contributor

Fixes#68532

What

Adds a new /dags/schedule page that visualizes the typical (mean / median) start and end time of each Dag over a 24-hour timeline, aggregated from recent successful Dag runs. This complements the existing per-Dag Gantt view, which only shows a single run at a time. The new view operates at a different level: it aggregates across all Dags to answer "across my whole deployment, when during the day does each Dag usually run?"

Why

Operators running many Dags currently have no at-a-glance way to see how scheduled load is distributed across the day. The new view makes the whole schedule visible at once, which helps to:

  • spot overlapping heavy windows where many Dags pile up at the same hour,
  • find quiet windows for maintenance / deploys / resource-intensive jobs,
  • notice drift when a Dag's typical run time shifts over recent runs,
  • plan capacity and SLAs across the fleet.

How

Backend

  • New /ui/dag_schedule_overview endpoint (in api_fastapi/core_api/routes/ui/dag_schedule_overview.py) gated by requires_access_dag(method="GET", access_entity=DagAccessEntity.RUN).
  • New DagScheduleOverviewService (api_fastapi/core_api/services/ui/dag_schedule_overview.py) that:
    • fetches all DagModel rows (so dags with no runs still appear as zero-statistic rows),
    • pulls up to 200 most-recent successful DagRuns per dag, optionally scoped by a run_after window,
    • computes mean/median of start_date and end_date mapped to seconds-of-day in UTC, plus mean/median duration,
    • supports dag_id_pattern and dag_display_name_pattern filters.
  • New datamodels DagScheduleOverviewEntry + DagScheduleOverviewCollectionResponse (api_fastapi/core_api/datamodels/ui/dag_schedule_overview.py).
  • 11 backend tests in tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py: 200/401/403, all-three-dag fixture, morning/evening/no-runs statistics, FAILED-run exclusion, pattern filters, run_after window edges.
  • OpenAPI _private_ui.yaml updated; pnpm codegen regenerated.

Frontend

  • New page src/pages/DagsList/ScheduleOverview/ mounted at /dags/schedule with:
    • one row per Dag (linkable to the Dag detail page),
    • a 24h tick rail at 0/3/6/9/12/15/18/21/24 hours,
    • a horizontal bar showing the mean start→end window,
    • a hover tooltip with mean and median start / end, mean and median duration, and the recent-runs date range,
    • a client-side dag-id filter input.
  • New sidebar nav button Schedule Overview (FiBarChart2) on the main nav, alongside Dags.
  • New i18n keys under dags.scheduleOverview.* and common.nav.scheduleOverview (English locale).

Verification

# Backend
uv run --project airflow-core pytest airflow-core/tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py --with-db-init
# 11 passed
uv run --project airflow-core ruff check + ruff format --check + mypy
# All clean
# Frontend
pnpm lint # eslint --quiet && tsc -p tsconfig.app.json — passes
pnpm prettier --check public/i18n/locales/en/dags.json public/i18n/locales/en/common.json
# Passes

The two pre-existing vitest failures in useTagFilter.test.tsx and Logs.test.tsx reproduce on main (verified with git stash), so they are unrelated to this PR.

@boring-cyborgboring-cyborgBot added area:API Airflow's REST/HTTP API area:translations area:UI Related to UI/UX. For Frontend Developers. translation:default labels Jun 14, 2026
@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from 64767cb to a33d4bdCompareJune 14, 2026 22:49
anmolxlight added a commit to anmolxlight/airflow that referenced this pull request Jun 15, 2026
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
@choo121600

Copy link
Copy Markdown
Member

Could you please attach a short recording of this feature following the PR template? It would be helpful for the review.

Adds a 24h Gantt-style view that answers the question `across my whole
deployment, when during the day does each Dag usually run?` — a
complement to the per-Dag Gantt view, which only shows a single run.
Backend
* New `/ui/dag_schedule_overview` endpoint that aggregates the
most-recent successful Dag runs per Dag and returns mean/median
start/end times-of-day in UTC, plus duration statistics.
* Optional `run_after_gte` / `run_after_lte` window and
`dag_id_pattern` / `dag_display_name_pattern` filters.
* 11 backend tests covering 200/401/403, all-three-dag fixture,
morning/evening/no-runs statistics, FAILED-run exclusion, and the
filter / window edge cases.
* OpenAPI YAML updated, `pnpm codegen` regenerated.
Frontend
* New `/dags/schedule` page (one row per Dag, mean start→end bar
across a 24h tick rail, hover tooltip with mean+median start, end,
duration, and recent-runs range).
* New `Schedule Overview` sidebar nav link.
* i18n keys for English locale.
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
The OpenAPI spec regen in the previous commit changed the DagScheduleOverviewEntry
schema: start_*_seconds, end_*_seconds, duration_*_seconds, and their median
counterparts were moved out of the 'required' list in Pydantic v2. The auto-generated
TS type now marks these fields as 'number | null | undefined'.
The hand-written formatTooltipBody in scheduleOverviewUtils.ts only accepted
'number | null', causing:
error TS2345 in ScheduleOverviewRow.tsx:44
Argument of type 'DagScheduleOverviewEntry' is not assignable to parameter
of type '{ readonly end_mean_seconds: number | null; ... }'
Types of property 'end_mean_seconds' are incompatible.
Type 'number | null | undefined' is not assignable to type 'number | null'.
Widen the formatTooltipBody parameter to allow 'undefined' for the now-optional
duration/start/end fields. The downstream helpers (formatSecondsAsClock,
formatDurationSeconds) already accept undefined and render it as em-dash.
Local verification:
- prek run ts-compile-lint-ui --files <changed file>: Passed
- prek run --files <changed file>: Passed
…nd params
The previous commit added '| undefined' to the duration/start/end types in
formatTooltipBody's parameter, but TypeScript's structural typing is stricter
about optional vs required properties. Since DagScheduleOverviewEntry marks
end_mean_seconds / end_median_seconds / start_mean_seconds / start_median_seconds
as '?: number | null' (truly optional, undefined comes from optionality, not
the value type), the parameter must mirror that with '?:' on the same fields.
'?: number | null' and ': number | null | undefined' are NOT structurally
assignable in either direction under TypeScript's strictness here.
Use '?: number | null' to match the auto-generated type shape. The
downstream helpers (formatSecondsAsClock, formatDurationSeconds) still
handle the undefined case and render em-dash.
Local verification:
- pnpm tsc --noEmit -p tsconfig.app.json: clean
- prek run --files <changed file>: Passed
- prek run ts-compile-lint-ui --files <changed file>: Passed
@anmolxlight

anmolxlight commented Jun 15, 2026

Copy link
Copy Markdown
ContributorAuthor
VID_20260615_232836_381.mp4

Screen recording of the new Schedule Overview page:

Shows the /dags/schedule page with the 24h UTC timeline bars, hover tooltips, and the filter working.

@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from a60f1d5 to b3353f0CompareJune 15, 2026 17:54

@choo121600choo121600 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the screen recording you attached, this appears to be somewhat inconsistent with the existing UI patterns.

I think we need more discussion in the issue before moving forward with an implementation. In particular, we should first align on whether this feature should be included at all, and if so, how it should be integrated into the UI (tab placement, design, UX, and overall approach).

Could you please close this PR for now? I'd prefer to continue the discussion in the issue first and revisit the implementation once we've reached agreement on the design.

Once we've aligned on the direction in the issue, I'd be happy to see this PR reopened.

@potiukpotiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jun 17, 2026
@jroachgolf84

Copy link
Copy Markdown
Collaborator

I don't think that this is a feature that should be implemented. See my comment from the issue:

Not quite sure how this is useful:

typical (mean / median) start and end time of each Dag over a 24-hour timeline

If I run a DAG hourly, that would make the average noon. That's pretty useless IMO. Plus, I don't think we could make this robust enough to handle all the scheduling use-cases that Airflow offers.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP APIarea:translationsarea:UIRelated to UI/UX. For Frontend Developers.ready for maintainer reviewSet after triaging when all criteria pass.translation:default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an aggregate Dag schedule view: typical daily run times across all Dags

4 participants

@anmolxlight@choo121600@jroachgolf84@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(ui): add aggregate Dag schedule overview view (#68532) - #68547

Closed
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532
Closed

feat(ui): add aggregate Dag schedule overview view (#68532)#68547
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532

Conversation

@anmolxlight

Copy link
Copy Markdown
Contributor

Fixes#68532

What

Adds a new /dags/schedule page that visualizes the typical (mean / median) start and end time of each Dag over a 24-hour timeline, aggregated from recent successful Dag runs. This complements the existing per-Dag Gantt view, which only shows a single run at a time. The new view operates at a different level: it aggregates across all Dags to answer "across my whole deployment, when during the day does each Dag usually run?"

Why

Operators running many Dags currently have no at-a-glance way to see how scheduled load is distributed across the day. The new view makes the whole schedule visible at once, which helps to:

  • spot overlapping heavy windows where many Dags pile up at the same hour,
  • find quiet windows for maintenance / deploys / resource-intensive jobs,
  • notice drift when a Dag's typical run time shifts over recent runs,
  • plan capacity and SLAs across the fleet.

How

Backend

  • New /ui/dag_schedule_overview endpoint (in api_fastapi/core_api/routes/ui/dag_schedule_overview.py) gated by requires_access_dag(method="GET", access_entity=DagAccessEntity.RUN).
  • New DagScheduleOverviewService (api_fastapi/core_api/services/ui/dag_schedule_overview.py) that:
    • fetches all DagModel rows (so dags with no runs still appear as zero-statistic rows),
    • pulls up to 200 most-recent successful DagRuns per dag, optionally scoped by a run_after window,
    • computes mean/median of start_date and end_date mapped to seconds-of-day in UTC, plus mean/median duration,
    • supports dag_id_pattern and dag_display_name_pattern filters.
  • New datamodels DagScheduleOverviewEntry + DagScheduleOverviewCollectionResponse (api_fastapi/core_api/datamodels/ui/dag_schedule_overview.py).
  • 11 backend tests in tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py: 200/401/403, all-three-dag fixture, morning/evening/no-runs statistics, FAILED-run exclusion, pattern filters, run_after window edges.
  • OpenAPI _private_ui.yaml updated; pnpm codegen regenerated.

Frontend

  • New page src/pages/DagsList/ScheduleOverview/ mounted at /dags/schedule with:
    • one row per Dag (linkable to the Dag detail page),
    • a 24h tick rail at 0/3/6/9/12/15/18/21/24 hours,
    • a horizontal bar showing the mean start→end window,
    • a hover tooltip with mean and median start / end, mean and median duration, and the recent-runs date range,
    • a client-side dag-id filter input.
  • New sidebar nav button Schedule Overview (FiBarChart2) on the main nav, alongside Dags.
  • New i18n keys under dags.scheduleOverview.* and common.nav.scheduleOverview (English locale).

Verification

# Backend
uv run --project airflow-core pytest airflow-core/tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py --with-db-init
# 11 passed
uv run --project airflow-core ruff check + ruff format --check + mypy
# All clean
# Frontend
pnpm lint # eslint --quiet && tsc -p tsconfig.app.json — passes
pnpm prettier --check public/i18n/locales/en/dags.json public/i18n/locales/en/common.json
# Passes

The two pre-existing vitest failures in useTagFilter.test.tsx and Logs.test.tsx reproduce on main (verified with git stash), so they are unrelated to this PR.

@boring-cyborgboring-cyborgBot added area:API Airflow's REST/HTTP API area:translations area:UI Related to UI/UX. For Frontend Developers. translation:default labels Jun 14, 2026
@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from 64767cb to a33d4bdCompareJune 14, 2026 22:49
anmolxlight added a commit to anmolxlight/airflow that referenced this pull request Jun 15, 2026
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
@choo121600

Copy link
Copy Markdown
Member

Could you please attach a short recording of this feature following the PR template? It would be helpful for the review.

Adds a 24h Gantt-style view that answers the question `across my whole
deployment, when during the day does each Dag usually run?` — a
complement to the per-Dag Gantt view, which only shows a single run.
Backend
* New `/ui/dag_schedule_overview` endpoint that aggregates the
most-recent successful Dag runs per Dag and returns mean/median
start/end times-of-day in UTC, plus duration statistics.
* Optional `run_after_gte` / `run_after_lte` window and
`dag_id_pattern` / `dag_display_name_pattern` filters.
* 11 backend tests covering 200/401/403, all-three-dag fixture,
morning/evening/no-runs statistics, FAILED-run exclusion, and the
filter / window edge cases.
* OpenAPI YAML updated, `pnpm codegen` regenerated.
Frontend
* New `/dags/schedule` page (one row per Dag, mean start→end bar
across a 24h tick rail, hover tooltip with mean+median start, end,
duration, and recent-runs range).
* New `Schedule Overview` sidebar nav link.
* i18n keys for English locale.
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
The OpenAPI spec regen in the previous commit changed the DagScheduleOverviewEntry
schema: start_*_seconds, end_*_seconds, duration_*_seconds, and their median
counterparts were moved out of the 'required' list in Pydantic v2. The auto-generated
TS type now marks these fields as 'number | null | undefined'.
The hand-written formatTooltipBody in scheduleOverviewUtils.ts only accepted
'number | null', causing:
error TS2345 in ScheduleOverviewRow.tsx:44
Argument of type 'DagScheduleOverviewEntry' is not assignable to parameter
of type '{ readonly end_mean_seconds: number | null; ... }'
Types of property 'end_mean_seconds' are incompatible.
Type 'number | null | undefined' is not assignable to type 'number | null'.
Widen the formatTooltipBody parameter to allow 'undefined' for the now-optional
duration/start/end fields. The downstream helpers (formatSecondsAsClock,
formatDurationSeconds) already accept undefined and render it as em-dash.
Local verification:
- prek run ts-compile-lint-ui --files <changed file>: Passed
- prek run --files <changed file>: Passed
…nd params
The previous commit added '| undefined' to the duration/start/end types in
formatTooltipBody's parameter, but TypeScript's structural typing is stricter
about optional vs required properties. Since DagScheduleOverviewEntry marks
end_mean_seconds / end_median_seconds / start_mean_seconds / start_median_seconds
as '?: number | null' (truly optional, undefined comes from optionality, not
the value type), the parameter must mirror that with '?:' on the same fields.
'?: number | null' and ': number | null | undefined' are NOT structurally
assignable in either direction under TypeScript's strictness here.
Use '?: number | null' to match the auto-generated type shape. The
downstream helpers (formatSecondsAsClock, formatDurationSeconds) still
handle the undefined case and render em-dash.
Local verification:
- pnpm tsc --noEmit -p tsconfig.app.json: clean
- prek run --files <changed file>: Passed
- prek run ts-compile-lint-ui --files <changed file>: Passed
@anmolxlight

anmolxlight commented Jun 15, 2026

Copy link
Copy Markdown
ContributorAuthor
VID_20260615_232836_381.mp4

Screen recording of the new Schedule Overview page:

Shows the /dags/schedule page with the 24h UTC timeline bars, hover tooltips, and the filter working.

@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from a60f1d5 to b3353f0CompareJune 15, 2026 17:54

@choo121600choo121600 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the screen recording you attached, this appears to be somewhat inconsistent with the existing UI patterns.

I think we need more discussion in the issue before moving forward with an implementation. In particular, we should first align on whether this feature should be included at all, and if so, how it should be integrated into the UI (tab placement, design, UX, and overall approach).

Could you please close this PR for now? I'd prefer to continue the discussion in the issue first and revisit the implementation once we've reached agreement on the design.

Once we've aligned on the direction in the issue, I'd be happy to see this PR reopened.

@potiukpotiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jun 17, 2026
@jroachgolf84

Copy link
Copy Markdown
Collaborator

I don't think that this is a feature that should be implemented. See my comment from the issue:

Not quite sure how this is useful:

typical (mean / median) start and end time of each Dag over a 24-hour timeline

If I run a DAG hourly, that would make the average noon. That's pretty useless IMO. Plus, I don't think we could make this robust enough to handle all the scheduling use-cases that Airflow offers.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP APIarea:translationsarea:UIRelated to UI/UX. For Frontend Developers.ready for maintainer reviewSet after triaging when all criteria pass.translation:default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an aggregate Dag schedule view: typical daily run times across all Dags

4 participants

@anmolxlight@choo121600@jroachgolf84@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

feat(ui): add aggregate Dag schedule overview view (#68532) - #68547

Closed
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532
Closed

feat(ui): add aggregate Dag schedule overview view (#68532)#68547
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532

Conversation

@anmolxlight

Copy link
Copy Markdown
Contributor

Fixes#68532

What

Adds a new /dags/schedule page that visualizes the typical (mean / median) start and end time of each Dag over a 24-hour timeline, aggregated from recent successful Dag runs. This complements the existing per-Dag Gantt view, which only shows a single run at a time. The new view operates at a different level: it aggregates across all Dags to answer "across my whole deployment, when during the day does each Dag usually run?"

Why

Operators running many Dags currently have no at-a-glance way to see how scheduled load is distributed across the day. The new view makes the whole schedule visible at once, which helps to:

  • spot overlapping heavy windows where many Dags pile up at the same hour,
  • find quiet windows for maintenance / deploys / resource-intensive jobs,
  • notice drift when a Dag's typical run time shifts over recent runs,
  • plan capacity and SLAs across the fleet.

How

Backend

  • New /ui/dag_schedule_overview endpoint (in api_fastapi/core_api/routes/ui/dag_schedule_overview.py) gated by requires_access_dag(method="GET", access_entity=DagAccessEntity.RUN).
  • New DagScheduleOverviewService (api_fastapi/core_api/services/ui/dag_schedule_overview.py) that:
    • fetches all DagModel rows (so dags with no runs still appear as zero-statistic rows),
    • pulls up to 200 most-recent successful DagRuns per dag, optionally scoped by a run_after window,
    • computes mean/median of start_date and end_date mapped to seconds-of-day in UTC, plus mean/median duration,
    • supports dag_id_pattern and dag_display_name_pattern filters.
  • New datamodels DagScheduleOverviewEntry + DagScheduleOverviewCollectionResponse (api_fastapi/core_api/datamodels/ui/dag_schedule_overview.py).
  • 11 backend tests in tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py: 200/401/403, all-three-dag fixture, morning/evening/no-runs statistics, FAILED-run exclusion, pattern filters, run_after window edges.
  • OpenAPI _private_ui.yaml updated; pnpm codegen regenerated.

Frontend

  • New page src/pages/DagsList/ScheduleOverview/ mounted at /dags/schedule with:
    • one row per Dag (linkable to the Dag detail page),
    • a 24h tick rail at 0/3/6/9/12/15/18/21/24 hours,
    • a horizontal bar showing the mean start→end window,
    • a hover tooltip with mean and median start / end, mean and median duration, and the recent-runs date range,
    • a client-side dag-id filter input.
  • New sidebar nav button Schedule Overview (FiBarChart2) on the main nav, alongside Dags.
  • New i18n keys under dags.scheduleOverview.* and common.nav.scheduleOverview (English locale).

Verification

# Backend
uv run --project airflow-core pytest airflow-core/tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py --with-db-init
# 11 passed
uv run --project airflow-core ruff check + ruff format --check + mypy
# All clean
# Frontend
pnpm lint # eslint --quiet && tsc -p tsconfig.app.json — passes
pnpm prettier --check public/i18n/locales/en/dags.json public/i18n/locales/en/common.json
# Passes

The two pre-existing vitest failures in useTagFilter.test.tsx and Logs.test.tsx reproduce on main (verified with git stash), so they are unrelated to this PR.

@boring-cyborgboring-cyborgBot added area:API Airflow's REST/HTTP API area:translations area:UI Related to UI/UX. For Frontend Developers. translation:default labels Jun 14, 2026
@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from 64767cb to a33d4bdCompareJune 14, 2026 22:49
anmolxlight added a commit to anmolxlight/airflow that referenced this pull request Jun 15, 2026
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
@choo121600

Copy link
Copy Markdown
Member

Could you please attach a short recording of this feature following the PR template? It would be helpful for the review.

Adds a 24h Gantt-style view that answers the question `across my whole
deployment, when during the day does each Dag usually run?` — a
complement to the per-Dag Gantt view, which only shows a single run.
Backend
* New `/ui/dag_schedule_overview` endpoint that aggregates the
most-recent successful Dag runs per Dag and returns mean/median
start/end times-of-day in UTC, plus duration statistics.
* Optional `run_after_gte` / `run_after_lte` window and
`dag_id_pattern` / `dag_display_name_pattern` filters.
* 11 backend tests covering 200/401/403, all-three-dag fixture,
morning/evening/no-runs statistics, FAILED-run exclusion, and the
filter / window edge cases.
* OpenAPI YAML updated, `pnpm codegen` regenerated.
Frontend
* New `/dags/schedule` page (one row per Dag, mean start→end bar
across a 24h tick rail, hover tooltip with mean+median start, end,
duration, and recent-runs range).
* New `Schedule Overview` sidebar nav link.
* i18n keys for English locale.
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
The OpenAPI spec regen in the previous commit changed the DagScheduleOverviewEntry
schema: start_*_seconds, end_*_seconds, duration_*_seconds, and their median
counterparts were moved out of the 'required' list in Pydantic v2. The auto-generated
TS type now marks these fields as 'number | null | undefined'.
The hand-written formatTooltipBody in scheduleOverviewUtils.ts only accepted
'number | null', causing:
error TS2345 in ScheduleOverviewRow.tsx:44
Argument of type 'DagScheduleOverviewEntry' is not assignable to parameter
of type '{ readonly end_mean_seconds: number | null; ... }'
Types of property 'end_mean_seconds' are incompatible.
Type 'number | null | undefined' is not assignable to type 'number | null'.
Widen the formatTooltipBody parameter to allow 'undefined' for the now-optional
duration/start/end fields. The downstream helpers (formatSecondsAsClock,
formatDurationSeconds) already accept undefined and render it as em-dash.
Local verification:
- prek run ts-compile-lint-ui --files <changed file>: Passed
- prek run --files <changed file>: Passed
…nd params
The previous commit added '| undefined' to the duration/start/end types in
formatTooltipBody's parameter, but TypeScript's structural typing is stricter
about optional vs required properties. Since DagScheduleOverviewEntry marks
end_mean_seconds / end_median_seconds / start_mean_seconds / start_median_seconds
as '?: number | null' (truly optional, undefined comes from optionality, not
the value type), the parameter must mirror that with '?:' on the same fields.
'?: number | null' and ': number | null | undefined' are NOT structurally
assignable in either direction under TypeScript's strictness here.
Use '?: number | null' to match the auto-generated type shape. The
downstream helpers (formatSecondsAsClock, formatDurationSeconds) still
handle the undefined case and render em-dash.
Local verification:
- pnpm tsc --noEmit -p tsconfig.app.json: clean
- prek run --files <changed file>: Passed
- prek run ts-compile-lint-ui --files <changed file>: Passed
@anmolxlight

anmolxlight commented Jun 15, 2026

Copy link
Copy Markdown
ContributorAuthor
VID_20260615_232836_381.mp4

Screen recording of the new Schedule Overview page:

Shows the /dags/schedule page with the 24h UTC timeline bars, hover tooltips, and the filter working.

@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from a60f1d5 to b3353f0CompareJune 15, 2026 17:54

@choo121600choo121600 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the screen recording you attached, this appears to be somewhat inconsistent with the existing UI patterns.

I think we need more discussion in the issue before moving forward with an implementation. In particular, we should first align on whether this feature should be included at all, and if so, how it should be integrated into the UI (tab placement, design, UX, and overall approach).

Could you please close this PR for now? I'd prefer to continue the discussion in the issue first and revisit the implementation once we've reached agreement on the design.

Once we've aligned on the direction in the issue, I'd be happy to see this PR reopened.

@potiukpotiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jun 17, 2026
@jroachgolf84

Copy link
Copy Markdown
Collaborator

I don't think that this is a feature that should be implemented. See my comment from the issue:

Not quite sure how this is useful:

typical (mean / median) start and end time of each Dag over a 24-hour timeline

If I run a DAG hourly, that would make the average noon. That's pretty useless IMO. Plus, I don't think we could make this robust enough to handle all the scheduling use-cases that Airflow offers.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP APIarea:translationsarea:UIRelated to UI/UX. For Frontend Developers.ready for maintainer reviewSet after triaging when all criteria pass.translation:default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an aggregate Dag schedule view: typical daily run times across all Dags

4 participants

@anmolxlight@choo121600@jroachgolf84@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(ui): add aggregate Dag schedule overview view (#68532) - #68547

Closed
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532
Closed

feat(ui): add aggregate Dag schedule overview view (#68532)#68547
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532

Conversation

@anmolxlight

Copy link
Copy Markdown
Contributor

Fixes#68532

What

Adds a new /dags/schedule page that visualizes the typical (mean / median) start and end time of each Dag over a 24-hour timeline, aggregated from recent successful Dag runs. This complements the existing per-Dag Gantt view, which only shows a single run at a time. The new view operates at a different level: it aggregates across all Dags to answer "across my whole deployment, when during the day does each Dag usually run?"

Why

Operators running many Dags currently have no at-a-glance way to see how scheduled load is distributed across the day. The new view makes the whole schedule visible at once, which helps to:

  • spot overlapping heavy windows where many Dags pile up at the same hour,
  • find quiet windows for maintenance / deploys / resource-intensive jobs,
  • notice drift when a Dag's typical run time shifts over recent runs,
  • plan capacity and SLAs across the fleet.

How

Backend

  • New /ui/dag_schedule_overview endpoint (in api_fastapi/core_api/routes/ui/dag_schedule_overview.py) gated by requires_access_dag(method="GET", access_entity=DagAccessEntity.RUN).
  • New DagScheduleOverviewService (api_fastapi/core_api/services/ui/dag_schedule_overview.py) that:
    • fetches all DagModel rows (so dags with no runs still appear as zero-statistic rows),
    • pulls up to 200 most-recent successful DagRuns per dag, optionally scoped by a run_after window,
    • computes mean/median of start_date and end_date mapped to seconds-of-day in UTC, plus mean/median duration,
    • supports dag_id_pattern and dag_display_name_pattern filters.
  • New datamodels DagScheduleOverviewEntry + DagScheduleOverviewCollectionResponse (api_fastapi/core_api/datamodels/ui/dag_schedule_overview.py).
  • 11 backend tests in tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py: 200/401/403, all-three-dag fixture, morning/evening/no-runs statistics, FAILED-run exclusion, pattern filters, run_after window edges.
  • OpenAPI _private_ui.yaml updated; pnpm codegen regenerated.

Frontend

  • New page src/pages/DagsList/ScheduleOverview/ mounted at /dags/schedule with:
    • one row per Dag (linkable to the Dag detail page),
    • a 24h tick rail at 0/3/6/9/12/15/18/21/24 hours,
    • a horizontal bar showing the mean start→end window,
    • a hover tooltip with mean and median start / end, mean and median duration, and the recent-runs date range,
    • a client-side dag-id filter input.
  • New sidebar nav button Schedule Overview (FiBarChart2) on the main nav, alongside Dags.
  • New i18n keys under dags.scheduleOverview.* and common.nav.scheduleOverview (English locale).

Verification

# Backend
uv run --project airflow-core pytest airflow-core/tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py --with-db-init
# 11 passed
uv run --project airflow-core ruff check + ruff format --check + mypy
# All clean
# Frontend
pnpm lint # eslint --quiet && tsc -p tsconfig.app.json — passes
pnpm prettier --check public/i18n/locales/en/dags.json public/i18n/locales/en/common.json
# Passes

The two pre-existing vitest failures in useTagFilter.test.tsx and Logs.test.tsx reproduce on main (verified with git stash), so they are unrelated to this PR.

@boring-cyborgboring-cyborgBot added area:API Airflow's REST/HTTP API area:translations area:UI Related to UI/UX. For Frontend Developers. translation:default labels Jun 14, 2026
@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from 64767cb to a33d4bdCompareJune 14, 2026 22:49
anmolxlight added a commit to anmolxlight/airflow that referenced this pull request Jun 15, 2026
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
@choo121600

Copy link
Copy Markdown
Member

Could you please attach a short recording of this feature following the PR template? It would be helpful for the review.

Adds a 24h Gantt-style view that answers the question `across my whole
deployment, when during the day does each Dag usually run?` — a
complement to the per-Dag Gantt view, which only shows a single run.
Backend
* New `/ui/dag_schedule_overview` endpoint that aggregates the
most-recent successful Dag runs per Dag and returns mean/median
start/end times-of-day in UTC, plus duration statistics.
* Optional `run_after_gte` / `run_after_lte` window and
`dag_id_pattern` / `dag_display_name_pattern` filters.
* 11 backend tests covering 200/401/403, all-three-dag fixture,
morning/evening/no-runs statistics, FAILED-run exclusion, and the
filter / window edge cases.
* OpenAPI YAML updated, `pnpm codegen` regenerated.
Frontend
* New `/dags/schedule` page (one row per Dag, mean start→end bar
across a 24h tick rail, hover tooltip with mean+median start, end,
duration, and recent-runs range).
* New `Schedule Overview` sidebar nav link.
* i18n keys for English locale.
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
The OpenAPI spec regen in the previous commit changed the DagScheduleOverviewEntry
schema: start_*_seconds, end_*_seconds, duration_*_seconds, and their median
counterparts were moved out of the 'required' list in Pydantic v2. The auto-generated
TS type now marks these fields as 'number | null | undefined'.
The hand-written formatTooltipBody in scheduleOverviewUtils.ts only accepted
'number | null', causing:
error TS2345 in ScheduleOverviewRow.tsx:44
Argument of type 'DagScheduleOverviewEntry' is not assignable to parameter
of type '{ readonly end_mean_seconds: number | null; ... }'
Types of property 'end_mean_seconds' are incompatible.
Type 'number | null | undefined' is not assignable to type 'number | null'.
Widen the formatTooltipBody parameter to allow 'undefined' for the now-optional
duration/start/end fields. The downstream helpers (formatSecondsAsClock,
formatDurationSeconds) already accept undefined and render it as em-dash.
Local verification:
- prek run ts-compile-lint-ui --files <changed file>: Passed
- prek run --files <changed file>: Passed
…nd params
The previous commit added '| undefined' to the duration/start/end types in
formatTooltipBody's parameter, but TypeScript's structural typing is stricter
about optional vs required properties. Since DagScheduleOverviewEntry marks
end_mean_seconds / end_median_seconds / start_mean_seconds / start_median_seconds
as '?: number | null' (truly optional, undefined comes from optionality, not
the value type), the parameter must mirror that with '?:' on the same fields.
'?: number | null' and ': number | null | undefined' are NOT structurally
assignable in either direction under TypeScript's strictness here.
Use '?: number | null' to match the auto-generated type shape. The
downstream helpers (formatSecondsAsClock, formatDurationSeconds) still
handle the undefined case and render em-dash.
Local verification:
- pnpm tsc --noEmit -p tsconfig.app.json: clean
- prek run --files <changed file>: Passed
- prek run ts-compile-lint-ui --files <changed file>: Passed
@anmolxlight

anmolxlight commented Jun 15, 2026

Copy link
Copy Markdown
ContributorAuthor
VID_20260615_232836_381.mp4

Screen recording of the new Schedule Overview page:

Shows the /dags/schedule page with the 24h UTC timeline bars, hover tooltips, and the filter working.

@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from a60f1d5 to b3353f0CompareJune 15, 2026 17:54

@choo121600choo121600 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the screen recording you attached, this appears to be somewhat inconsistent with the existing UI patterns.

I think we need more discussion in the issue before moving forward with an implementation. In particular, we should first align on whether this feature should be included at all, and if so, how it should be integrated into the UI (tab placement, design, UX, and overall approach).

Could you please close this PR for now? I'd prefer to continue the discussion in the issue first and revisit the implementation once we've reached agreement on the design.

Once we've aligned on the direction in the issue, I'd be happy to see this PR reopened.

@potiukpotiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jun 17, 2026
@jroachgolf84

Copy link
Copy Markdown
Collaborator

I don't think that this is a feature that should be implemented. See my comment from the issue:

Not quite sure how this is useful:

typical (mean / median) start and end time of each Dag over a 24-hour timeline

If I run a DAG hourly, that would make the average noon. That's pretty useless IMO. Plus, I don't think we could make this robust enough to handle all the scheduling use-cases that Airflow offers.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP APIarea:translationsarea:UIRelated to UI/UX. For Frontend Developers.ready for maintainer reviewSet after triaging when all criteria pass.translation:default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an aggregate Dag schedule view: typical daily run times across all Dags

4 participants

@anmolxlight@choo121600@jroachgolf84@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(ui): add aggregate Dag schedule overview view (#68532) - #68547

Closed
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532
Closed

feat(ui): add aggregate Dag schedule overview view (#68532)#68547
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532

Conversation

@anmolxlight

Copy link
Copy Markdown
Contributor

Fixes#68532

What

Adds a new /dags/schedule page that visualizes the typical (mean / median) start and end time of each Dag over a 24-hour timeline, aggregated from recent successful Dag runs. This complements the existing per-Dag Gantt view, which only shows a single run at a time. The new view operates at a different level: it aggregates across all Dags to answer "across my whole deployment, when during the day does each Dag usually run?"

Why

Operators running many Dags currently have no at-a-glance way to see how scheduled load is distributed across the day. The new view makes the whole schedule visible at once, which helps to:

  • spot overlapping heavy windows where many Dags pile up at the same hour,
  • find quiet windows for maintenance / deploys / resource-intensive jobs,
  • notice drift when a Dag's typical run time shifts over recent runs,
  • plan capacity and SLAs across the fleet.

How

Backend

  • New /ui/dag_schedule_overview endpoint (in api_fastapi/core_api/routes/ui/dag_schedule_overview.py) gated by requires_access_dag(method="GET", access_entity=DagAccessEntity.RUN).
  • New DagScheduleOverviewService (api_fastapi/core_api/services/ui/dag_schedule_overview.py) that:
    • fetches all DagModel rows (so dags with no runs still appear as zero-statistic rows),
    • pulls up to 200 most-recent successful DagRuns per dag, optionally scoped by a run_after window,
    • computes mean/median of start_date and end_date mapped to seconds-of-day in UTC, plus mean/median duration,
    • supports dag_id_pattern and dag_display_name_pattern filters.
  • New datamodels DagScheduleOverviewEntry + DagScheduleOverviewCollectionResponse (api_fastapi/core_api/datamodels/ui/dag_schedule_overview.py).
  • 11 backend tests in tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py: 200/401/403, all-three-dag fixture, morning/evening/no-runs statistics, FAILED-run exclusion, pattern filters, run_after window edges.
  • OpenAPI _private_ui.yaml updated; pnpm codegen regenerated.

Frontend

  • New page src/pages/DagsList/ScheduleOverview/ mounted at /dags/schedule with:
    • one row per Dag (linkable to the Dag detail page),
    • a 24h tick rail at 0/3/6/9/12/15/18/21/24 hours,
    • a horizontal bar showing the mean start→end window,
    • a hover tooltip with mean and median start / end, mean and median duration, and the recent-runs date range,
    • a client-side dag-id filter input.
  • New sidebar nav button Schedule Overview (FiBarChart2) on the main nav, alongside Dags.
  • New i18n keys under dags.scheduleOverview.* and common.nav.scheduleOverview (English locale).

Verification

# Backend
uv run --project airflow-core pytest airflow-core/tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py --with-db-init
# 11 passed
uv run --project airflow-core ruff check + ruff format --check + mypy
# All clean
# Frontend
pnpm lint # eslint --quiet && tsc -p tsconfig.app.json — passes
pnpm prettier --check public/i18n/locales/en/dags.json public/i18n/locales/en/common.json
# Passes

The two pre-existing vitest failures in useTagFilter.test.tsx and Logs.test.tsx reproduce on main (verified with git stash), so they are unrelated to this PR.

@boring-cyborgboring-cyborgBot added area:API Airflow's REST/HTTP API area:translations area:UI Related to UI/UX. For Frontend Developers. translation:default labels Jun 14, 2026
@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from 64767cb to a33d4bdCompareJune 14, 2026 22:49
anmolxlight added a commit to anmolxlight/airflow that referenced this pull request Jun 15, 2026
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
@choo121600

Copy link
Copy Markdown
Member

Could you please attach a short recording of this feature following the PR template? It would be helpful for the review.

Adds a 24h Gantt-style view that answers the question `across my whole
deployment, when during the day does each Dag usually run?` — a
complement to the per-Dag Gantt view, which only shows a single run.
Backend
* New `/ui/dag_schedule_overview` endpoint that aggregates the
most-recent successful Dag runs per Dag and returns mean/median
start/end times-of-day in UTC, plus duration statistics.
* Optional `run_after_gte` / `run_after_lte` window and
`dag_id_pattern` / `dag_display_name_pattern` filters.
* 11 backend tests covering 200/401/403, all-three-dag fixture,
morning/evening/no-runs statistics, FAILED-run exclusion, and the
filter / window edge cases.
* OpenAPI YAML updated, `pnpm codegen` regenerated.
Frontend
* New `/dags/schedule` page (one row per Dag, mean start→end bar
across a 24h tick rail, hover tooltip with mean+median start, end,
duration, and recent-runs range).
* New `Schedule Overview` sidebar nav link.
* i18n keys for English locale.
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
The OpenAPI spec regen in the previous commit changed the DagScheduleOverviewEntry
schema: start_*_seconds, end_*_seconds, duration_*_seconds, and their median
counterparts were moved out of the 'required' list in Pydantic v2. The auto-generated
TS type now marks these fields as 'number | null | undefined'.
The hand-written formatTooltipBody in scheduleOverviewUtils.ts only accepted
'number | null', causing:
error TS2345 in ScheduleOverviewRow.tsx:44
Argument of type 'DagScheduleOverviewEntry' is not assignable to parameter
of type '{ readonly end_mean_seconds: number | null; ... }'
Types of property 'end_mean_seconds' are incompatible.
Type 'number | null | undefined' is not assignable to type 'number | null'.
Widen the formatTooltipBody parameter to allow 'undefined' for the now-optional
duration/start/end fields. The downstream helpers (formatSecondsAsClock,
formatDurationSeconds) already accept undefined and render it as em-dash.
Local verification:
- prek run ts-compile-lint-ui --files <changed file>: Passed
- prek run --files <changed file>: Passed
…nd params
The previous commit added '| undefined' to the duration/start/end types in
formatTooltipBody's parameter, but TypeScript's structural typing is stricter
about optional vs required properties. Since DagScheduleOverviewEntry marks
end_mean_seconds / end_median_seconds / start_mean_seconds / start_median_seconds
as '?: number | null' (truly optional, undefined comes from optionality, not
the value type), the parameter must mirror that with '?:' on the same fields.
'?: number | null' and ': number | null | undefined' are NOT structurally
assignable in either direction under TypeScript's strictness here.
Use '?: number | null' to match the auto-generated type shape. The
downstream helpers (formatSecondsAsClock, formatDurationSeconds) still
handle the undefined case and render em-dash.
Local verification:
- pnpm tsc --noEmit -p tsconfig.app.json: clean
- prek run --files <changed file>: Passed
- prek run ts-compile-lint-ui --files <changed file>: Passed
@anmolxlight

anmolxlight commented Jun 15, 2026

Copy link
Copy Markdown
ContributorAuthor
VID_20260615_232836_381.mp4

Screen recording of the new Schedule Overview page:

Shows the /dags/schedule page with the 24h UTC timeline bars, hover tooltips, and the filter working.

@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from a60f1d5 to b3353f0CompareJune 15, 2026 17:54

@choo121600choo121600 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the screen recording you attached, this appears to be somewhat inconsistent with the existing UI patterns.

I think we need more discussion in the issue before moving forward with an implementation. In particular, we should first align on whether this feature should be included at all, and if so, how it should be integrated into the UI (tab placement, design, UX, and overall approach).

Could you please close this PR for now? I'd prefer to continue the discussion in the issue first and revisit the implementation once we've reached agreement on the design.

Once we've aligned on the direction in the issue, I'd be happy to see this PR reopened.

@potiukpotiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jun 17, 2026
@jroachgolf84

Copy link
Copy Markdown
Collaborator

I don't think that this is a feature that should be implemented. See my comment from the issue:

Not quite sure how this is useful:

typical (mean / median) start and end time of each Dag over a 24-hour timeline

If I run a DAG hourly, that would make the average noon. That's pretty useless IMO. Plus, I don't think we could make this robust enough to handle all the scheduling use-cases that Airflow offers.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP APIarea:translationsarea:UIRelated to UI/UX. For Frontend Developers.ready for maintainer reviewSet after triaging when all criteria pass.translation:default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an aggregate Dag schedule view: typical daily run times across all Dags

4 participants

@anmolxlight@choo121600@jroachgolf84@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

feat(ui): add aggregate Dag schedule overview view (#68532) - #68547

Closed
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532
Closed

feat(ui): add aggregate Dag schedule overview view (#68532)#68547
anmolxlight wants to merge 4 commits into
apache:mainfrom
anmolxlight:feature/dag-schedule-overview-68532

Conversation

@anmolxlight

Copy link
Copy Markdown
Contributor

Fixes#68532

What

Adds a new /dags/schedule page that visualizes the typical (mean / median) start and end time of each Dag over a 24-hour timeline, aggregated from recent successful Dag runs. This complements the existing per-Dag Gantt view, which only shows a single run at a time. The new view operates at a different level: it aggregates across all Dags to answer "across my whole deployment, when during the day does each Dag usually run?"

Why

Operators running many Dags currently have no at-a-glance way to see how scheduled load is distributed across the day. The new view makes the whole schedule visible at once, which helps to:

  • spot overlapping heavy windows where many Dags pile up at the same hour,
  • find quiet windows for maintenance / deploys / resource-intensive jobs,
  • notice drift when a Dag's typical run time shifts over recent runs,
  • plan capacity and SLAs across the fleet.

How

Backend

  • New /ui/dag_schedule_overview endpoint (in api_fastapi/core_api/routes/ui/dag_schedule_overview.py) gated by requires_access_dag(method="GET", access_entity=DagAccessEntity.RUN).
  • New DagScheduleOverviewService (api_fastapi/core_api/services/ui/dag_schedule_overview.py) that:
    • fetches all DagModel rows (so dags with no runs still appear as zero-statistic rows),
    • pulls up to 200 most-recent successful DagRuns per dag, optionally scoped by a run_after window,
    • computes mean/median of start_date and end_date mapped to seconds-of-day in UTC, plus mean/median duration,
    • supports dag_id_pattern and dag_display_name_pattern filters.
  • New datamodels DagScheduleOverviewEntry + DagScheduleOverviewCollectionResponse (api_fastapi/core_api/datamodels/ui/dag_schedule_overview.py).
  • 11 backend tests in tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py: 200/401/403, all-three-dag fixture, morning/evening/no-runs statistics, FAILED-run exclusion, pattern filters, run_after window edges.
  • OpenAPI _private_ui.yaml updated; pnpm codegen regenerated.

Frontend

  • New page src/pages/DagsList/ScheduleOverview/ mounted at /dags/schedule with:
    • one row per Dag (linkable to the Dag detail page),
    • a 24h tick rail at 0/3/6/9/12/15/18/21/24 hours,
    • a horizontal bar showing the mean start→end window,
    • a hover tooltip with mean and median start / end, mean and median duration, and the recent-runs date range,
    • a client-side dag-id filter input.
  • New sidebar nav button Schedule Overview (FiBarChart2) on the main nav, alongside Dags.
  • New i18n keys under dags.scheduleOverview.* and common.nav.scheduleOverview (English locale).

Verification

# Backend
uv run --project airflow-core pytest airflow-core/tests/unit/api_fastapi/core_api/routes/ui/test_dag_schedule_overview.py --with-db-init
# 11 passed
uv run --project airflow-core ruff check + ruff format --check + mypy
# All clean
# Frontend
pnpm lint # eslint --quiet && tsc -p tsconfig.app.json — passes
pnpm prettier --check public/i18n/locales/en/dags.json public/i18n/locales/en/common.json
# Passes

The two pre-existing vitest failures in useTagFilter.test.tsx and Logs.test.tsx reproduce on main (verified with git stash), so they are unrelated to this PR.

@boring-cyborgboring-cyborgBot added area:API Airflow's REST/HTTP API area:translations area:UI Related to UI/UX. For Frontend Developers. translation:default labels Jun 14, 2026
@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from 64767cb to a33d4bdCompareJune 14, 2026 22:49
anmolxlight added a commit to anmolxlight/airflow that referenced this pull request Jun 15, 2026
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
@choo121600

Copy link
Copy Markdown
Member

Could you please attach a short recording of this feature following the PR template? It would be helpful for the review.

Adds a 24h Gantt-style view that answers the question `across my whole
deployment, when during the day does each Dag usually run?` — a
complement to the per-Dag Gantt view, which only shows a single run.
Backend
* New `/ui/dag_schedule_overview` endpoint that aggregates the
most-recent successful Dag runs per Dag and returns mean/median
start/end times-of-day in UTC, plus duration statistics.
* Optional `run_after_gte` / `run_after_lte` window and
`dag_id_pattern` / `dag_display_name_pattern` filters.
* 11 backend tests covering 200/401/403, all-three-dag fixture,
morning/evening/no-runs statistics, FAILED-run exclusion, and the
filter / window edge cases.
* OpenAPI YAML updated, `pnpm codegen` regenerated.
Frontend
* New `/dags/schedule` page (one row per Dag, mean start→end bar
across a 24h tick rail, hover tooltip with mean+median start, end,
duration, and recent-runs range).
* New `Schedule Overview` sidebar nav link.
* i18n keys for English locale.
…ssion
Address failures of CI image checks / Static checks on PR apache#68547:
1. Regenerate airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
and the airflow-core/src/airflow/ui/openapi-gen/ TS client files. The prek
generate-openapi-spec hook wanted to reorder /ui/teams and /ui/dag_schedule_overview,
expand the multi-line docstring for get_dag_schedule_overview, switch minimum: 0
to minimum: 0.0 (Pydantic v2 float coercion), and drop start_*_seconds / end_*_seconds
from the DagScheduleOverviewEntry required list. This change matches what prek
would have done in CI.
2. Move the @provide_session 'session' argument to keyword-only in the
autouse setup fixture of test_dag_schedule_overview.py. New @provide_session
functions must declare session after a bare '*' per scripts/ci/prek/check_provide_session_kwargs.py.
Local verification:
- prek run --files <changed files>: passed
- prek run mypy-airflow-core --files <changed files>: passed
- prek run check-no-new-provide-session-positional --files <test file>: passed
- uv run --project airflow-core pytest test_dag_schedule_overview.py: 11/11 passed
The OpenAPI spec regen in the previous commit changed the DagScheduleOverviewEntry
schema: start_*_seconds, end_*_seconds, duration_*_seconds, and their median
counterparts were moved out of the 'required' list in Pydantic v2. The auto-generated
TS type now marks these fields as 'number | null | undefined'.
The hand-written formatTooltipBody in scheduleOverviewUtils.ts only accepted
'number | null', causing:
error TS2345 in ScheduleOverviewRow.tsx:44
Argument of type 'DagScheduleOverviewEntry' is not assignable to parameter
of type '{ readonly end_mean_seconds: number | null; ... }'
Types of property 'end_mean_seconds' are incompatible.
Type 'number | null | undefined' is not assignable to type 'number | null'.
Widen the formatTooltipBody parameter to allow 'undefined' for the now-optional
duration/start/end fields. The downstream helpers (formatSecondsAsClock,
formatDurationSeconds) already accept undefined and render it as em-dash.
Local verification:
- prek run ts-compile-lint-ui --files <changed file>: Passed
- prek run --files <changed file>: Passed
…nd params
The previous commit added '| undefined' to the duration/start/end types in
formatTooltipBody's parameter, but TypeScript's structural typing is stricter
about optional vs required properties. Since DagScheduleOverviewEntry marks
end_mean_seconds / end_median_seconds / start_mean_seconds / start_median_seconds
as '?: number | null' (truly optional, undefined comes from optionality, not
the value type), the parameter must mirror that with '?:' on the same fields.
'?: number | null' and ': number | null | undefined' are NOT structurally
assignable in either direction under TypeScript's strictness here.
Use '?: number | null' to match the auto-generated type shape. The
downstream helpers (formatSecondsAsClock, formatDurationSeconds) still
handle the undefined case and render em-dash.
Local verification:
- pnpm tsc --noEmit -p tsconfig.app.json: clean
- prek run --files <changed file>: Passed
- prek run ts-compile-lint-ui --files <changed file>: Passed
@anmolxlight

anmolxlight commented Jun 15, 2026

Copy link
Copy Markdown
ContributorAuthor
VID_20260615_232836_381.mp4

Screen recording of the new Schedule Overview page:

Shows the /dags/schedule page with the 24h UTC timeline bars, hover tooltips, and the filter working.

@anmolxlight
anmolxlightforce-pushed the feature/dag-schedule-overview-68532 branch from a60f1d5 to b3353f0CompareJune 15, 2026 17:54

@choo121600choo121600 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the screen recording you attached, this appears to be somewhat inconsistent with the existing UI patterns.

I think we need more discussion in the issue before moving forward with an implementation. In particular, we should first align on whether this feature should be included at all, and if so, how it should be integrated into the UI (tab placement, design, UX, and overall approach).

Could you please close this PR for now? I'd prefer to continue the discussion in the issue first and revisit the implementation once we've reached agreement on the design.

Once we've aligned on the direction in the issue, I'd be happy to see this PR reopened.

@potiukpotiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jun 17, 2026
@jroachgolf84

Copy link
Copy Markdown
Collaborator

I don't think that this is a feature that should be implemented. See my comment from the issue:

Not quite sure how this is useful:

typical (mean / median) start and end time of each Dag over a 24-hour timeline

If I run a DAG hourly, that would make the average noon. That's pretty useless IMO. Plus, I don't think we could make this robust enough to handle all the scheduling use-cases that Airflow offers.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP APIarea:translationsarea:UIRelated to UI/UX. For Frontend Developers.ready for maintainer reviewSet after triaging when all criteria pass.translation:default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an aggregate Dag schedule view: typical daily run times across all Dags

4 participants

@anmolxlight@choo121600@jroachgolf84@potiuk