Uh oh!
There was an error while loading. Please reload this page.
UI: Add cross-Dag Time Schedule view - #71558
Conversation
Features: - Add Day and Week timelines for Dag run timing patterns - Group runs by Dag, state, time bucket, and weekday - Support Mean, Max, and Min duration aggregation - Show expected runs for scheduled Dags without matching runs - Add Dag run, tag, timetable type, and team filters Interactions: - Add pointer-anchored timeline zoom controls - Link timeline bars to their Dag or Dag run - Add readable tooltips for run details - Persist view, aggregation, limit, and filter preferences - Support overlapping runs in Day and Week layouts Data loading: - Add selectable limits from 200 runs through all runs - Paginate using the API-configured page size - Prevent gaps between paginated Dag run results - Limit Dag detail requests to expected-run markers Documentation and tests: - Document Time Schedule behavior in the UI guide - Add light and dark screenshots - Cover layouts, aggregation, filters, zoom, and pagination
bbovenzi
commented
Aug 21, 2026
Very cool! I could see a world where we could even show arrows between runs that are connected via partition keys. Or highlighting when Max active dag runs is being hit. Love the minute detailing. That's great for dags that run every minute! Just needs a label.
|
bbovenzi
left a comment
There was a problem hiding this comment.
This is a cool demo. But it definitely needs some work to "productionize" the code to be usable for deployments with any degree of scale.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Capture changes during partial author review: Data access and correctness: - Avoid browser request fan-out and client-side filtering as Dag and run counts grow. - Bound each view to 5,000 matching runs rather than allowing unbounded reads. - Calculate only the selected Day or Week aggregation with access controls and filters applied. Interaction quality: - Stream timeline batches so useful data appears before the complete result is ready. - Preserve displayed bars during zoom-driven refreshes while the final zoom level settles. - Keep planned Dag markers visible even when no Dag runs match the current view. Contract and verification: - Keep the private UI OpenAPI contract and generated client aligned with the endpoint. - Cover streaming, limits, filtering, aggregation, planned runs, and zoom behavior.
- Avoid MySQL's unsupported LIMIT subquery in an IN predicate. - Mark NDJSON reads as sequential and fix TimeSchedule type errors. - Confirm earlier UI import failures no longer affect asset builds.
Keep Time Schedule compatible with the shared FilterBar refactor merged from main. - Deleted Dag list filter imports caused Vite module-resolution failures. - The failure blocked React UI tests and CI jobs that build UI assets. - Reuse the shared filter state for Tag and Timetable Type filtering.
minyeamer
commented
Aug 29, 2026
@bbovenzi Since b13f993, excluding merge commits, I made the following changes:
The current flow is: flowchart TD
UI[Time Schedule UI] --> Hook[useTimeScheduleData]
Hook -->|One GET request| API[GET /ui/time-schedule]
API --> Filter[Apply authorization and filters in SQL]
Filter --> Limit[Select up to 5,000 Dag runs]
Limit --> Batch[Process 25 Dags per batch]
Batch --> Aggregate[Aggregate selected Day or Week view]
Aggregate --> Stream[Return NDJSON TimeScheduleBatch]
Stream --> Hook
Hook --> Render[Progressively render timeline bars]
The browser now receives only the filtered data needed for the selected view, while the backend controls the result size and aggregation cost. I initially intended this change to remain UI-only and did not plan to add a backend API. Because the review identified the scalability limitations of the original approach, I needed to first understand Airflow's backend API structure, streaming session lifecycle, authorization filters, and OpenAPI generation flow. I also spent additional time reviewing the AI-assisted code carefully, which is why the follow-up changes took about 5 days. The implementation is now focused on the production-scale concerns from the review while preserving the existing Airflow UI/API patterns. |
minyeamer
commented
Aug 30, 2026
I missed these additional suggestions in my previous follow-up. I plan to address the following:
For the partition-key arrows, I understand this may include dependencies between Dag runs, such as those created through I’m happy to implement this with more specific guidance on which relationships to show and how they should be laid out. With the current context, I’m hesitant to add a potentially confusing visualization. Drafted-by: GPT-5 |
Add a Time Schedule tab for comparing Dag start times and durations across an Airflow environment.
Features:
This is a UI-only change. It reuses existing endpoints and adds no schema or backend API changes.
Earlier work in #68532 and #68547 informed the time-bucket design. The view placement follows #45497, while bounded loading follows the work in #47900, #60241, and #65388.
Related: #22001
Screenshots:
Videos:
time_schedule-day-zoom_in_out.mp4
time_schedule-week-zoom_in_out.mp4
time_schedule-day-bar_link.mp4
time_schedule-week-display_n_dags.mp4
time_schedule-day-aggregation.mp4
Was generative AI tooling used to co-author this PR?
Generated-by: Codex (GPT-5.6 Sol) following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.