Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns #57776

Closed
Closed
Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns#57776
Assignees
bbovenzi
Labels
affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug

Description

@g-dailey
g-dailey
opened on Nov 3, 2025

Apache Airflow version

3.1.0

If "Other Airflow 2/3 version" selected, which one?

No response

What happened?

While testing Airflow 3.1.1, I observed moderate performance degradation in the Grid View when working with a large DAG that has around 180–200 tasks and 30+ dagruns.

After refreshing the Grid View several times or triggering new DAG runs, the page loads very slowly and eventually becomes unresponsive. The issue becomes apparent once the number of dagruns exceeds roughly 25.

The issue appears to stem from multiple areas:
• Backend inefficiency: TaskInstance summaries are fetched per dagrun, leading to heavy and repetitive processing.
• Frontend inefficiency: The Grid View performs full data reloads for all visible dagruns on every refresh.
• These behaviors are consistent with scaling limitations previously reported in apache/airflow#50928.

Addressing this likely requires coordinated changes to both backend batching and frontend rendering logic.

Possible Improvements
• Implement batched TaskInstance summary fetching to avoid redundant per-dagrun queries.
• Introduce pagination or virtualization for dagruns in the Grid View.
• Optimize frontend rendering to reduce full data reprocessing.

What you think should happen instead?

Expected Behavior
• Grid View should remain responsive even for DAGs with hundreds of tasks and many historical dagruns.
• Memory usage should remain stable regardless of dagrun count or refresh frequency.
• Triggering additional DAG runs should not cause UI or API lag.

How to reproduce

Steps to Reproduce
1. Create or use a DAG with approximately 180–200 tasks.
2. Allow the DAG to accumulate 30+ dagruns.
3. Open the Grid View and refresh the page multiple times, or trigger additional DAG runs a few times.
4. Observe increasing page load times and memory consumption.

Operating System

Operating System: Debian GNU/Linux 12 (bookworm) (via Astro Runtime 3.1-3 container)

Versions of Apache Airflow Providers

No response

Deployment

Astronomer

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

Labels

affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
     blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

    Skip to content
    Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns #57776
    Closed
    Closed
    Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns#57776
    Assignees
    bbovenzi
    Labels
    affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
    Description
    @g-dailey
    g-dailey
    opened on Nov 3, 2025
    Apache Airflow version
    3.1.0
    If "Other Airflow 2/3 version" selected, which one?
    No response
    What happened?
    While testing Airflow 3.1.1, I observed moderate performance degradation in the Grid View when working with a large DAG that has around 180–200 tasks and 30+ dagruns.
    After refreshing the Grid View several times or triggering new DAG runs, the page loads very slowly and eventually becomes unresponsive. The issue becomes apparent once the number of dagruns exceeds roughly 25.
    The issue appears to stem from multiple areas:
    
•	Backend inefficiency: TaskInstance summaries are fetched per dagrun, leading to heavy and repetitive processing.
    
•	Frontend inefficiency: The Grid View performs full data reloads for all visible dagruns on every refresh.
    
•	These behaviors are consistent with scaling limitations previously reported in apache/airflow#50928.
    Addressing this likely requires coordinated changes to both backend batching and frontend rendering logic.
    Possible Improvements
    
•	Implement batched TaskInstance summary fetching to avoid redundant per-dagrun queries.
    
•	Introduce pagination or virtualization for dagruns in the Grid View.
    
•	Optimize frontend rendering to reduce full data reprocessing.
    What you think should happen instead?
    Expected Behavior
    
•	Grid View should remain responsive even for DAGs with hundreds of tasks and many historical dagruns.
    
•	Memory usage should remain stable regardless of dagrun count or refresh frequency.
    
•	Triggering additional DAG runs should not cause UI or API lag.
    How to reproduce
    Steps to Reproduce
    
1.	Create or use a DAG with approximately 180–200 tasks.
    
2.	Allow the DAG to accumulate 30+ dagruns.
    
3.	Open the Grid View and refresh the page multiple times, or trigger additional DAG runs a few times.
    
4.	Observe increasing page load times and memory consumption.
    Operating System
    Operating System: Debian GNU/Linux 12 (bookworm) (via Astro Runtime 3.1-3 container)
    Versions of Apache Airflow Providers
    No response
    Deployment
    Astronomer
    Deployment details
    No response
    Anything else?
    No response
    Are you willing to submit PR?
    •  Yes I am willing to submit a PR!
    Code of Conduct
    Metadata
    Metadata
    Assignees
    Labels
    affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
    Type
    No type
    Projects
    No projects
      Milestone
      No milestone
      Relationships
      None yet
      Development
      No branches or pull requests
      Issue actions
      
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

      Skip to content
      Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns #57776
      Closed
      Closed
      Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns#57776
      Assignees
      bbovenzi
      Labels
      affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
      Description
      @g-dailey
      g-dailey
      opened on Nov 3, 2025
      Apache Airflow version
      3.1.0
      If "Other Airflow 2/3 version" selected, which one?
      No response
      What happened?
      While testing Airflow 3.1.1, I observed moderate performance degradation in the Grid View when working with a large DAG that has around 180–200 tasks and 30+ dagruns.
      After refreshing the Grid View several times or triggering new DAG runs, the page loads very slowly and eventually becomes unresponsive. The issue becomes apparent once the number of dagruns exceeds roughly 25.
      The issue appears to stem from multiple areas:
      
•	Backend inefficiency: TaskInstance summaries are fetched per dagrun, leading to heavy and repetitive processing.
      
•	Frontend inefficiency: The Grid View performs full data reloads for all visible dagruns on every refresh.
      
•	These behaviors are consistent with scaling limitations previously reported in apache/airflow#50928.
      Addressing this likely requires coordinated changes to both backend batching and frontend rendering logic.
      Possible Improvements
      
•	Implement batched TaskInstance summary fetching to avoid redundant per-dagrun queries.
      
•	Introduce pagination or virtualization for dagruns in the Grid View.
      
•	Optimize frontend rendering to reduce full data reprocessing.
      What you think should happen instead?
      Expected Behavior
      
•	Grid View should remain responsive even for DAGs with hundreds of tasks and many historical dagruns.
      
•	Memory usage should remain stable regardless of dagrun count or refresh frequency.
      
•	Triggering additional DAG runs should not cause UI or API lag.
      How to reproduce
      Steps to Reproduce
      
1.	Create or use a DAG with approximately 180–200 tasks.
      
2.	Allow the DAG to accumulate 30+ dagruns.
      
3.	Open the Grid View and refresh the page multiple times, or trigger additional DAG runs a few times.
      
4.	Observe increasing page load times and memory consumption.
      Operating System
      Operating System: Debian GNU/Linux 12 (bookworm) (via Astro Runtime 3.1-3 container)
      Versions of Apache Airflow Providers
      No response
      Deployment
      Astronomer
      Deployment details
      No response
      Anything else?
      No response
      Are you willing to submit PR?
      •  Yes I am willing to submit a PR!
      Code of Conduct
      Metadata
      Metadata
      Assignees
      Labels
      affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
      Type
      No type
      Projects
      No projects
        Milestone
        No milestone
        Relationships
        None yet
        Development
        No branches or pull requests
        Issue actions
        
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

        Skip to content
        Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns #57776
        Closed
        Closed
        Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns#57776
        Assignees
        bbovenzi
        Labels
        affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
        Description
        @g-dailey
        g-dailey
        opened on Nov 3, 2025
        Apache Airflow version
        3.1.0
        If "Other Airflow 2/3 version" selected, which one?
        No response
        What happened?
        While testing Airflow 3.1.1, I observed moderate performance degradation in the Grid View when working with a large DAG that has around 180–200 tasks and 30+ dagruns.
        After refreshing the Grid View several times or triggering new DAG runs, the page loads very slowly and eventually becomes unresponsive. The issue becomes apparent once the number of dagruns exceeds roughly 25.
        The issue appears to stem from multiple areas:
        
•	Backend inefficiency: TaskInstance summaries are fetched per dagrun, leading to heavy and repetitive processing.
        
•	Frontend inefficiency: The Grid View performs full data reloads for all visible dagruns on every refresh.
        
•	These behaviors are consistent with scaling limitations previously reported in apache/airflow#50928.
        Addressing this likely requires coordinated changes to both backend batching and frontend rendering logic.
        Possible Improvements
        
•	Implement batched TaskInstance summary fetching to avoid redundant per-dagrun queries.
        
•	Introduce pagination or virtualization for dagruns in the Grid View.
        
•	Optimize frontend rendering to reduce full data reprocessing.
        What you think should happen instead?
        Expected Behavior
        
•	Grid View should remain responsive even for DAGs with hundreds of tasks and many historical dagruns.
        
•	Memory usage should remain stable regardless of dagrun count or refresh frequency.
        
•	Triggering additional DAG runs should not cause UI or API lag.
        How to reproduce
        Steps to Reproduce
        
1.	Create or use a DAG with approximately 180–200 tasks.
        
2.	Allow the DAG to accumulate 30+ dagruns.
        
3.	Open the Grid View and refresh the page multiple times, or trigger additional DAG runs a few times.
        
4.	Observe increasing page load times and memory consumption.
        Operating System
        Operating System: Debian GNU/Linux 12 (bookworm) (via Astro Runtime 3.1-3 container)
        Versions of Apache Airflow Providers
        No response
        Deployment
        Astronomer
        Deployment details
        No response
        Anything else?
        No response
        Are you willing to submit PR?
        •  Yes I am willing to submit a PR!
        Code of Conduct
        Metadata
        Metadata
        Assignees
        Labels
        affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
        Type
        No type
        Projects
        No projects
          Milestone
          No milestone
          Relationships
          None yet
          Development
          No branches or pull requests
          Issue actions
          
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

          Skip to content
          Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns #57776
          Closed
          Closed
          Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns#57776
          Assignees
          bbovenzi
          Labels
          affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
          Description
          @g-dailey
          g-dailey
          opened on Nov 3, 2025
          Apache Airflow version
          3.1.0
          If "Other Airflow 2/3 version" selected, which one?
          No response
          What happened?
          While testing Airflow 3.1.1, I observed moderate performance degradation in the Grid View when working with a large DAG that has around 180–200 tasks and 30+ dagruns.
          After refreshing the Grid View several times or triggering new DAG runs, the page loads very slowly and eventually becomes unresponsive. The issue becomes apparent once the number of dagruns exceeds roughly 25.
          The issue appears to stem from multiple areas:
          
•	Backend inefficiency: TaskInstance summaries are fetched per dagrun, leading to heavy and repetitive processing.
          
•	Frontend inefficiency: The Grid View performs full data reloads for all visible dagruns on every refresh.
          
•	These behaviors are consistent with scaling limitations previously reported in apache/airflow#50928.
          Addressing this likely requires coordinated changes to both backend batching and frontend rendering logic.
          Possible Improvements
          
•	Implement batched TaskInstance summary fetching to avoid redundant per-dagrun queries.
          
•	Introduce pagination or virtualization for dagruns in the Grid View.
          
•	Optimize frontend rendering to reduce full data reprocessing.
          What you think should happen instead?
          Expected Behavior
          
•	Grid View should remain responsive even for DAGs with hundreds of tasks and many historical dagruns.
          
•	Memory usage should remain stable regardless of dagrun count or refresh frequency.
          
•	Triggering additional DAG runs should not cause UI or API lag.
          How to reproduce
          Steps to Reproduce
          
1.	Create or use a DAG with approximately 180–200 tasks.
          
2.	Allow the DAG to accumulate 30+ dagruns.
          
3.	Open the Grid View and refresh the page multiple times, or trigger additional DAG runs a few times.
          
4.	Observe increasing page load times and memory consumption.
          Operating System
          Operating System: Debian GNU/Linux 12 (bookworm) (via Astro Runtime 3.1-3 container)
          Versions of Apache Airflow Providers
          No response
          Deployment
          Astronomer
          Deployment details
          No response
          Anything else?
          No response
          Are you willing to submit PR?
          •  Yes I am willing to submit a PR!
          Code of Conduct
          Metadata
          Metadata
          Assignees
          Labels
          affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
          Type
          No type
          Projects
          No projects
            Milestone
            No milestone
            Relationships
            None yet
            Development
            No branches or pull requests
            Issue actions
            
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

            Skip to content
            Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns #57776
            Closed
            Closed
            Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns#57776
            Assignees
            bbovenzi
            Labels
            affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
            Description
            @g-dailey
            g-dailey
            opened on Nov 3, 2025
            Apache Airflow version
            3.1.0
            If "Other Airflow 2/3 version" selected, which one?
            No response
            What happened?
            While testing Airflow 3.1.1, I observed moderate performance degradation in the Grid View when working with a large DAG that has around 180–200 tasks and 30+ dagruns.
            After refreshing the Grid View several times or triggering new DAG runs, the page loads very slowly and eventually becomes unresponsive. The issue becomes apparent once the number of dagruns exceeds roughly 25.
            The issue appears to stem from multiple areas:
            
•	Backend inefficiency: TaskInstance summaries are fetched per dagrun, leading to heavy and repetitive processing.
            
•	Frontend inefficiency: The Grid View performs full data reloads for all visible dagruns on every refresh.
            
•	These behaviors are consistent with scaling limitations previously reported in apache/airflow#50928.
            Addressing this likely requires coordinated changes to both backend batching and frontend rendering logic.
            Possible Improvements
            
•	Implement batched TaskInstance summary fetching to avoid redundant per-dagrun queries.
            
•	Introduce pagination or virtualization for dagruns in the Grid View.
            
•	Optimize frontend rendering to reduce full data reprocessing.
            What you think should happen instead?
            Expected Behavior
            
•	Grid View should remain responsive even for DAGs with hundreds of tasks and many historical dagruns.
            
•	Memory usage should remain stable regardless of dagrun count or refresh frequency.
            
•	Triggering additional DAG runs should not cause UI or API lag.
            How to reproduce
            Steps to Reproduce
            
1.	Create or use a DAG with approximately 180–200 tasks.
            
2.	Allow the DAG to accumulate 30+ dagruns.
            
3.	Open the Grid View and refresh the page multiple times, or trigger additional DAG runs a few times.
            
4.	Observe increasing page load times and memory consumption.
            Operating System
            Operating System: Debian GNU/Linux 12 (bookworm) (via Astro Runtime 3.1-3 container)
            Versions of Apache Airflow Providers
            No response
            Deployment
            Astronomer
            Deployment details
            No response
            Anything else?
            No response
            Are you willing to submit PR?
            •  Yes I am willing to submit a PR!
            Code of Conduct
            Metadata
            Metadata
            Assignees
            Labels
            affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
            Type
            No type
            Projects
            No projects
              Milestone
              No milestone
              Relationships
              None yet
              Development
              No branches or pull requests
              Issue actions
              
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

              Skip to content
              Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns #57776
              Closed
              Closed
              Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns#57776
              Assignees
              bbovenzi
              Labels
              affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
              Description
              @g-dailey
              g-dailey
              opened on Nov 3, 2025
              Apache Airflow version
              3.1.0
              If "Other Airflow 2/3 version" selected, which one?
              No response
              What happened?
              While testing Airflow 3.1.1, I observed moderate performance degradation in the Grid View when working with a large DAG that has around 180–200 tasks and 30+ dagruns.
              After refreshing the Grid View several times or triggering new DAG runs, the page loads very slowly and eventually becomes unresponsive. The issue becomes apparent once the number of dagruns exceeds roughly 25.
              The issue appears to stem from multiple areas:
              
•	Backend inefficiency: TaskInstance summaries are fetched per dagrun, leading to heavy and repetitive processing.
              
•	Frontend inefficiency: The Grid View performs full data reloads for all visible dagruns on every refresh.
              
•	These behaviors are consistent with scaling limitations previously reported in apache/airflow#50928.
              Addressing this likely requires coordinated changes to both backend batching and frontend rendering logic.
              Possible Improvements
              
•	Implement batched TaskInstance summary fetching to avoid redundant per-dagrun queries.
              
•	Introduce pagination or virtualization for dagruns in the Grid View.
              
•	Optimize frontend rendering to reduce full data reprocessing.
              What you think should happen instead?
              Expected Behavior
              
•	Grid View should remain responsive even for DAGs with hundreds of tasks and many historical dagruns.
              
•	Memory usage should remain stable regardless of dagrun count or refresh frequency.
              
•	Triggering additional DAG runs should not cause UI or API lag.
              How to reproduce
              Steps to Reproduce
              
1.	Create or use a DAG with approximately 180–200 tasks.
              
2.	Allow the DAG to accumulate 30+ dagruns.
              
3.	Open the Grid View and refresh the page multiple times, or trigger additional DAG runs a few times.
              
4.	Observe increasing page load times and memory consumption.
              Operating System
              Operating System: Debian GNU/Linux 12 (bookworm) (via Astro Runtime 3.1-3 container)
              Versions of Apache Airflow Providers
              No response
              Deployment
              Astronomer
              Deployment details
              No response
              Anything else?
              No response
              Are you willing to submit PR?
              •  Yes I am willing to submit a PR!
              Code of Conduct
              Metadata
              Metadata
              Assignees
              Labels
              affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
              Type
              No type
              Projects
              No projects
                Milestone
                No milestone
                Relationships
                None yet
                Development
                No branches or pull requests
                Issue actions
                
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

                Skip to content
                Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns #57776
                Closed
                Closed
                Grid View Performance Degradation for Large DAGs (180+ tasks) and Large Number of Dagruns#57776
                Assignees
                bbovenzi
                Labels
                affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
                Description
                @g-dailey
                g-dailey
                opened on Nov 3, 2025
                Apache Airflow version
                3.1.0
                If "Other Airflow 2/3 version" selected, which one?
                No response
                What happened?
                While testing Airflow 3.1.1, I observed moderate performance degradation in the Grid View when working with a large DAG that has around 180–200 tasks and 30+ dagruns.
                After refreshing the Grid View several times or triggering new DAG runs, the page loads very slowly and eventually becomes unresponsive. The issue becomes apparent once the number of dagruns exceeds roughly 25.
                The issue appears to stem from multiple areas:
                
•	Backend inefficiency: TaskInstance summaries are fetched per dagrun, leading to heavy and repetitive processing.
                
•	Frontend inefficiency: The Grid View performs full data reloads for all visible dagruns on every refresh.
                
•	These behaviors are consistent with scaling limitations previously reported in apache/airflow#50928.
                Addressing this likely requires coordinated changes to both backend batching and frontend rendering logic.
                Possible Improvements
                
•	Implement batched TaskInstance summary fetching to avoid redundant per-dagrun queries.
                
•	Introduce pagination or virtualization for dagruns in the Grid View.
                
•	Optimize frontend rendering to reduce full data reprocessing.
                What you think should happen instead?
                Expected Behavior
                
•	Grid View should remain responsive even for DAGs with hundreds of tasks and many historical dagruns.
                
•	Memory usage should remain stable regardless of dagrun count or refresh frequency.
                
•	Triggering additional DAG runs should not cause UI or API lag.
                How to reproduce
                Steps to Reproduce
                
1.	Create or use a DAG with approximately 180–200 tasks.
                
2.	Allow the DAG to accumulate 30+ dagruns.
                
3.	Open the Grid View and refresh the page multiple times, or trigger additional DAG runs a few times.
                
4.	Observe increasing page load times and memory consumption.
                Operating System
                Operating System: Debian GNU/Linux 12 (bookworm) (via Astro Runtime 3.1-3 container)
                Versions of Apache Airflow Providers
                No response
                Deployment
                Astronomer
                Deployment details
                No response
                Anything else?
                No response
                Are you willing to submit PR?
                •  Yes I am willing to submit a PR!
                Code of Conduct
                Metadata
                Metadata
                Assignees
                Labels
                affected_version:3.1Issues Reported for 3.1area:UIRelated to UI/UX. For Frontend Developers.area:corekind:bugThis is a clearly a bug
                Type
                No type
                Projects
                No projects
                  Milestone
                  No milestone
                  Relationships
                  None yet
                  Development
                  No branches or pull requests
                  Issue actions