Skip to content

Slow API server website load times #56635

Closed
Closed
Slow API server website load times#56635
Assignees
pierrejeambrun
Labels
affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases

Description

@trau-sca
trau-sca
opened on Oct 14, 2025

Apache Airflow version

3.1.0

If "Other Airflow 2/3 version" selected, which one?

No response

What happened?

Hi, we've been battling issues for the past few weeks on our api-server pod being very slow to display the website information (up to a minute to load the dags card screen). Sometimes the api server pod even restarts due to the liveness probe. I don't see any errors when it happens either.

Quite a few requests look long, but the biggest one is dags?dag_runs_limit=14&limit=50&offset=0&exclude_stale=true&order_by=-dag_display_name taking 51s, but even dags?is_favorite=true takes 4 seconds.

When remoting into the dag-processor and doing a curl command for the first one - it also takes 51s. I used the private endpoint assigned to the api-server while I was in the dag-processor.

We are running on airflow 3.1.0 and the 1.18.0 helm chart in an Azure Kubernetes cluster with an Azure PostgreSQL database.
No pods are going past their resource requests when this issue occurs, and the resources for the database are very low in usage.

I have disabled sqlalchemy pooling, and use pgbouncer in airflow as well as in the PostgreSQL database.
I have the following configuration for airflow:

pgbouncer:
enabled: true
maxClientConn: 500
metadataPoolSize: 50
resultBackendPoolSize: 10
extraIni: |
pool_mode = transaction

On PostgreSQL, I have pgbouncer's pool_mode set to session, and the default_pool_size is 100.

We have 16 DAGs, and have a daily job that runs a db clean for anything older than 90 days.

What you think should happen instead?

No response

How to reproduce

Clicking on DAGs card screen - I have 16 DAGs running with an Azure PostgreSQL backend.

Operating System

Azure Kubernetes

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

Azure PostgreSQL 16.9, Azure Kubernetes v1.32.7

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

Labels

affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    , 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
     blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Slow API server website load times · Issue #56635 · apache/airflow · GitHub
    Skip to content
    Slow API server website load times #56635
    Closed
    Closed
    Slow API server website load times#56635
    Assignees
    pierrejeambrun
    Labels
    affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
    Description
    @trau-sca
    trau-sca
    opened on Oct 14, 2025
    Apache Airflow version
    3.1.0
    If "Other Airflow 2/3 version" selected, which one?
    No response
    What happened?
    Hi, we've been battling issues for the past few weeks on our api-server pod being very slow to display the website information (up to a minute to load the dags card screen). Sometimes the api server pod even restarts due to the liveness probe. I don't see any errors when it happens either.
    Quite a few requests look long, but the biggest one is dags?dag_runs_limit=14&limit=50&offset=0&exclude_stale=true&order_by=-dag_display_name taking 51s, but even dags?is_favorite=true takes 4 seconds.
    When remoting into the dag-processor and doing a curl command for the first one - it also takes 51s. I used the private endpoint assigned to the api-server while I was in the dag-processor.
    We are running on airflow 3.1.0 and the 1.18.0 helm chart in an Azure Kubernetes cluster with an Azure PostgreSQL database.
    
No pods are going past their resource requests when this issue occurs, and the resources for the database are very low in usage.
    I have disabled sqlalchemy pooling, and use pgbouncer in airflow as well as in the PostgreSQL database.
    
I have the following configuration for airflow:
    pgbouncer:
enabled: true
maxClientConn: 500
metadataPoolSize: 50
resultBackendPoolSize: 10
extraIni: |
pool_mode = transaction

    On PostgreSQL, I have pgbouncer's pool_mode set to session, and the default_pool_size is 100.
    We have 16 DAGs, and have a daily job that runs a db clean for anything older than 90 days.
    What you think should happen instead?
    No response
    How to reproduce
    Clicking on DAGs card screen - I have 16 DAGs running with an Azure PostgreSQL backend.
    Operating System
    Azure Kubernetes
    Versions of Apache Airflow Providers
    No response
    Deployment
    Official Apache Airflow Helm Chart
    Deployment details
    Azure PostgreSQL 16.9, Azure Kubernetes v1.32.7
    Anything else?
    No response
    Are you willing to submit PR?
    •  Yes I am willing to submit a PR!
    Code of Conduct
    Metadata
    Metadata
    Assignees
    Labels
    affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
    Type
    No type
    Projects
    No projects
      Milestone
      No milestone
      Relationships
      None yet
      Development
      No branches or pull requests
      Issue actions
      
, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Slow API server website load times · Issue #56635 · apache/airflow · GitHub
      Skip to content
      Slow API server website load times #56635
      Closed
      Closed
      Slow API server website load times#56635
      Assignees
      pierrejeambrun
      Labels
      affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
      Description
      @trau-sca
      trau-sca
      opened on Oct 14, 2025
      Apache Airflow version
      3.1.0
      If "Other Airflow 2/3 version" selected, which one?
      No response
      What happened?
      Hi, we've been battling issues for the past few weeks on our api-server pod being very slow to display the website information (up to a minute to load the dags card screen). Sometimes the api server pod even restarts due to the liveness probe. I don't see any errors when it happens either.
      Quite a few requests look long, but the biggest one is dags?dag_runs_limit=14&limit=50&offset=0&exclude_stale=true&order_by=-dag_display_name taking 51s, but even dags?is_favorite=true takes 4 seconds.
      When remoting into the dag-processor and doing a curl command for the first one - it also takes 51s. I used the private endpoint assigned to the api-server while I was in the dag-processor.
      We are running on airflow 3.1.0 and the 1.18.0 helm chart in an Azure Kubernetes cluster with an Azure PostgreSQL database.
      
No pods are going past their resource requests when this issue occurs, and the resources for the database are very low in usage.
      I have disabled sqlalchemy pooling, and use pgbouncer in airflow as well as in the PostgreSQL database.
      
I have the following configuration for airflow:
      pgbouncer:
enabled: true
maxClientConn: 500
metadataPoolSize: 50
resultBackendPoolSize: 10
extraIni: |
pool_mode = transaction

      On PostgreSQL, I have pgbouncer's pool_mode set to session, and the default_pool_size is 100.
      We have 16 DAGs, and have a daily job that runs a db clean for anything older than 90 days.
      What you think should happen instead?
      No response
      How to reproduce
      Clicking on DAGs card screen - I have 16 DAGs running with an Azure PostgreSQL backend.
      Operating System
      Azure Kubernetes
      Versions of Apache Airflow Providers
      No response
      Deployment
      Official Apache Airflow Helm Chart
      Deployment details
      Azure PostgreSQL 16.9, Azure Kubernetes v1.32.7
      Anything else?
      No response
      Are you willing to submit PR?
      •  Yes I am willing to submit a PR!
      Code of Conduct
      Metadata
      Metadata
      Assignees
      Labels
      affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
      Type
      No type
      Projects
      No projects
        Milestone
        No milestone
        Relationships
        None yet
        Development
        No branches or pull requests
        Issue actions
        
, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Slow API server website load times · Issue #56635 · apache/airflow · GitHub
        Skip to content
        Slow API server website load times #56635
        Closed
        Closed
        Slow API server website load times#56635
        Assignees
        pierrejeambrun
        Labels
        affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
        Description
        @trau-sca
        trau-sca
        opened on Oct 14, 2025
        Apache Airflow version
        3.1.0
        If "Other Airflow 2/3 version" selected, which one?
        No response
        What happened?
        Hi, we've been battling issues for the past few weeks on our api-server pod being very slow to display the website information (up to a minute to load the dags card screen). Sometimes the api server pod even restarts due to the liveness probe. I don't see any errors when it happens either.
        Quite a few requests look long, but the biggest one is dags?dag_runs_limit=14&limit=50&offset=0&exclude_stale=true&order_by=-dag_display_name taking 51s, but even dags?is_favorite=true takes 4 seconds.
        When remoting into the dag-processor and doing a curl command for the first one - it also takes 51s. I used the private endpoint assigned to the api-server while I was in the dag-processor.
        We are running on airflow 3.1.0 and the 1.18.0 helm chart in an Azure Kubernetes cluster with an Azure PostgreSQL database.
        
No pods are going past their resource requests when this issue occurs, and the resources for the database are very low in usage.
        I have disabled sqlalchemy pooling, and use pgbouncer in airflow as well as in the PostgreSQL database.
        
I have the following configuration for airflow:
        pgbouncer:
enabled: true
maxClientConn: 500
metadataPoolSize: 50
resultBackendPoolSize: 10
extraIni: |
pool_mode = transaction

        On PostgreSQL, I have pgbouncer's pool_mode set to session, and the default_pool_size is 100.
        We have 16 DAGs, and have a daily job that runs a db clean for anything older than 90 days.
        What you think should happen instead?
        No response
        How to reproduce
        Clicking on DAGs card screen - I have 16 DAGs running with an Azure PostgreSQL backend.
        Operating System
        Azure Kubernetes
        Versions of Apache Airflow Providers
        No response
        Deployment
        Official Apache Airflow Helm Chart
        Deployment details
        Azure PostgreSQL 16.9, Azure Kubernetes v1.32.7
        Anything else?
        No response
        Are you willing to submit PR?
        •  Yes I am willing to submit a PR!
        Code of Conduct
        Metadata
        Metadata
        Assignees
        Labels
        affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
        Type
        No type
        Projects
        No projects
          Milestone
          No milestone
          Relationships
          None yet
          Development
          No branches or pull requests
          Issue actions
          
, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
Slow API server website load times · Issue #56635 · apache/airflow · GitHub
          Skip to content
          Slow API server website load times #56635
          Closed
          Closed
          Slow API server website load times#56635
          Assignees
          pierrejeambrun
          Labels
          affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
          Description
          @trau-sca
          trau-sca
          opened on Oct 14, 2025
          Apache Airflow version
          3.1.0
          If "Other Airflow 2/3 version" selected, which one?
          No response
          What happened?
          Hi, we've been battling issues for the past few weeks on our api-server pod being very slow to display the website information (up to a minute to load the dags card screen). Sometimes the api server pod even restarts due to the liveness probe. I don't see any errors when it happens either.
          Quite a few requests look long, but the biggest one is dags?dag_runs_limit=14&limit=50&offset=0&exclude_stale=true&order_by=-dag_display_name taking 51s, but even dags?is_favorite=true takes 4 seconds.
          When remoting into the dag-processor and doing a curl command for the first one - it also takes 51s. I used the private endpoint assigned to the api-server while I was in the dag-processor.
          We are running on airflow 3.1.0 and the 1.18.0 helm chart in an Azure Kubernetes cluster with an Azure PostgreSQL database.
          
No pods are going past their resource requests when this issue occurs, and the resources for the database are very low in usage.
          I have disabled sqlalchemy pooling, and use pgbouncer in airflow as well as in the PostgreSQL database.
          
I have the following configuration for airflow:
          pgbouncer:
enabled: true
maxClientConn: 500
metadataPoolSize: 50
resultBackendPoolSize: 10
extraIni: |
pool_mode = transaction

          On PostgreSQL, I have pgbouncer's pool_mode set to session, and the default_pool_size is 100.
          We have 16 DAGs, and have a daily job that runs a db clean for anything older than 90 days.
          What you think should happen instead?
          No response
          How to reproduce
          Clicking on DAGs card screen - I have 16 DAGs running with an Azure PostgreSQL backend.
          Operating System
          Azure Kubernetes
          Versions of Apache Airflow Providers
          No response
          Deployment
          Official Apache Airflow Helm Chart
          Deployment details
          Azure PostgreSQL 16.9, Azure Kubernetes v1.32.7
          Anything else?
          No response
          Are you willing to submit PR?
          •  Yes I am willing to submit a PR!
          Code of Conduct
          Metadata
          Metadata
          Assignees
          Labels
          affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
          Type
          No type
          Projects
          No projects
            Milestone
            No milestone
            Relationships
            None yet
            Development
            No branches or pull requests
            Issue actions
            
, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Slow API server website load times · Issue #56635 · apache/airflow · GitHub
            Skip to content
            Slow API server website load times #56635
            Closed
            Closed
            Slow API server website load times#56635
            Assignees
            pierrejeambrun
            Labels
            affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
            Description
            @trau-sca
            trau-sca
            opened on Oct 14, 2025
            Apache Airflow version
            3.1.0
            If "Other Airflow 2/3 version" selected, which one?
            No response
            What happened?
            Hi, we've been battling issues for the past few weeks on our api-server pod being very slow to display the website information (up to a minute to load the dags card screen). Sometimes the api server pod even restarts due to the liveness probe. I don't see any errors when it happens either.
            Quite a few requests look long, but the biggest one is dags?dag_runs_limit=14&limit=50&offset=0&exclude_stale=true&order_by=-dag_display_name taking 51s, but even dags?is_favorite=true takes 4 seconds.
            When remoting into the dag-processor and doing a curl command for the first one - it also takes 51s. I used the private endpoint assigned to the api-server while I was in the dag-processor.
            We are running on airflow 3.1.0 and the 1.18.0 helm chart in an Azure Kubernetes cluster with an Azure PostgreSQL database.
            
No pods are going past their resource requests when this issue occurs, and the resources for the database are very low in usage.
            I have disabled sqlalchemy pooling, and use pgbouncer in airflow as well as in the PostgreSQL database.
            
I have the following configuration for airflow:
            pgbouncer:
enabled: true
maxClientConn: 500
metadataPoolSize: 50
resultBackendPoolSize: 10
extraIni: |
pool_mode = transaction

            On PostgreSQL, I have pgbouncer's pool_mode set to session, and the default_pool_size is 100.
            We have 16 DAGs, and have a daily job that runs a db clean for anything older than 90 days.
            What you think should happen instead?
            No response
            How to reproduce
            Clicking on DAGs card screen - I have 16 DAGs running with an Azure PostgreSQL backend.
            Operating System
            Azure Kubernetes
            Versions of Apache Airflow Providers
            No response
            Deployment
            Official Apache Airflow Helm Chart
            Deployment details
            Azure PostgreSQL 16.9, Azure Kubernetes v1.32.7
            Anything else?
            No response
            Are you willing to submit PR?
            •  Yes I am willing to submit a PR!
            Code of Conduct
            Metadata
            Metadata
            Assignees
            Labels
            affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
            Type
            No type
            Projects
            No projects
              Milestone
              No milestone
              Relationships
              None yet
              Development
              No branches or pull requests
              Issue actions
              
, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Slow API server website load times · Issue #56635 · apache/airflow · GitHub
              Skip to content
              Slow API server website load times #56635
              Closed
              Closed
              Slow API server website load times#56635
              Assignees
              pierrejeambrun
              Labels
              affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
              Description
              @trau-sca
              trau-sca
              opened on Oct 14, 2025
              Apache Airflow version
              3.1.0
              If "Other Airflow 2/3 version" selected, which one?
              No response
              What happened?
              Hi, we've been battling issues for the past few weeks on our api-server pod being very slow to display the website information (up to a minute to load the dags card screen). Sometimes the api server pod even restarts due to the liveness probe. I don't see any errors when it happens either.
              Quite a few requests look long, but the biggest one is dags?dag_runs_limit=14&limit=50&offset=0&exclude_stale=true&order_by=-dag_display_name taking 51s, but even dags?is_favorite=true takes 4 seconds.
              When remoting into the dag-processor and doing a curl command for the first one - it also takes 51s. I used the private endpoint assigned to the api-server while I was in the dag-processor.
              We are running on airflow 3.1.0 and the 1.18.0 helm chart in an Azure Kubernetes cluster with an Azure PostgreSQL database.
              
No pods are going past their resource requests when this issue occurs, and the resources for the database are very low in usage.
              I have disabled sqlalchemy pooling, and use pgbouncer in airflow as well as in the PostgreSQL database.
              
I have the following configuration for airflow:
              pgbouncer:
enabled: true
maxClientConn: 500
metadataPoolSize: 50
resultBackendPoolSize: 10
extraIni: |
pool_mode = transaction

              On PostgreSQL, I have pgbouncer's pool_mode set to session, and the default_pool_size is 100.
              We have 16 DAGs, and have a daily job that runs a db clean for anything older than 90 days.
              What you think should happen instead?
              No response
              How to reproduce
              Clicking on DAGs card screen - I have 16 DAGs running with an Azure PostgreSQL backend.
              Operating System
              Azure Kubernetes
              Versions of Apache Airflow Providers
              No response
              Deployment
              Official Apache Airflow Helm Chart
              Deployment details
              Azure PostgreSQL 16.9, Azure Kubernetes v1.32.7
              Anything else?
              No response
              Are you willing to submit PR?
              •  Yes I am willing to submit a PR!
              Code of Conduct
              Metadata
              Metadata
              Assignees
              Labels
              affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
              Type
              No type
              Projects
              No projects
                Milestone
                No milestone
                Relationships
                None yet
                Development
                No branches or pull requests
                Issue actions
                
, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
Slow API server website load times · Issue #56635 · apache/airflow · GitHub
                Skip to content
                Slow API server website load times #56635
                Closed
                Closed
                Slow API server website load times#56635
                Assignees
                pierrejeambrun
                Labels
                affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
                Description
                @trau-sca
                trau-sca
                opened on Oct 14, 2025
                Apache Airflow version
                3.1.0
                If "Other Airflow 2/3 version" selected, which one?
                No response
                What happened?
                Hi, we've been battling issues for the past few weeks on our api-server pod being very slow to display the website information (up to a minute to load the dags card screen). Sometimes the api server pod even restarts due to the liveness probe. I don't see any errors when it happens either.
                Quite a few requests look long, but the biggest one is dags?dag_runs_limit=14&limit=50&offset=0&exclude_stale=true&order_by=-dag_display_name taking 51s, but even dags?is_favorite=true takes 4 seconds.
                When remoting into the dag-processor and doing a curl command for the first one - it also takes 51s. I used the private endpoint assigned to the api-server while I was in the dag-processor.
                We are running on airflow 3.1.0 and the 1.18.0 helm chart in an Azure Kubernetes cluster with an Azure PostgreSQL database.
                
No pods are going past their resource requests when this issue occurs, and the resources for the database are very low in usage.
                I have disabled sqlalchemy pooling, and use pgbouncer in airflow as well as in the PostgreSQL database.
                
I have the following configuration for airflow:
                pgbouncer:
enabled: true
maxClientConn: 500
metadataPoolSize: 50
resultBackendPoolSize: 10
extraIni: |
pool_mode = transaction

                On PostgreSQL, I have pgbouncer's pool_mode set to session, and the default_pool_size is 100.
                We have 16 DAGs, and have a daily job that runs a db clean for anything older than 90 days.
                What you think should happen instead?
                No response
                How to reproduce
                Clicking on DAGs card screen - I have 16 DAGs running with an Azure PostgreSQL backend.
                Operating System
                Azure Kubernetes
                Versions of Apache Airflow Providers
                No response
                Deployment
                Official Apache Airflow Helm Chart
                Deployment details
                Azure PostgreSQL 16.9, Azure Kubernetes v1.32.7
                Anything else?
                No response
                Are you willing to submit PR?
                •  Yes I am willing to submit a PR!
                Code of Conduct
                Metadata
                Metadata
                Assignees
                Labels
                affected_version:3.1Issues Reported for 3.1area:APIAirflow's REST/HTTP APIarea:corearea:performancekind:bugThis is a clearly a bugtesting statusStatus of testing releases
                Type
                No type
                Projects
                No projects
                  Milestone
                  No milestone
                  Relationships
                  None yet
                  Development
                  No branches or pull requests
                  Issue actions