Skip to content

Add FAQ entry for API server memory growth with gunicorn worker recycling - #65036

Merged
kaxil merged 2 commits into
apache:mainfrom
astronomer:faq-api-server-memory-gunicorn
Apr 10, 2026
Merged

Add FAQ entry for API server memory growth with gunicorn worker recycling#65036
kaxil merged 2 commits into
apache:mainfrom
astronomer:faq-api-server-memory-gunicorn

Conversation

@kaxil

@kaxilkaxil commented Apr 10, 2026

Copy link
Copy Markdown
Member

Adds an FAQ entry documenting gunicorn with rolling worker restarts as the recommended solution for API server memory growth.

The API server's DBDagBag caches serialized DAG objects in an unbounded dict that grows as DAG versions accumulate. Users hitting this in production (see #60804, #64326) may not know that gunicorn worker recycling (added in #60940) already handles this by periodically restarting workers and releasing all accumulated memory.

The new FAQ entry:

  • Explains the problem (serialized DAG cache growth) and links to the existing dag version inflation FAQ
  • Documents the gunicorn + worker_refresh_interval configuration
  • Notes that worker recycling handles memory growth from any source, not just the DAG cache

@kaxil
Add FAQ entry for API server memory growth with gunicorn fix
24fddfe
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
@kaxilkaxil added this to the Airflow 3.2.1 milestone Apr 10, 2026
dheerajturaga
dheerajturaga reviewed Apr 10, 2026
View reviewed changes

@dheerajturagadheerajturaga left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a comment mentioning that this is available starting 3.2/3.2.1

Other than that LGTM!

@kaxil
kaxil merged commit 74283fb into apache:mainApr 10, 2026
67 checks passed
@kaxil
kaxil deleted the faq-api-server-memory-gunicorn branch April 10, 2026 23:56
github-actionsBot pushed a commit that referenced this pull request Apr 10, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6667871
…worker recycling (#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
@github-actions

github-actionsBot commented Apr 10, 2026

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
✅v3-2-testPR Link

github-actionsBot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Apr 11, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6152a53
…worker recycling (apache#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
kaxil added a commit that referenced this pull request Apr 11, 2026
@github-actions@kaxil
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ff70972
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit that referenced this pull request Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
5110dc9
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit that referenced this pull request Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
8c36139
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit that referenced this pull request Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
34916cf
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit that referenced this pull request Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ad89865
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dheerajturagadheerajturagadheerajturaga approved these changes
@pierrejeambrunpierrejeambrunAwaiting requested review from pierrejeambrun

Assignees

No one assigned

Labels

kind:documentation

Projects

None yet

Milestone

Airflow 3.2.1

Development

Successfully merging this pull request may close these issues.

2 participants

@kaxil@dheerajturaga
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Add FAQ entry for API server memory growth with gunicorn worker recycling by kaxil · Pull Request #65036 · apache/airflow · GitHub
Skip to content
Add FAQ entry for API server memory growth with gunicorn worker recycling - #65036
Merged
kaxil merged 2 commits into
apache:mainfrom 
astronomer:faq-api-server-memory-gunicorn
Apr 10, 2026
Merged
Add FAQ entry for API server memory growth with gunicorn worker recycling#65036
kaxil merged 2 commits into
apache:mainfrom 
astronomer:faq-api-server-memory-gunicorn
Conversation
@kaxil
@kaxilkaxil
commented
Apr 10, 2026

Copy link


Copy Markdown

Member
Adds an FAQ entry documenting gunicorn with rolling worker restarts as the recommended solution for API server memory growth.
The API server's DBDagBag caches serialized DAG objects in an unbounded dict that grows as DAG versions accumulate. Users hitting this in production (see #60804, #64326) may not know that gunicorn worker recycling (added in #60940) already handles this by periodically restarting workers and releasing all accumulated memory.
The new FAQ entry:
  • Explains the problem (serialized DAG cache growth) and links to the existing dag version inflation FAQ
  • Documents the gunicorn + worker_refresh_interval configuration
  • Notes that worker recycling handles memory growth from any source, not just the DAG cache
@kaxil
Add FAQ entry for API server memory growth with gunicorn fix
24fddfe
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
@kaxilkaxil
added this to the Airflow 3.2.1 milestone
Apr 10, 2026
dheerajturaga
dheerajturaga
reviewed
Apr 10, 2026
View reviewed changes
@dheerajturagadheerajturaga
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a comment mentioning that this is available starting 3.2/3.2.1
Other than that LGTM!
@kaxil
kaxil
merged commit 74283fb
into
apache:mainApr 10, 2026

67 checks passed

@kaxil
kaxil
deleted the

faq-api-server-memory-gunicorn

branch
April 10, 2026 23:56
github-actionsBot
pushed a commit
that referenced
this pull request
Apr 10, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6667871
…worker recycling (#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
@github-actions
github-actionsBot
commented
Apr 10, 2026

Copy link


Copy Markdown

Contributor
Backport successfully created: v3-2-test
Note: As of Merging PRs targeted for Airflow 3.X

the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
In matter of doubt please ask in #release-management Slack channel.
StatusBranchResult
✅v3-2-testPR Link
github-actionsBot
pushed a commit
to aws-mwaa/upstream-to-airflow
that referenced
this pull request
Apr 11, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6152a53
…worker recycling (apache#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
kaxil added a commit
that referenced
this pull request
Apr 11, 2026
@github-actions@kaxil
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ff70972
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
5110dc9
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
8c36139
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
34916cf
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ad89865
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@dheerajturagadheerajturagadheerajturaga approved these changes
@pierrejeambrunpierrejeambrunAwaiting requested review from pierrejeambrun

Assignees


No one assigned


Labels

kind:documentation

Projects


None yet


Milestone

Airflow 3.2.1

Development

Successfully merging this pull request may close these issues.

2 participants

@kaxil@dheerajturaga

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Add FAQ entry for API server memory growth with gunicorn worker recycling by kaxil · Pull Request #65036 · apache/airflow · GitHub
Skip to content
Add FAQ entry for API server memory growth with gunicorn worker recycling - #65036
Merged
kaxil merged 2 commits into
apache:mainfrom 
astronomer:faq-api-server-memory-gunicorn
Apr 10, 2026
Merged
Add FAQ entry for API server memory growth with gunicorn worker recycling#65036
kaxil merged 2 commits into
apache:mainfrom 
astronomer:faq-api-server-memory-gunicorn
Conversation
@kaxil
@kaxilkaxil
commented
Apr 10, 2026

Copy link


Copy Markdown

Member
Adds an FAQ entry documenting gunicorn with rolling worker restarts as the recommended solution for API server memory growth.
The API server's DBDagBag caches serialized DAG objects in an unbounded dict that grows as DAG versions accumulate. Users hitting this in production (see #60804, #64326) may not know that gunicorn worker recycling (added in #60940) already handles this by periodically restarting workers and releasing all accumulated memory.
The new FAQ entry:
  • Explains the problem (serialized DAG cache growth) and links to the existing dag version inflation FAQ
  • Documents the gunicorn + worker_refresh_interval configuration
  • Notes that worker recycling handles memory growth from any source, not just the DAG cache
@kaxil
Add FAQ entry for API server memory growth with gunicorn fix
24fddfe
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
@kaxilkaxil
added this to the Airflow 3.2.1 milestone
Apr 10, 2026
dheerajturaga
dheerajturaga
reviewed
Apr 10, 2026
View reviewed changes
@dheerajturagadheerajturaga
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a comment mentioning that this is available starting 3.2/3.2.1
Other than that LGTM!
@kaxil
kaxil
merged commit 74283fb
into
apache:mainApr 10, 2026

67 checks passed

@kaxil
kaxil
deleted the

faq-api-server-memory-gunicorn

branch
April 10, 2026 23:56
github-actionsBot
pushed a commit
that referenced
this pull request
Apr 10, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6667871
…worker recycling (#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
@github-actions
github-actionsBot
commented
Apr 10, 2026

Copy link


Copy Markdown

Contributor
Backport successfully created: v3-2-test
Note: As of Merging PRs targeted for Airflow 3.X

the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
In matter of doubt please ask in #release-management Slack channel.
StatusBranchResult
✅v3-2-testPR Link
github-actionsBot
pushed a commit
to aws-mwaa/upstream-to-airflow
that referenced
this pull request
Apr 11, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6152a53
…worker recycling (apache#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
kaxil added a commit
that referenced
this pull request
Apr 11, 2026
@github-actions@kaxil
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ff70972
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
5110dc9
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
8c36139
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
34916cf
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ad89865
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@dheerajturagadheerajturagadheerajturaga approved these changes
@pierrejeambrunpierrejeambrunAwaiting requested review from pierrejeambrun

Assignees


No one assigned


Labels

kind:documentation

Projects


None yet


Milestone

Airflow 3.2.1

Development

Successfully merging this pull request may close these issues.

2 participants

@kaxil@dheerajturaga

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Add FAQ entry for API server memory growth with gunicorn worker recycling by kaxil · Pull Request #65036 · apache/airflow · GitHub
Skip to content
Add FAQ entry for API server memory growth with gunicorn worker recycling - #65036
Merged
kaxil merged 2 commits into
apache:mainfrom 
astronomer:faq-api-server-memory-gunicorn
Apr 10, 2026
Merged
Add FAQ entry for API server memory growth with gunicorn worker recycling#65036
kaxil merged 2 commits into
apache:mainfrom 
astronomer:faq-api-server-memory-gunicorn
Conversation
@kaxil
@kaxilkaxil
commented
Apr 10, 2026

Copy link


Copy Markdown

Member
Adds an FAQ entry documenting gunicorn with rolling worker restarts as the recommended solution for API server memory growth.
The API server's DBDagBag caches serialized DAG objects in an unbounded dict that grows as DAG versions accumulate. Users hitting this in production (see #60804, #64326) may not know that gunicorn worker recycling (added in #60940) already handles this by periodically restarting workers and releasing all accumulated memory.
The new FAQ entry:
  • Explains the problem (serialized DAG cache growth) and links to the existing dag version inflation FAQ
  • Documents the gunicorn + worker_refresh_interval configuration
  • Notes that worker recycling handles memory growth from any source, not just the DAG cache
@kaxil
Add FAQ entry for API server memory growth with gunicorn fix
24fddfe
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
@kaxilkaxil
added this to the Airflow 3.2.1 milestone
Apr 10, 2026
dheerajturaga
dheerajturaga
reviewed
Apr 10, 2026
View reviewed changes
@dheerajturagadheerajturaga
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a comment mentioning that this is available starting 3.2/3.2.1
Other than that LGTM!
@kaxil
kaxil
merged commit 74283fb
into
apache:mainApr 10, 2026

67 checks passed

@kaxil
kaxil
deleted the

faq-api-server-memory-gunicorn

branch
April 10, 2026 23:56
github-actionsBot
pushed a commit
that referenced
this pull request
Apr 10, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6667871
…worker recycling (#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
@github-actions
github-actionsBot
commented
Apr 10, 2026

Copy link


Copy Markdown

Contributor
Backport successfully created: v3-2-test
Note: As of Merging PRs targeted for Airflow 3.X

the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
In matter of doubt please ask in #release-management Slack channel.
StatusBranchResult
✅v3-2-testPR Link
github-actionsBot
pushed a commit
to aws-mwaa/upstream-to-airflow
that referenced
this pull request
Apr 11, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6152a53
…worker recycling (apache#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
kaxil added a commit
that referenced
this pull request
Apr 11, 2026
@github-actions@kaxil
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ff70972
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
5110dc9
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
8c36139
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
34916cf
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ad89865
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@dheerajturagadheerajturagadheerajturaga approved these changes
@pierrejeambrunpierrejeambrunAwaiting requested review from pierrejeambrun

Assignees


No one assigned


Labels

kind:documentation

Projects


None yet


Milestone

Airflow 3.2.1

Development

Successfully merging this pull request may close these issues.

2 participants

@kaxil@dheerajturaga

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
Add FAQ entry for API server memory growth with gunicorn worker recycling by kaxil · Pull Request #65036 · apache/airflow · GitHub
Skip to content
Add FAQ entry for API server memory growth with gunicorn worker recycling - #65036
Merged
kaxil merged 2 commits into
apache:mainfrom 
astronomer:faq-api-server-memory-gunicorn
Apr 10, 2026
Merged
Add FAQ entry for API server memory growth with gunicorn worker recycling#65036
kaxil merged 2 commits into
apache:mainfrom 
astronomer:faq-api-server-memory-gunicorn
Conversation
@kaxil
@kaxilkaxil
commented
Apr 10, 2026

Copy link


Copy Markdown

Member
Adds an FAQ entry documenting gunicorn with rolling worker restarts as the recommended solution for API server memory growth.
The API server's DBDagBag caches serialized DAG objects in an unbounded dict that grows as DAG versions accumulate. Users hitting this in production (see #60804, #64326) may not know that gunicorn worker recycling (added in #60940) already handles this by periodically restarting workers and releasing all accumulated memory.
The new FAQ entry:
  • Explains the problem (serialized DAG cache growth) and links to the existing dag version inflation FAQ
  • Documents the gunicorn + worker_refresh_interval configuration
  • Notes that worker recycling handles memory growth from any source, not just the DAG cache
@kaxil
Add FAQ entry for API server memory growth with gunicorn fix
24fddfe
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
@kaxilkaxil
added this to the Airflow 3.2.1 milestone
Apr 10, 2026
dheerajturaga
dheerajturaga
reviewed
Apr 10, 2026
View reviewed changes
@dheerajturagadheerajturaga
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a comment mentioning that this is available starting 3.2/3.2.1
Other than that LGTM!
@kaxil
kaxil
merged commit 74283fb
into
apache:mainApr 10, 2026

67 checks passed

@kaxil
kaxil
deleted the

faq-api-server-memory-gunicorn

branch
April 10, 2026 23:56
github-actionsBot
pushed a commit
that referenced
this pull request
Apr 10, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6667871
…worker recycling (#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
@github-actions
github-actionsBot
commented
Apr 10, 2026

Copy link


Copy Markdown

Contributor
Backport successfully created: v3-2-test
Note: As of Merging PRs targeted for Airflow 3.X

the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
In matter of doubt please ask in #release-management Slack channel.
StatusBranchResult
✅v3-2-testPR Link
github-actionsBot
pushed a commit
to aws-mwaa/upstream-to-airflow
that referenced
this pull request
Apr 11, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6152a53
…worker recycling (apache#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
kaxil added a commit
that referenced
this pull request
Apr 11, 2026
@github-actions@kaxil
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ff70972
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
5110dc9
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
8c36139
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
34916cf
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ad89865
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@dheerajturagadheerajturagadheerajturaga approved these changes
@pierrejeambrunpierrejeambrunAwaiting requested review from pierrejeambrun

Assignees


No one assigned


Labels

kind:documentation

Projects


None yet


Milestone

Airflow 3.2.1

Development

Successfully merging this pull request may close these issues.

2 participants

@kaxil@dheerajturaga

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Add FAQ entry for API server memory growth with gunicorn worker recycling by kaxil · Pull Request #65036 · apache/airflow · GitHub
Skip to content
Add FAQ entry for API server memory growth with gunicorn worker recycling - #65036
Merged
kaxil merged 2 commits into
apache:mainfrom 
astronomer:faq-api-server-memory-gunicorn
Apr 10, 2026
Merged
Add FAQ entry for API server memory growth with gunicorn worker recycling#65036
kaxil merged 2 commits into
apache:mainfrom 
astronomer:faq-api-server-memory-gunicorn
Conversation
@kaxil
@kaxilkaxil
commented
Apr 10, 2026

Copy link


Copy Markdown

Member
Adds an FAQ entry documenting gunicorn with rolling worker restarts as the recommended solution for API server memory growth.
The API server's DBDagBag caches serialized DAG objects in an unbounded dict that grows as DAG versions accumulate. Users hitting this in production (see #60804, #64326) may not know that gunicorn worker recycling (added in #60940) already handles this by periodically restarting workers and releasing all accumulated memory.
The new FAQ entry:
  • Explains the problem (serialized DAG cache growth) and links to the existing dag version inflation FAQ
  • Documents the gunicorn + worker_refresh_interval configuration
  • Notes that worker recycling handles memory growth from any source, not just the DAG cache
@kaxil
Add FAQ entry for API server memory growth with gunicorn fix
24fddfe
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
@kaxilkaxil
added this to the Airflow 3.2.1 milestone
Apr 10, 2026
dheerajturaga
dheerajturaga
reviewed
Apr 10, 2026
View reviewed changes
@dheerajturagadheerajturaga
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a comment mentioning that this is available starting 3.2/3.2.1
Other than that LGTM!
@kaxil
kaxil
merged commit 74283fb
into
apache:mainApr 10, 2026

67 checks passed

@kaxil
kaxil
deleted the

faq-api-server-memory-gunicorn

branch
April 10, 2026 23:56
github-actionsBot
pushed a commit
that referenced
this pull request
Apr 10, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6667871
…worker recycling (#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
@github-actions
github-actionsBot
commented
Apr 10, 2026

Copy link


Copy Markdown

Contributor
Backport successfully created: v3-2-test
Note: As of Merging PRs targeted for Airflow 3.X

the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
In matter of doubt please ask in #release-management Slack channel.
StatusBranchResult
✅v3-2-testPR Link
github-actionsBot
pushed a commit
to aws-mwaa/upstream-to-airflow
that referenced
this pull request
Apr 11, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6152a53
…worker recycling (apache#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
kaxil added a commit
that referenced
this pull request
Apr 11, 2026
@github-actions@kaxil
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ff70972
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
5110dc9
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
8c36139
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
34916cf
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ad89865
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@dheerajturagadheerajturagadheerajturaga approved these changes
@pierrejeambrunpierrejeambrunAwaiting requested review from pierrejeambrun

Assignees


No one assigned


Labels

kind:documentation

Projects


None yet


Milestone

Airflow 3.2.1

Development

Successfully merging this pull request may close these issues.

2 participants

@kaxil@dheerajturaga

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
})();
Add FAQ entry for API server memory growth with gunicorn worker recycling by kaxil · Pull Request #65036 · apache/airflow · GitHub
Skip to content
Add FAQ entry for API server memory growth with gunicorn worker recycling - #65036
Merged
kaxil merged 2 commits into
apache:mainfrom 
astronomer:faq-api-server-memory-gunicorn
Apr 10, 2026
Merged
Add FAQ entry for API server memory growth with gunicorn worker recycling#65036
kaxil merged 2 commits into
apache:mainfrom 
astronomer:faq-api-server-memory-gunicorn
Conversation
@kaxil
@kaxilkaxil
commented
Apr 10, 2026

Copy link


Copy Markdown

Member
Adds an FAQ entry documenting gunicorn with rolling worker restarts as the recommended solution for API server memory growth.
The API server's DBDagBag caches serialized DAG objects in an unbounded dict that grows as DAG versions accumulate. Users hitting this in production (see #60804, #64326) may not know that gunicorn worker recycling (added in #60940) already handles this by periodically restarting workers and releasing all accumulated memory.
The new FAQ entry:
  • Explains the problem (serialized DAG cache growth) and links to the existing dag version inflation FAQ
  • Documents the gunicorn + worker_refresh_interval configuration
  • Notes that worker recycling handles memory growth from any source, not just the DAG cache
@kaxil
Add FAQ entry for API server memory growth with gunicorn fix
24fddfe
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
@kaxilkaxil
added this to the Airflow 3.2.1 milestone
Apr 10, 2026
dheerajturaga
dheerajturaga
reviewed
Apr 10, 2026
View reviewed changes
@dheerajturagadheerajturaga
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a comment mentioning that this is available starting 3.2/3.2.1
Other than that LGTM!
@kaxil
kaxil
merged commit 74283fb
into
apache:mainApr 10, 2026

67 checks passed

@kaxil
kaxil
deleted the

faq-api-server-memory-gunicorn

branch
April 10, 2026 23:56
github-actionsBot
pushed a commit
that referenced
this pull request
Apr 10, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6667871
…worker recycling (#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
@github-actions
github-actionsBot
commented
Apr 10, 2026

Copy link


Copy Markdown

Contributor
Backport successfully created: v3-2-test
Note: As of Merging PRs targeted for Airflow 3.X

the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
In matter of doubt please ask in #release-management Slack channel.
StatusBranchResult
✅v3-2-testPR Link
github-actionsBot
pushed a commit
to aws-mwaa/upstream-to-airflow
that referenced
this pull request
Apr 11, 2026
@kaxilYour friendly bot
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
6152a53
…worker recycling (apache#65036)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
kaxil added a commit
that referenced
this pull request
Apr 11, 2026
@github-actions@kaxil
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ff70972
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
5110dc9
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
8c36139
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
34916cf
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
vatsrahul1001 pushed a commit
that referenced
this pull request
Apr 15, 2026
@github-actions@kaxil@vatsrahul1001
[v3-2-test] Add FAQ entry for API server memory growth with gunicorn …
ad89865
…worker recycling (#65036) (#65037)
* Add FAQ entry for API server memory growth with gunicorn fix
Users experiencing API server memory growth from accumulated
serialized DAG cache entries can use gunicorn with rolling worker
restarts as the recommended solution. This FAQ entry documents the
configuration and cross-references the dag version inflation FAQ.
* Add version availability note for gunicorn support
(cherry picked from commit 74283fb)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@dheerajturagadheerajturagadheerajturaga approved these changes
@pierrejeambrunpierrejeambrunAwaiting requested review from pierrejeambrun

Assignees


No one assigned


Labels

kind:documentation

Projects


None yet


Milestone

Airflow 3.2.1

Development

Successfully merging this pull request may close these issues.

2 participants

@kaxil@dheerajturaga