Add configurable LRU+TTL caching for API server DAG retrieval#60804

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kaxil merged 7 commits into from
Apr 17, 2026
Merged

Add configurable LRU+TTL caching for API server DAG retrieval #60804

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
9ed6486
Add configurable LRU+TTL caching for API server DAG retrieval
kaxil Apr 11, 2026
d3dc177
Add docs and newsfragment for DAG cache feature
kaxil Apr 11, 2026
99faf78
Add bounded DAG caching section to API server memory FAQ
kaxil Apr 11, 2026
f8b3398
Fix config wording: cache_size=0 means unbounded dict, not disabled
kaxil Apr 11, 2026
03ed0f8
Fix version_added to 3.3.0 and update FAQ version
kaxil Apr 15, 2026
dcb2ab5
Remove unbounded _dag_models dict
kaxil Apr 15, 2026
d31c6ef
Apply suggestions from code review
kaxil Apr 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions airflow-core/docs/administration-and-deployment/web-stack.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -142,6 +142,8 @@ The following configuration options are available in the ``[api]`` section:
- ``server_type``: ``uvicorn`` (default) or ``gunicorn``
- ``worker_refresh_interval``: Seconds between worker refresh cycles (0 = disabled, default)
- ``worker_refresh_batch_size``: Number of workers to refresh per cycle (default: 1)
- ``dag_cache_size``: Max cached SerializedDAG versions in the API server (default: 64, 0 = unbounded)
- ``dag_cache_ttl``: TTL in seconds for cached DAGs (default: 3600, 0 = LRU only)
- ``reload_on_plugin_change``: Reload when plugin files change (default: False)

When to Use Gunicorn
Expand DownExpand Up@@ -187,6 +189,10 @@ For example, to trigger a rolling restart of the API server pods:

kubectl rollout restart deployment airflow-api-server

The API server also supports bounded DAG caching via ``dag_cache_size`` and
``dag_cache_ttl``, which limits memory consumed by cached SerializedDAG objects.
This reduces memory growth from DAG version accumulation regardless of server type.

In many Kubernetes environments, relying solely on Kubernetes OOM kills or
crash restarts is not recommended, as memory growth may not always trigger an
OOM event. For production deployments that require automatic worker recycling
Expand Down
29 changes: 25 additions & 4 deletions airflow-core/docs/faq.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -677,8 +677,28 @@ How to prevent API server memory growth?
The API server caches serialized Dag objects in memory. Over time, as Dag versions accumulate
(see :ref:`faq:dag-version-inflation`), this cache grows and can consume several gigabytes of memory.

The recommended solution (available since Airflow 3.2.0) is to use **gunicorn** with **rolling worker
restarts**. Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
There are two complementary approaches:

**1. Bounded DAG caching (available since Airflow 3.3.0)**

The API server supports LRU+TTL caching that bounds how many serialized Dag versions are kept
in memory. Configure this in the ``[api]`` section:

.. code-block:: ini

[api]
dag_cache_size = 64 ; max cached versions (0 = unbounded, pre-3.2 behavior)
dag_cache_ttl = 3600 ; seconds before a cached entry expires (0 = LRU only)

The cache is keyed by Dag version ID. After a Dag is updated, the API server may serve the
previous version until the cached entry expires (controlled by ``dag_cache_ttl``).

See :ref:`config:api__dag_cache_size` and :ref:`config:api__dag_cache_ttl` for the full
configuration reference.

**2. Gunicorn with rolling worker restarts (available since Airflow 3.2.0)**

Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
uses ``preload`` + ``fork``, so workers share read-only memory pages via copy-on-write, reducing overall
memory usage by 40-50% compared to uvicorn's multiprocess mode.

Expand All@@ -699,8 +719,9 @@ See :ref:`config:api__server_type`, :ref:`config:api__worker_refresh_interval`,

.. note::

Worker recycling handles memory growth from *any* source, not just the Dag cache. It is the
recommended approach for production API server deployments.
Worker recycling handles memory growth from *any* source, not just the Dag cache.
For production deployments, using both bounded caching and gunicorn worker recycling
provides the best results.


MySQL and MySQL variant Databases
Expand Down
1 change: 1 addition & 0 deletions airflow-core/newsfragments/60804.feature.rst
View file
Open in desktop
Comment thread
kaxil marked this conversation as resolved.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
Add configurable LRU+TTL caching for API server DAG retrieval via ``dag_cache_size`` and ``dag_cache_ttl`` config options in the ``[api]`` section. This bounds memory growth from accumulated SerializedDAG objects in long-running API server processes.
1 change: 1 addition & 0 deletions airflow-core/pyproject.toml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,7 @@ version = "3.3.0"

dependencies = [
"a2wsgi>=1.10.8",
"cachetools>=6.0.0",
# aiosqlite 0.22.0 has a problem with hanging pytest sessions and we excluded it
# See https://github.com/omnilib/aiosqlite/issues/369
# It seems that while our test issues are fixed in 0.22.1, sqlalchemy 2 itself
Expand Down
25 changes: 23 additions & 2 deletions airflow-core/src/airflow/api_fastapi/common/dagbag.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -16,21 +16,42 @@
# under the License.
from __future__ import annotations

import logging
from typing import TYPE_CHECKING, Annotated

from fastapi import Depends, HTTPException, Request, status
from sqlalchemy.orm import Session

from airflow.configuration import conf
from airflow.models.dagbag import DBDagBag

if TYPE_CHECKING:
from airflow.models.dagrun import DagRun
from airflow.serialization.definitions.dag import SerializedDAG

log = logging.getLogger(__name__)


def create_dag_bag() -> DBDagBag:
"""Create DagBag to retrieve DAGs from the database."""
return DBDagBag()
"""Create DagBag with configurable LRU+TTL caching for API server usage."""
cache_size = conf.getint("api", "dag_cache_size", fallback=64)
cache_ttl_config = conf.getint("api", "dag_cache_ttl", fallback=3600)

if cache_size < 0:
log.warning("dag_cache_size must be >= 0, using unbounded dict")
cache_size = 0
if cache_ttl_config < 0:
log.warning("dag_cache_ttl must be >= 0, disabling TTL")
cache_ttl_config = 0

# Use unbounded dict (no eviction) if cache_size is 0
if cache_size <= 0:
return DBDagBag(cache_size=0)

# Disable TTL if cache_ttl is 0
Comment thread
kaxil marked this conversation as resolved.
cache_ttl: int | None = cache_ttl_config if cache_ttl_config > 0 else None

return DBDagBag(cache_size=cache_size, cache_ttl=cache_ttl)


def dag_bag_from_app(request: Request) -> DBDagBag:
Expand Down
24 changes: 24 additions & 0 deletions airflow-core/src/airflow/config_templates/config.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1606,6 +1606,30 @@
type: string
example: ~
default: "False"
dag_cache_size:
description: |
Size of the LRU cache for SerializedDAG objects in the API server.
Set to 0 to use an unbounded dict (no eviction, pre-3.2 behavior). The cache is keyed by DAG version ID,

Check failure on line 1612 in airflow-core/src/airflow/config_templates/config.yml

View workflow job for this annotation

GitHub Actions/ CI image checks / Static checks

1612:111 [line-length] line too long (112 > 110 characters)
so lookups by DAG ID (e.g., viewing a DAG's details) always query
the database for the latest version, but the deserialized result is
cached for subsequent version-specific lookups.
version_added: 3.3.0
type: integer
example: ~
default: "64"
dag_cache_ttl:
description: |
Time-to-live (seconds) for cached SerializedDAG objects in the API server.
After this time, cached DAGs will be re-fetched from the database on next access.
Set to 0 to disable TTL (cache entries will only be evicted by LRU policy).

Note: After a DAG is updated, the API server may serve the previous version
until the cached entry expires. Lower values reduce staleness but increase
database load.
version_added: 3.3.0
type: integer
example: ~
default: "3600"
base_url:
description: |
The base url of the API server. Airflow cannot guess what domain or CNAME you are using.
Expand Down
152 changes: 115 additions & 37 deletions airflow-core/src/airflow/models/dagbag.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -18,12 +18,17 @@
from __future__ import annotations

import hashlib
from collections.abc import MutableMapping
from contextlib import nullcontext
from threading import RLock
from typing import TYPE_CHECKING, Any
from uuid import UUID

from cachetools import LRUCache, TTLCache
from sqlalchemy import String, select
from sqlalchemy.orm import Mapped, joinedload, mapped_column

from airflow._shared.observability.metrics.stats import Stats
from airflow.models.base import Base, StringID
from airflow.models.dag_version import DagVersion

Expand All@@ -39,50 +44,117 @@

class DBDagBag:
"""
Internal class for retrieving and caching dags in the scheduler.
Internal class for retrieving dags from the database.

Optionally supports LRU+TTL caching when cache_size is provided.
The scheduler uses this without caching, while the API server can
enable caching via configuration.

:meta private:
"""

def __init__(self, load_op_links: bool = True) -> None:
self._dags: dict[UUID, SerializedDagModel] = {} # dag_version_id to dag
self.load_op_links = load_op_links
def __init__(
self,
load_op_links: bool = True,
cache_size: int | None = None,
cache_ttl: int | None = None,
) -> None:
"""
Initialize DBDagBag.

def _read_dag(self, serialized_dag_model: SerializedDagModel) -> SerializedDAG | None:
serialized_dag_model.load_op_links = self.load_op_links
if dag := serialized_dag_model.dag:
self._dags[serialized_dag_model.dag_version_id] = serialized_dag_model
:param load_op_links: Should the extra operator link be loaded when de-serializing the DAG?
:param cache_size: Size of LRU cache. If None or 0, uses unbounded dict (no eviction).
:param cache_ttl: Time-to-live for cache entries in seconds. If None or 0, no TTL (LRU only).
"""
self.load_op_links = load_op_links
self._dags: MutableMapping[UUID | str, SerializedDAG] = {}
self._use_cache = False

# Initialize bounded cache if cache_size is provided and > 0
if cache_size and cache_size > 0:
Comment thread
kaxil marked this conversation as resolved.
if cache_ttl and cache_ttl > 0:
self._dags = TTLCache(maxsize=cache_size, ttl=cache_ttl)
else:
self._dags = LRUCache(maxsize=cache_size)
self._use_cache = True

# Lock required for bounded caches: cachetools caches are NOT thread-safe
# (LRU reordering and TTL cleanup mutate internal linked lists).
# nullcontext for unbounded dict avoids lock overhead in the scheduler path.
self._lock: RLock | nullcontext = RLock() if self._use_cache else nullcontext()

Comment thread
kaxil marked this conversation as resolved.
def _read_dag(self, serdag: SerializedDagModel) -> SerializedDAG | None:
"""Read and optionally cache a SerializedDAG from a SerializedDagModel."""
serdag.load_op_links = self.load_op_links
dag = serdag.dag
if not dag:
return None
with self._lock:
self._dags[serdag.dag_version_id] = dag
cache_size = len(self._dags)
if self._use_cache:
Stats.gauge("api_server.dag_bag.cache_size", cache_size, rate=0.1)
return dag

def get_serialized_dag_model(self, version_id: UUID, session: Session) -> SerializedDagModel | None:
def _get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
# Check cache first
with self._lock:
dag = self._dags.get(version_id)

if dag:
if self._use_cache:
Comment thread
kaxil marked this conversation as resolved.
Stats.incr("api_server.dag_bag.cache_hit")
return dag

dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
if not dag_version:
return None
if not (serdag := dag_version.serialized_dag):
return None

# Double-checked locking: another thread may have cached it while we queried DB.
# Only emit the miss metric after confirming no other thread cached it, to avoid
# counting a single lookup as both a miss and a hit.
if self._use_cache:
with self._lock:
if dag := self._dags.get(version_id):
Comment thread
kaxil marked this conversation as resolved.
Stats.incr("api_server.dag_bag.cache_hit")
return dag
Stats.incr("api_server.dag_bag.cache_miss")
return self._read_dag(serdag)
Comment thread
kaxil marked this conversation as resolved.

def get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
"""Get a dag by its version id, using cache if enabled."""
return self._get_dag(version_id=version_id, session=session)

def get_serialized_dag_model(self, version_id: UUID | str, session: Session) -> SerializedDagModel | None:
"""
Return the SerializedDagModel for a given dag version id.

This will first consult the in-memory cache keyed by the dag version id. If the
model is not cached, the database is queried for a corresponding :class:`DagVersion`
and its associated :class:`SerializedDagModel`.
Always queries the database. The triggerer needs the full model
for ``serialized_dag_model.data``, which cannot be stored in the
LRU/TTL cache (it stores deserialized SerializedDAG objects).
"""
dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
if not dag_version or not (serdag := dag_version.serialized_dag):
return None
serdag.load_op_links = self.load_op_links
return serdag

:param version_id: The UUID of the dag version to look up.
:param session: SQLAlchemy session used to query the database.
:return: The serialized DAG model if found either in the cache or the database; ``None``
is returned when no :class:`DagVersion` exists for the given ``version_id`` or
when that :class:`DagVersion` does not have an associated :class:`SerializedDagModel`.
:rtype: SerializedDagModel | None
def clear_cache(self) -> int:
"""
Clear all cached DAGs and serialized DAG models.

Note: If a serialized dag model is found in the database it will be stored in the
internal cache (``self._dags``) before being returned.
:return: Number of entries cleared from the DAG cache.
"""
Comment thread
kaxil marked this conversation as resolved.
if not (serialized_dag_model := self._dags.get(version_id)):
dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
if not dag_version or not (serialized_dag_model := dag_version.serialized_dag):
return None
self._read_dag(serialized_dag_model)
return serialized_dag_model

def get_dag(self, version_id: UUID, session: Session) -> SerializedDAG | None:
if serialized_dag_model := self.get_serialized_dag_model(version_id=version_id, session=session):
return serialized_dag_model.dag
return None
with self._lock:
count = len(self._dags)
self._dags.clear()

if self._use_cache:
Stats.incr("api_server.dag_bag.cache_clear")
Stats.gauge("api_server.dag_bag.cache_size", 0)
return count
Comment thread
kaxil marked this conversation as resolved.

@staticmethod
def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:
Expand All@@ -94,24 +166,30 @@ def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:

def get_dag_for_run(self, dag_run: DagRun, session: Session) -> SerializedDAG | None:
if version_id := self._version_from_dag_run(dag_run=dag_run, session=session):
return self.get_dag(version_id=version_id, session=session)
return self._get_dag(version_id=version_id, session=session)
return None

def iter_all_latest_version_dags(self, *, session: Session) -> Generator[SerializedDAG, None, None]:
"""Walk through all latest version dags available in the database."""
"""
Walk through all latest version dags available in the database.

Note: This method does NOT cache the DAGs to avoid cache thrashing when
iterating over many DAGs. Each DAG is deserialized fresh from the database.
"""
from airflow.models.serialized_dag import SerializedDagModel

for serialized_dag_model in session.scalars(select(SerializedDagModel)):
if dag := self._read_dag(serialized_dag_model):
for sdm in session.scalars(select(SerializedDagModel)):
sdm.load_op_links = self.load_op_links
if dag := sdm.dag:
yield dag

def get_latest_version_of_dag(self, dag_id: str, *, session: Session) -> SerializedDAG | None:
"""Get the latest version of a dag by its id."""
from airflow.models.serialized_dag import SerializedDagModel

if not (serialized_dag_model := SerializedDagModel.get(dag_id, session=session)):
if not (serdag := SerializedDagModel.get(dag_id, session=session)):
return None
return self._read_dag(serialized_dag_model)
return self._read_dag(serdag)


def generate_md5_hash(context):
Expand Down
28 changes: 28 additions & 0 deletions airflow-core/tests/unit/api_fastapi/common/test_dagbag.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -19,6 +19,7 @@
from unittest import mock

import pytest
from cachetools import LRUCache, TTLCache

from airflow.api_fastapi.app import purge_cached_app
from airflow.sdk import BaseOperator
Expand DownExpand Up@@ -82,3 +83,30 @@ def test_dagbag_used_as_singleton_in_dependency(self, session, dag_maker, test_c
assert resp2.status_code == 200

assert self.dagbag_call_counter["count"] == 1


class TestCreateDagBag:
"""Tests for create_dag_bag() function."""

Comment thread
kaxil marked this conversation as resolved.
@pytest.mark.parametrize(
["cache_size", "cache_ttl", "expected_use_cache", "expected_dags_type"],
[
pytest.param(64, 3600, True, TTLCache, id="default_ttl_cache"),
pytest.param(0, 3600, False, dict, id="size_zero_unbounded"),
pytest.param(64, 0, True, LRUCache, id="ttl_zero_lru_only"),
],
)
@mock.patch("airflow.api_fastapi.common.dagbag.conf")
def test_create_dag_bag_cache_modes(
self, mock_conf, cache_size, cache_ttl, expected_use_cache, expected_dags_type
):
from airflow.api_fastapi.common.dagbag import create_dag_bag

mock_conf.getint.side_effect = lambda section, key, fallback: {
"dag_cache_size": cache_size,
"dag_cache_ttl": cache_ttl,
}.get(key, fallback)

dag_bag = create_dag_bag()
assert dag_bag._use_cache is expected_use_cache
assert isinstance(dag_bag._dags, expected_dags_type)
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Add configurable LRU+TTL caching for API server DAG retrieval#60804

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kaxil
merged 7 commits into

from

Apr 17, 2026
 Merged


Add configurable LRU+TTL caching for API server DAG retrieval
#60804
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
9ed6486

Add configurable LRU+TTL caching for API server DAG retrieval


kaxil Apr 11, 2026
d3dc177

Add docs and newsfragment for DAG cache feature


kaxil Apr 11, 2026
99faf78

Add bounded DAG caching section to API server memory FAQ


kaxil Apr 11, 2026
f8b3398

Fix config wording: cache_size=0 means unbounded dict, not disabled


kaxil Apr 11, 2026
03ed0f8

Fix version_added to 3.3.0 and update FAQ version


kaxil Apr 15, 2026
dcb2ab5

Remove unbounded _dag_models dict


kaxil Apr 15, 2026
d31c6ef

Apply suggestions from code review


kaxil Apr 17, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


6 changes: 6 additions & 0 deletions
airflow-core/docs/administration-and-deployment/web-stack.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -142,6 +142,8 @@ The following configuration options are available in the ``[api]`` section:
- ``server_type``: ``uvicorn`` (default) or ``gunicorn``
- ``worker_refresh_interval``: Seconds between worker refresh cycles (0 = disabled, default)
- ``worker_refresh_batch_size``: Number of workers to refresh per cycle (default: 1)
- ``dag_cache_size``: Max cached SerializedDAG versions in the API server (default: 64, 0 = unbounded)
- ``dag_cache_ttl``: TTL in seconds for cached DAGs (default: 3600, 0 = LRU only)
- ``reload_on_plugin_change``: Reload when plugin files change (default: False)

When to Use Gunicorn
Expand DownExpand Up@@ -187,6 +189,10 @@ For example, to trigger a rolling restart of the API server pods:

 kubectl rollout restart deployment airflow-api-server

The API server also supports bounded DAG caching via ``dag_cache_size`` and
``dag_cache_ttl``, which limits memory consumed by cached SerializedDAG objects.
This reduces memory growth from DAG version accumulation regardless of server type.

In many Kubernetes environments, relying solely on Kubernetes OOM kills or
crash restarts is not recommended, as memory growth may not always trigger an
OOM event. For production deployments that require automatic worker recycling
Expand Down

29 changes: 25 additions & 4 deletions
airflow-core/docs/faq.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -677,8 +677,28 @@ How to prevent API server memory growth?
The API server caches serialized Dag objects in memory. Over time, as Dag versions accumulate
(see :ref:`faq:dag-version-inflation`), this cache grows and can consume several gigabytes of memory.

The recommended solution (available since Airflow 3.2.0) is to use **gunicorn** with **rolling worker
restarts**. Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
There are two complementary approaches:

**1. Bounded DAG caching (available since Airflow 3.3.0)**

The API server supports LRU+TTL caching that bounds how many serialized Dag versions are kept
in memory. Configure this in the ``[api]`` section:

.. code-block:: ini

 [api]
 dag_cache_size = 64 ; max cached versions (0 = unbounded, pre-3.2 behavior)
 dag_cache_ttl = 3600 ; seconds before a cached entry expires (0 = LRU only)

The cache is keyed by Dag version ID. After a Dag is updated, the API server may serve the
previous version until the cached entry expires (controlled by ``dag_cache_ttl``).

See :ref:`config:api__dag_cache_size` and :ref:`config:api__dag_cache_ttl` for the full
configuration reference.

**2. Gunicorn with rolling worker restarts (available since Airflow 3.2.0)**

Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
uses ``preload`` + ``fork``, so workers share read-only memory pages via copy-on-write, reducing overall
memory usage by 40-50% compared to uvicorn's multiprocess mode.

Expand All@@ -699,8 +719,9 @@ See :ref:`config:api__server_type`, :ref:`config:api__worker_refresh_interval`,

.. note::

 Worker recycling handles memory growth from *any* source, not just the Dag cache. It is the
 recommended approach for production API server deployments.
 Worker recycling handles memory growth from *any* source, not just the Dag cache.
 For production deployments, using both bounded caching and gunicorn worker recycling
 provides the best results.


MySQL and MySQL variant Databases
Expand Down

1 change: 1 addition & 0 deletions
airflow-core/newsfragments/60804.feature.rst

View file


Open in desktop

Comment thread
kaxil marked this conversation as resolved.

Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
Add configurable LRU+TTL caching for API server DAG retrieval via ``dag_cache_size`` and ``dag_cache_ttl`` config options in the ``[api]`` section. This bounds memory growth from accumulated SerializedDAG objects in long-running API server processes.

1 change: 1 addition & 0 deletions
airflow-core/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,7 @@ version = "3.3.0"

dependencies = [
 "a2wsgi>=1.10.8",
 "cachetools>=6.0.0",
 # aiosqlite 0.22.0 has a problem with hanging pytest sessions and we excluded it
 # See https://github.com/omnilib/aiosqlite/issues/369
 # It seems that while our test issues are fixed in 0.22.1, sqlalchemy 2 itself
Expand Down

25 changes: 23 additions & 2 deletions
airflow-core/src/airflow/api_fastapi/common/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,21 +16,42 @@
# under the License.
from __future__ import annotations

import logging
from typing import TYPE_CHECKING, Annotated

from fastapi import Depends, HTTPException, Request, status
from sqlalchemy.orm import Session

from airflow.configuration import conf
from airflow.models.dagbag import DBDagBag

if TYPE_CHECKING:
 from airflow.models.dagrun import DagRun
 from airflow.serialization.definitions.dag import SerializedDAG

log = logging.getLogger(__name__)


def create_dag_bag() -> DBDagBag:
 """Create DagBag to retrieve DAGs from the database."""
 return DBDagBag()
 """Create DagBag with configurable LRU+TTL caching for API server usage."""
 cache_size = conf.getint("api", "dag_cache_size", fallback=64)
 cache_ttl_config = conf.getint("api", "dag_cache_ttl", fallback=3600)

 if cache_size < 0:
 log.warning("dag_cache_size must be >= 0, using unbounded dict")
 cache_size = 0
 if cache_ttl_config < 0:
 log.warning("dag_cache_ttl must be >= 0, disabling TTL")
 cache_ttl_config = 0

 # Use unbounded dict (no eviction) if cache_size is 0
 if cache_size <= 0:
 return DBDagBag(cache_size=0)

 # Disable TTL if cache_ttl is 0
Comment thread
kaxil marked this conversation as resolved.

 cache_ttl: int | None = cache_ttl_config if cache_ttl_config > 0 else None

 return DBDagBag(cache_size=cache_size, cache_ttl=cache_ttl)


def dag_bag_from_app(request: Request) -> DBDagBag:
Expand Down

24 changes: 24 additions & 0 deletions
airflow-core/src/airflow/config_templates/config.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1606,6 +1606,30 @@
 type: string
 example: ~
 default: "False"
 dag_cache_size:
 description: |
 Size of the LRU cache for SerializedDAG objects in the API server.
 Set to 0 to use an unbounded dict (no eviction, pre-3.2 behavior). The cache is keyed by DAG version ID,

Check failure
on line 1612
in airflow-core/src/airflow/config_templates/config.yml

View workflow job for this annotation
GitHub Actions/ CI image checks / Static checks
1612:111 [line-length] line too long (112 > 110 characters)
 so lookups by DAG ID (e.g., viewing a DAG's details) always query
 the database for the latest version, but the deserialized result is
 cached for subsequent version-specific lookups.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "64"
 dag_cache_ttl:
 description: |
 Time-to-live (seconds) for cached SerializedDAG objects in the API server.
 After this time, cached DAGs will be re-fetched from the database on next access.
 Set to 0 to disable TTL (cache entries will only be evicted by LRU policy).

 Note: After a DAG is updated, the API server may serve the previous version
 until the cached entry expires. Lower values reduce staleness but increase
 database load.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "3600"
 base_url:
 description: |
 The base url of the API server. Airflow cannot guess what domain or CNAME you are using.
Expand Down

152 changes: 115 additions & 37 deletions
airflow-core/src/airflow/models/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -18,12 +18,17 @@
from __future__ import annotations

import hashlib
from collections.abc import MutableMapping
from contextlib import nullcontext
from threading import RLock
from typing import TYPE_CHECKING, Any
from uuid import UUID

from cachetools import LRUCache, TTLCache
from sqlalchemy import String, select
from sqlalchemy.orm import Mapped, joinedload, mapped_column

from airflow._shared.observability.metrics.stats import Stats
from airflow.models.base import Base, StringID
from airflow.models.dag_version import DagVersion

Expand All@@ -39,50 +44,117 @@

class DBDagBag:
 """
 Internal class for retrieving and caching dags in the scheduler.
 Internal class for retrieving dags from the database.

 Optionally supports LRU+TTL caching when cache_size is provided.
 The scheduler uses this without caching, while the API server can
 enable caching via configuration.

 :meta private:
 """

 def __init__(self, load_op_links: bool = True) -> None:
 self._dags: dict[UUID, SerializedDagModel] = {} # dag_version_id to dag
 self.load_op_links = load_op_links
 def __init__(
 self,
 load_op_links: bool = True,
 cache_size: int | None = None,
 cache_ttl: int | None = None,
 ) -> None:
 """
 Initialize DBDagBag.

 def _read_dag(self, serialized_dag_model: SerializedDagModel) -> SerializedDAG | None:
 serialized_dag_model.load_op_links = self.load_op_links
 if dag := serialized_dag_model.dag:
 self._dags[serialized_dag_model.dag_version_id] = serialized_dag_model
 :param load_op_links: Should the extra operator link be loaded when de-serializing the DAG?
 :param cache_size: Size of LRU cache. If None or 0, uses unbounded dict (no eviction).
 :param cache_ttl: Time-to-live for cache entries in seconds. If None or 0, no TTL (LRU only).
 """
 self.load_op_links = load_op_links
 self._dags: MutableMapping[UUID | str, SerializedDAG] = {}
 self._use_cache = False

 # Initialize bounded cache if cache_size is provided and > 0
 if cache_size and cache_size > 0:
Comment thread
kaxil marked this conversation as resolved.

 if cache_ttl and cache_ttl > 0:
 self._dags = TTLCache(maxsize=cache_size, ttl=cache_ttl)
 else:
 self._dags = LRUCache(maxsize=cache_size)
 self._use_cache = True

 # Lock required for bounded caches: cachetools caches are NOT thread-safe
 # (LRU reordering and TTL cleanup mutate internal linked lists).
 # nullcontext for unbounded dict avoids lock overhead in the scheduler path.
 self._lock: RLock | nullcontext = RLock() if self._use_cache else nullcontext()

Comment thread
kaxil marked this conversation as resolved.

 def _read_dag(self, serdag: SerializedDagModel) -> SerializedDAG | None:
 """Read and optionally cache a SerializedDAG from a SerializedDagModel."""
 serdag.load_op_links = self.load_op_links
 dag = serdag.dag
 if not dag:
 return None
 with self._lock:
 self._dags[serdag.dag_version_id] = dag
 cache_size = len(self._dags)
 if self._use_cache:
 Stats.gauge("api_server.dag_bag.cache_size", cache_size, rate=0.1)
 return dag

 def get_serialized_dag_model(self, version_id: UUID, session: Session) -> SerializedDagModel | None:
 def _get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 # Check cache first
 with self._lock:
 dag = self._dags.get(version_id)

 if dag:
 if self._use_cache:
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag

 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version:
 return None
 if not (serdag := dag_version.serialized_dag):
 return None

 # Double-checked locking: another thread may have cached it while we queried DB.
 # Only emit the miss metric after confirming no other thread cached it, to avoid
 # counting a single lookup as both a miss and a hit.
 if self._use_cache:
 with self._lock:
 if dag := self._dags.get(version_id):
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag
 Stats.incr("api_server.dag_bag.cache_miss")
 return self._read_dag(serdag)
Comment thread
kaxil marked this conversation as resolved.


 def get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 """Get a dag by its version id, using cache if enabled."""
 return self._get_dag(version_id=version_id, session=session)

 def get_serialized_dag_model(self, version_id: UUID | str, session: Session) -> SerializedDagModel | None:
 """
 Return the SerializedDagModel for a given dag version id.

 This will first consult the in-memory cache keyed by the dag version id. If the
 model is not cached, the database is queried for a corresponding :class:`DagVersion`
 and its associated :class:`SerializedDagModel`.
 Always queries the database. The triggerer needs the full model
 for ``serialized_dag_model.data``, which cannot be stored in the
 LRU/TTL cache (it stores deserialized SerializedDAG objects).
 """
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serdag := dag_version.serialized_dag):
 return None
 serdag.load_op_links = self.load_op_links
 return serdag

 :param version_id: The UUID of the dag version to look up.
 :param session: SQLAlchemy session used to query the database.
 :return: The serialized DAG model if found either in the cache or the database; ``None``
 is returned when no :class:`DagVersion` exists for the given ``version_id`` or
 when that :class:`DagVersion` does not have an associated :class:`SerializedDagModel`.
 :rtype: SerializedDagModel | None
 def clear_cache(self) -> int:
 """
 Clear all cached DAGs and serialized DAG models.

 Note: If a serialized dag model is found in the database it will be stored in the
 internal cache (``self._dags``) before being returned.
 :return: Number of entries cleared from the DAG cache.
 """
Comment thread
kaxil marked this conversation as resolved.

 if not (serialized_dag_model := self._dags.get(version_id)):
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serialized_dag_model := dag_version.serialized_dag):
 return None
 self._read_dag(serialized_dag_model)
 return serialized_dag_model

 def get_dag(self, version_id: UUID, session: Session) -> SerializedDAG | None:
 if serialized_dag_model := self.get_serialized_dag_model(version_id=version_id, session=session):
 return serialized_dag_model.dag
 return None
 with self._lock:
 count = len(self._dags)
 self._dags.clear()

 if self._use_cache:
 Stats.incr("api_server.dag_bag.cache_clear")
 Stats.gauge("api_server.dag_bag.cache_size", 0)
 return count
Comment thread
kaxil marked this conversation as resolved.


 @staticmethod
 def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:
Expand All@@ -94,24 +166,30 @@ def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:

 def get_dag_for_run(self, dag_run: DagRun, session: Session) -> SerializedDAG | None:
 if version_id := self._version_from_dag_run(dag_run=dag_run, session=session):
 return self.get_dag(version_id=version_id, session=session)
 return self._get_dag(version_id=version_id, session=session)
 return None

 def iter_all_latest_version_dags(self, *, session: Session) -> Generator[SerializedDAG, None, None]:
 """Walk through all latest version dags available in the database."""
 """
 Walk through all latest version dags available in the database.

 Note: This method does NOT cache the DAGs to avoid cache thrashing when
 iterating over many DAGs. Each DAG is deserialized fresh from the database.
 """
 from airflow.models.serialized_dag import SerializedDagModel

 for serialized_dag_model in session.scalars(select(SerializedDagModel)):
 if dag := self._read_dag(serialized_dag_model):
 for sdm in session.scalars(select(SerializedDagModel)):
 sdm.load_op_links = self.load_op_links
 if dag := sdm.dag:
 yield dag

 def get_latest_version_of_dag(self, dag_id: str, *, session: Session) -> SerializedDAG | None:
 """Get the latest version of a dag by its id."""
 from airflow.models.serialized_dag import SerializedDagModel

 if not (serialized_dag_model := SerializedDagModel.get(dag_id, session=session)):
 if not (serdag := SerializedDagModel.get(dag_id, session=session)):
 return None
 return self._read_dag(serialized_dag_model)
 return self._read_dag(serdag)


def generate_md5_hash(context):
Expand Down

28 changes: 28 additions & 0 deletions
airflow-core/tests/unit/api_fastapi/common/test_dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,6 +19,7 @@
from unittest import mock

import pytest
from cachetools import LRUCache, TTLCache

from airflow.api_fastapi.app import purge_cached_app
from airflow.sdk import BaseOperator
Expand DownExpand Up@@ -82,3 +83,30 @@ def test_dagbag_used_as_singleton_in_dependency(self, session, dag_maker, test_c
 assert resp2.status_code == 200

 assert self.dagbag_call_counter["count"] == 1


class TestCreateDagBag:
 """Tests for create_dag_bag() function."""

Comment thread
kaxil marked this conversation as resolved.

 @pytest.mark.parametrize(
 ["cache_size", "cache_ttl", "expected_use_cache", "expected_dags_type"],
 [
 pytest.param(64, 3600, True, TTLCache, id="default_ttl_cache"),
 pytest.param(0, 3600, False, dict, id="size_zero_unbounded"),
 pytest.param(64, 0, True, LRUCache, id="ttl_zero_lru_only"),
 ],
 )
 @mock.patch("airflow.api_fastapi.common.dagbag.conf")
 def test_create_dag_bag_cache_modes(
 self, mock_conf, cache_size, cache_ttl, expected_use_cache, expected_dags_type
 ):
 from airflow.api_fastapi.common.dagbag import create_dag_bag

 mock_conf.getint.side_effect = lambda section, key, fallback: {
 "dag_cache_size": cache_size,
 "dag_cache_ttl": cache_ttl,
 }.get(key, fallback)

 dag_bag = create_dag_bag()
 assert dag_bag._use_cache is expected_use_cache
 assert isinstance(dag_bag._dags, expected_dags_type)
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Add configurable LRU+TTL caching for API server DAG retrieval#60804

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kaxil
merged 7 commits into

from

Apr 17, 2026
 Merged


Add configurable LRU+TTL caching for API server DAG retrieval
#60804
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
9ed6486

Add configurable LRU+TTL caching for API server DAG retrieval


kaxil Apr 11, 2026
d3dc177

Add docs and newsfragment for DAG cache feature


kaxil Apr 11, 2026
99faf78

Add bounded DAG caching section to API server memory FAQ


kaxil Apr 11, 2026
f8b3398

Fix config wording: cache_size=0 means unbounded dict, not disabled


kaxil Apr 11, 2026
03ed0f8

Fix version_added to 3.3.0 and update FAQ version


kaxil Apr 15, 2026
dcb2ab5

Remove unbounded _dag_models dict


kaxil Apr 15, 2026
d31c6ef

Apply suggestions from code review


kaxil Apr 17, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


6 changes: 6 additions & 0 deletions
airflow-core/docs/administration-and-deployment/web-stack.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -142,6 +142,8 @@ The following configuration options are available in the ``[api]`` section:
- ``server_type``: ``uvicorn`` (default) or ``gunicorn``
- ``worker_refresh_interval``: Seconds between worker refresh cycles (0 = disabled, default)
- ``worker_refresh_batch_size``: Number of workers to refresh per cycle (default: 1)
- ``dag_cache_size``: Max cached SerializedDAG versions in the API server (default: 64, 0 = unbounded)
- ``dag_cache_ttl``: TTL in seconds for cached DAGs (default: 3600, 0 = LRU only)
- ``reload_on_plugin_change``: Reload when plugin files change (default: False)

When to Use Gunicorn
Expand DownExpand Up@@ -187,6 +189,10 @@ For example, to trigger a rolling restart of the API server pods:

 kubectl rollout restart deployment airflow-api-server

The API server also supports bounded DAG caching via ``dag_cache_size`` and
``dag_cache_ttl``, which limits memory consumed by cached SerializedDAG objects.
This reduces memory growth from DAG version accumulation regardless of server type.

In many Kubernetes environments, relying solely on Kubernetes OOM kills or
crash restarts is not recommended, as memory growth may not always trigger an
OOM event. For production deployments that require automatic worker recycling
Expand Down

29 changes: 25 additions & 4 deletions
airflow-core/docs/faq.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -677,8 +677,28 @@ How to prevent API server memory growth?
The API server caches serialized Dag objects in memory. Over time, as Dag versions accumulate
(see :ref:`faq:dag-version-inflation`), this cache grows and can consume several gigabytes of memory.

The recommended solution (available since Airflow 3.2.0) is to use **gunicorn** with **rolling worker
restarts**. Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
There are two complementary approaches:

**1. Bounded DAG caching (available since Airflow 3.3.0)**

The API server supports LRU+TTL caching that bounds how many serialized Dag versions are kept
in memory. Configure this in the ``[api]`` section:

.. code-block:: ini

 [api]
 dag_cache_size = 64 ; max cached versions (0 = unbounded, pre-3.2 behavior)
 dag_cache_ttl = 3600 ; seconds before a cached entry expires (0 = LRU only)

The cache is keyed by Dag version ID. After a Dag is updated, the API server may serve the
previous version until the cached entry expires (controlled by ``dag_cache_ttl``).

See :ref:`config:api__dag_cache_size` and :ref:`config:api__dag_cache_ttl` for the full
configuration reference.

**2. Gunicorn with rolling worker restarts (available since Airflow 3.2.0)**

Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
uses ``preload`` + ``fork``, so workers share read-only memory pages via copy-on-write, reducing overall
memory usage by 40-50% compared to uvicorn's multiprocess mode.

Expand All@@ -699,8 +719,9 @@ See :ref:`config:api__server_type`, :ref:`config:api__worker_refresh_interval`,

.. note::

 Worker recycling handles memory growth from *any* source, not just the Dag cache. It is the
 recommended approach for production API server deployments.
 Worker recycling handles memory growth from *any* source, not just the Dag cache.
 For production deployments, using both bounded caching and gunicorn worker recycling
 provides the best results.


MySQL and MySQL variant Databases
Expand Down

1 change: 1 addition & 0 deletions
airflow-core/newsfragments/60804.feature.rst

View file


Open in desktop

Comment thread
kaxil marked this conversation as resolved.

Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
Add configurable LRU+TTL caching for API server DAG retrieval via ``dag_cache_size`` and ``dag_cache_ttl`` config options in the ``[api]`` section. This bounds memory growth from accumulated SerializedDAG objects in long-running API server processes.

1 change: 1 addition & 0 deletions
airflow-core/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,7 @@ version = "3.3.0"

dependencies = [
 "a2wsgi>=1.10.8",
 "cachetools>=6.0.0",
 # aiosqlite 0.22.0 has a problem with hanging pytest sessions and we excluded it
 # See https://github.com/omnilib/aiosqlite/issues/369
 # It seems that while our test issues are fixed in 0.22.1, sqlalchemy 2 itself
Expand Down

25 changes: 23 additions & 2 deletions
airflow-core/src/airflow/api_fastapi/common/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,21 +16,42 @@
# under the License.
from __future__ import annotations

import logging
from typing import TYPE_CHECKING, Annotated

from fastapi import Depends, HTTPException, Request, status
from sqlalchemy.orm import Session

from airflow.configuration import conf
from airflow.models.dagbag import DBDagBag

if TYPE_CHECKING:
 from airflow.models.dagrun import DagRun
 from airflow.serialization.definitions.dag import SerializedDAG

log = logging.getLogger(__name__)


def create_dag_bag() -> DBDagBag:
 """Create DagBag to retrieve DAGs from the database."""
 return DBDagBag()
 """Create DagBag with configurable LRU+TTL caching for API server usage."""
 cache_size = conf.getint("api", "dag_cache_size", fallback=64)
 cache_ttl_config = conf.getint("api", "dag_cache_ttl", fallback=3600)

 if cache_size < 0:
 log.warning("dag_cache_size must be >= 0, using unbounded dict")
 cache_size = 0
 if cache_ttl_config < 0:
 log.warning("dag_cache_ttl must be >= 0, disabling TTL")
 cache_ttl_config = 0

 # Use unbounded dict (no eviction) if cache_size is 0
 if cache_size <= 0:
 return DBDagBag(cache_size=0)

 # Disable TTL if cache_ttl is 0
Comment thread
kaxil marked this conversation as resolved.

 cache_ttl: int | None = cache_ttl_config if cache_ttl_config > 0 else None

 return DBDagBag(cache_size=cache_size, cache_ttl=cache_ttl)


def dag_bag_from_app(request: Request) -> DBDagBag:
Expand Down

24 changes: 24 additions & 0 deletions
airflow-core/src/airflow/config_templates/config.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1606,6 +1606,30 @@
 type: string
 example: ~
 default: "False"
 dag_cache_size:
 description: |
 Size of the LRU cache for SerializedDAG objects in the API server.
 Set to 0 to use an unbounded dict (no eviction, pre-3.2 behavior). The cache is keyed by DAG version ID,

Check failure
on line 1612
in airflow-core/src/airflow/config_templates/config.yml

View workflow job for this annotation
GitHub Actions/ CI image checks / Static checks
1612:111 [line-length] line too long (112 > 110 characters)
 so lookups by DAG ID (e.g., viewing a DAG's details) always query
 the database for the latest version, but the deserialized result is
 cached for subsequent version-specific lookups.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "64"
 dag_cache_ttl:
 description: |
 Time-to-live (seconds) for cached SerializedDAG objects in the API server.
 After this time, cached DAGs will be re-fetched from the database on next access.
 Set to 0 to disable TTL (cache entries will only be evicted by LRU policy).

 Note: After a DAG is updated, the API server may serve the previous version
 until the cached entry expires. Lower values reduce staleness but increase
 database load.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "3600"
 base_url:
 description: |
 The base url of the API server. Airflow cannot guess what domain or CNAME you are using.
Expand Down

152 changes: 115 additions & 37 deletions
airflow-core/src/airflow/models/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -18,12 +18,17 @@
from __future__ import annotations

import hashlib
from collections.abc import MutableMapping
from contextlib import nullcontext
from threading import RLock
from typing import TYPE_CHECKING, Any
from uuid import UUID

from cachetools import LRUCache, TTLCache
from sqlalchemy import String, select
from sqlalchemy.orm import Mapped, joinedload, mapped_column

from airflow._shared.observability.metrics.stats import Stats
from airflow.models.base import Base, StringID
from airflow.models.dag_version import DagVersion

Expand All@@ -39,50 +44,117 @@

class DBDagBag:
 """
 Internal class for retrieving and caching dags in the scheduler.
 Internal class for retrieving dags from the database.

 Optionally supports LRU+TTL caching when cache_size is provided.
 The scheduler uses this without caching, while the API server can
 enable caching via configuration.

 :meta private:
 """

 def __init__(self, load_op_links: bool = True) -> None:
 self._dags: dict[UUID, SerializedDagModel] = {} # dag_version_id to dag
 self.load_op_links = load_op_links
 def __init__(
 self,
 load_op_links: bool = True,
 cache_size: int | None = None,
 cache_ttl: int | None = None,
 ) -> None:
 """
 Initialize DBDagBag.

 def _read_dag(self, serialized_dag_model: SerializedDagModel) -> SerializedDAG | None:
 serialized_dag_model.load_op_links = self.load_op_links
 if dag := serialized_dag_model.dag:
 self._dags[serialized_dag_model.dag_version_id] = serialized_dag_model
 :param load_op_links: Should the extra operator link be loaded when de-serializing the DAG?
 :param cache_size: Size of LRU cache. If None or 0, uses unbounded dict (no eviction).
 :param cache_ttl: Time-to-live for cache entries in seconds. If None or 0, no TTL (LRU only).
 """
 self.load_op_links = load_op_links
 self._dags: MutableMapping[UUID | str, SerializedDAG] = {}
 self._use_cache = False

 # Initialize bounded cache if cache_size is provided and > 0
 if cache_size and cache_size > 0:
Comment thread
kaxil marked this conversation as resolved.

 if cache_ttl and cache_ttl > 0:
 self._dags = TTLCache(maxsize=cache_size, ttl=cache_ttl)
 else:
 self._dags = LRUCache(maxsize=cache_size)
 self._use_cache = True

 # Lock required for bounded caches: cachetools caches are NOT thread-safe
 # (LRU reordering and TTL cleanup mutate internal linked lists).
 # nullcontext for unbounded dict avoids lock overhead in the scheduler path.
 self._lock: RLock | nullcontext = RLock() if self._use_cache else nullcontext()

Comment thread
kaxil marked this conversation as resolved.

 def _read_dag(self, serdag: SerializedDagModel) -> SerializedDAG | None:
 """Read and optionally cache a SerializedDAG from a SerializedDagModel."""
 serdag.load_op_links = self.load_op_links
 dag = serdag.dag
 if not dag:
 return None
 with self._lock:
 self._dags[serdag.dag_version_id] = dag
 cache_size = len(self._dags)
 if self._use_cache:
 Stats.gauge("api_server.dag_bag.cache_size", cache_size, rate=0.1)
 return dag

 def get_serialized_dag_model(self, version_id: UUID, session: Session) -> SerializedDagModel | None:
 def _get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 # Check cache first
 with self._lock:
 dag = self._dags.get(version_id)

 if dag:
 if self._use_cache:
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag

 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version:
 return None
 if not (serdag := dag_version.serialized_dag):
 return None

 # Double-checked locking: another thread may have cached it while we queried DB.
 # Only emit the miss metric after confirming no other thread cached it, to avoid
 # counting a single lookup as both a miss and a hit.
 if self._use_cache:
 with self._lock:
 if dag := self._dags.get(version_id):
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag
 Stats.incr("api_server.dag_bag.cache_miss")
 return self._read_dag(serdag)
Comment thread
kaxil marked this conversation as resolved.


 def get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 """Get a dag by its version id, using cache if enabled."""
 return self._get_dag(version_id=version_id, session=session)

 def get_serialized_dag_model(self, version_id: UUID | str, session: Session) -> SerializedDagModel | None:
 """
 Return the SerializedDagModel for a given dag version id.

 This will first consult the in-memory cache keyed by the dag version id. If the
 model is not cached, the database is queried for a corresponding :class:`DagVersion`
 and its associated :class:`SerializedDagModel`.
 Always queries the database. The triggerer needs the full model
 for ``serialized_dag_model.data``, which cannot be stored in the
 LRU/TTL cache (it stores deserialized SerializedDAG objects).
 """
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serdag := dag_version.serialized_dag):
 return None
 serdag.load_op_links = self.load_op_links
 return serdag

 :param version_id: The UUID of the dag version to look up.
 :param session: SQLAlchemy session used to query the database.
 :return: The serialized DAG model if found either in the cache or the database; ``None``
 is returned when no :class:`DagVersion` exists for the given ``version_id`` or
 when that :class:`DagVersion` does not have an associated :class:`SerializedDagModel`.
 :rtype: SerializedDagModel | None
 def clear_cache(self) -> int:
 """
 Clear all cached DAGs and serialized DAG models.

 Note: If a serialized dag model is found in the database it will be stored in the
 internal cache (``self._dags``) before being returned.
 :return: Number of entries cleared from the DAG cache.
 """
Comment thread
kaxil marked this conversation as resolved.

 if not (serialized_dag_model := self._dags.get(version_id)):
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serialized_dag_model := dag_version.serialized_dag):
 return None
 self._read_dag(serialized_dag_model)
 return serialized_dag_model

 def get_dag(self, version_id: UUID, session: Session) -> SerializedDAG | None:
 if serialized_dag_model := self.get_serialized_dag_model(version_id=version_id, session=session):
 return serialized_dag_model.dag
 return None
 with self._lock:
 count = len(self._dags)
 self._dags.clear()

 if self._use_cache:
 Stats.incr("api_server.dag_bag.cache_clear")
 Stats.gauge("api_server.dag_bag.cache_size", 0)
 return count
Comment thread
kaxil marked this conversation as resolved.


 @staticmethod
 def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:
Expand All@@ -94,24 +166,30 @@ def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:

 def get_dag_for_run(self, dag_run: DagRun, session: Session) -> SerializedDAG | None:
 if version_id := self._version_from_dag_run(dag_run=dag_run, session=session):
 return self.get_dag(version_id=version_id, session=session)
 return self._get_dag(version_id=version_id, session=session)
 return None

 def iter_all_latest_version_dags(self, *, session: Session) -> Generator[SerializedDAG, None, None]:
 """Walk through all latest version dags available in the database."""
 """
 Walk through all latest version dags available in the database.

 Note: This method does NOT cache the DAGs to avoid cache thrashing when
 iterating over many DAGs. Each DAG is deserialized fresh from the database.
 """
 from airflow.models.serialized_dag import SerializedDagModel

 for serialized_dag_model in session.scalars(select(SerializedDagModel)):
 if dag := self._read_dag(serialized_dag_model):
 for sdm in session.scalars(select(SerializedDagModel)):
 sdm.load_op_links = self.load_op_links
 if dag := sdm.dag:
 yield dag

 def get_latest_version_of_dag(self, dag_id: str, *, session: Session) -> SerializedDAG | None:
 """Get the latest version of a dag by its id."""
 from airflow.models.serialized_dag import SerializedDagModel

 if not (serialized_dag_model := SerializedDagModel.get(dag_id, session=session)):
 if not (serdag := SerializedDagModel.get(dag_id, session=session)):
 return None
 return self._read_dag(serialized_dag_model)
 return self._read_dag(serdag)


def generate_md5_hash(context):
Expand Down

28 changes: 28 additions & 0 deletions
airflow-core/tests/unit/api_fastapi/common/test_dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,6 +19,7 @@
from unittest import mock

import pytest
from cachetools import LRUCache, TTLCache

from airflow.api_fastapi.app import purge_cached_app
from airflow.sdk import BaseOperator
Expand DownExpand Up@@ -82,3 +83,30 @@ def test_dagbag_used_as_singleton_in_dependency(self, session, dag_maker, test_c
 assert resp2.status_code == 200

 assert self.dagbag_call_counter["count"] == 1


class TestCreateDagBag:
 """Tests for create_dag_bag() function."""

Comment thread
kaxil marked this conversation as resolved.

 @pytest.mark.parametrize(
 ["cache_size", "cache_ttl", "expected_use_cache", "expected_dags_type"],
 [
 pytest.param(64, 3600, True, TTLCache, id="default_ttl_cache"),
 pytest.param(0, 3600, False, dict, id="size_zero_unbounded"),
 pytest.param(64, 0, True, LRUCache, id="ttl_zero_lru_only"),
 ],
 )
 @mock.patch("airflow.api_fastapi.common.dagbag.conf")
 def test_create_dag_bag_cache_modes(
 self, mock_conf, cache_size, cache_ttl, expected_use_cache, expected_dags_type
 ):
 from airflow.api_fastapi.common.dagbag import create_dag_bag

 mock_conf.getint.side_effect = lambda section, key, fallback: {
 "dag_cache_size": cache_size,
 "dag_cache_ttl": cache_ttl,
 }.get(key, fallback)

 dag_bag = create_dag_bag()
 assert dag_bag._use_cache is expected_use_cache
 assert isinstance(dag_bag._dags, expected_dags_type)
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Add configurable LRU+TTL caching for API server DAG retrieval#60804

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kaxil
merged 7 commits into

from

Apr 17, 2026
 Merged


Add configurable LRU+TTL caching for API server DAG retrieval
#60804
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
9ed6486

Add configurable LRU+TTL caching for API server DAG retrieval


kaxil Apr 11, 2026
d3dc177

Add docs and newsfragment for DAG cache feature


kaxil Apr 11, 2026
99faf78

Add bounded DAG caching section to API server memory FAQ


kaxil Apr 11, 2026
f8b3398

Fix config wording: cache_size=0 means unbounded dict, not disabled


kaxil Apr 11, 2026
03ed0f8

Fix version_added to 3.3.0 and update FAQ version


kaxil Apr 15, 2026
dcb2ab5

Remove unbounded _dag_models dict


kaxil Apr 15, 2026
d31c6ef

Apply suggestions from code review


kaxil Apr 17, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


6 changes: 6 additions & 0 deletions
airflow-core/docs/administration-and-deployment/web-stack.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -142,6 +142,8 @@ The following configuration options are available in the ``[api]`` section:
- ``server_type``: ``uvicorn`` (default) or ``gunicorn``
- ``worker_refresh_interval``: Seconds between worker refresh cycles (0 = disabled, default)
- ``worker_refresh_batch_size``: Number of workers to refresh per cycle (default: 1)
- ``dag_cache_size``: Max cached SerializedDAG versions in the API server (default: 64, 0 = unbounded)
- ``dag_cache_ttl``: TTL in seconds for cached DAGs (default: 3600, 0 = LRU only)
- ``reload_on_plugin_change``: Reload when plugin files change (default: False)

When to Use Gunicorn
Expand DownExpand Up@@ -187,6 +189,10 @@ For example, to trigger a rolling restart of the API server pods:

 kubectl rollout restart deployment airflow-api-server

The API server also supports bounded DAG caching via ``dag_cache_size`` and
``dag_cache_ttl``, which limits memory consumed by cached SerializedDAG objects.
This reduces memory growth from DAG version accumulation regardless of server type.

In many Kubernetes environments, relying solely on Kubernetes OOM kills or
crash restarts is not recommended, as memory growth may not always trigger an
OOM event. For production deployments that require automatic worker recycling
Expand Down

29 changes: 25 additions & 4 deletions
airflow-core/docs/faq.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -677,8 +677,28 @@ How to prevent API server memory growth?
The API server caches serialized Dag objects in memory. Over time, as Dag versions accumulate
(see :ref:`faq:dag-version-inflation`), this cache grows and can consume several gigabytes of memory.

The recommended solution (available since Airflow 3.2.0) is to use **gunicorn** with **rolling worker
restarts**. Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
There are two complementary approaches:

**1. Bounded DAG caching (available since Airflow 3.3.0)**

The API server supports LRU+TTL caching that bounds how many serialized Dag versions are kept
in memory. Configure this in the ``[api]`` section:

.. code-block:: ini

 [api]
 dag_cache_size = 64 ; max cached versions (0 = unbounded, pre-3.2 behavior)
 dag_cache_ttl = 3600 ; seconds before a cached entry expires (0 = LRU only)

The cache is keyed by Dag version ID. After a Dag is updated, the API server may serve the
previous version until the cached entry expires (controlled by ``dag_cache_ttl``).

See :ref:`config:api__dag_cache_size` and :ref:`config:api__dag_cache_ttl` for the full
configuration reference.

**2. Gunicorn with rolling worker restarts (available since Airflow 3.2.0)**

Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
uses ``preload`` + ``fork``, so workers share read-only memory pages via copy-on-write, reducing overall
memory usage by 40-50% compared to uvicorn's multiprocess mode.

Expand All@@ -699,8 +719,9 @@ See :ref:`config:api__server_type`, :ref:`config:api__worker_refresh_interval`,

.. note::

 Worker recycling handles memory growth from *any* source, not just the Dag cache. It is the
 recommended approach for production API server deployments.
 Worker recycling handles memory growth from *any* source, not just the Dag cache.
 For production deployments, using both bounded caching and gunicorn worker recycling
 provides the best results.


MySQL and MySQL variant Databases
Expand Down

1 change: 1 addition & 0 deletions
airflow-core/newsfragments/60804.feature.rst

View file


Open in desktop

Comment thread
kaxil marked this conversation as resolved.

Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
Add configurable LRU+TTL caching for API server DAG retrieval via ``dag_cache_size`` and ``dag_cache_ttl`` config options in the ``[api]`` section. This bounds memory growth from accumulated SerializedDAG objects in long-running API server processes.

1 change: 1 addition & 0 deletions
airflow-core/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,7 @@ version = "3.3.0"

dependencies = [
 "a2wsgi>=1.10.8",
 "cachetools>=6.0.0",
 # aiosqlite 0.22.0 has a problem with hanging pytest sessions and we excluded it
 # See https://github.com/omnilib/aiosqlite/issues/369
 # It seems that while our test issues are fixed in 0.22.1, sqlalchemy 2 itself
Expand Down

25 changes: 23 additions & 2 deletions
airflow-core/src/airflow/api_fastapi/common/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,21 +16,42 @@
# under the License.
from __future__ import annotations

import logging
from typing import TYPE_CHECKING, Annotated

from fastapi import Depends, HTTPException, Request, status
from sqlalchemy.orm import Session

from airflow.configuration import conf
from airflow.models.dagbag import DBDagBag

if TYPE_CHECKING:
 from airflow.models.dagrun import DagRun
 from airflow.serialization.definitions.dag import SerializedDAG

log = logging.getLogger(__name__)


def create_dag_bag() -> DBDagBag:
 """Create DagBag to retrieve DAGs from the database."""
 return DBDagBag()
 """Create DagBag with configurable LRU+TTL caching for API server usage."""
 cache_size = conf.getint("api", "dag_cache_size", fallback=64)
 cache_ttl_config = conf.getint("api", "dag_cache_ttl", fallback=3600)

 if cache_size < 0:
 log.warning("dag_cache_size must be >= 0, using unbounded dict")
 cache_size = 0
 if cache_ttl_config < 0:
 log.warning("dag_cache_ttl must be >= 0, disabling TTL")
 cache_ttl_config = 0

 # Use unbounded dict (no eviction) if cache_size is 0
 if cache_size <= 0:
 return DBDagBag(cache_size=0)

 # Disable TTL if cache_ttl is 0
Comment thread
kaxil marked this conversation as resolved.

 cache_ttl: int | None = cache_ttl_config if cache_ttl_config > 0 else None

 return DBDagBag(cache_size=cache_size, cache_ttl=cache_ttl)


def dag_bag_from_app(request: Request) -> DBDagBag:
Expand Down

24 changes: 24 additions & 0 deletions
airflow-core/src/airflow/config_templates/config.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1606,6 +1606,30 @@
 type: string
 example: ~
 default: "False"
 dag_cache_size:
 description: |
 Size of the LRU cache for SerializedDAG objects in the API server.
 Set to 0 to use an unbounded dict (no eviction, pre-3.2 behavior). The cache is keyed by DAG version ID,

Check failure
on line 1612
in airflow-core/src/airflow/config_templates/config.yml

View workflow job for this annotation
GitHub Actions/ CI image checks / Static checks
1612:111 [line-length] line too long (112 > 110 characters)
 so lookups by DAG ID (e.g., viewing a DAG's details) always query
 the database for the latest version, but the deserialized result is
 cached for subsequent version-specific lookups.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "64"
 dag_cache_ttl:
 description: |
 Time-to-live (seconds) for cached SerializedDAG objects in the API server.
 After this time, cached DAGs will be re-fetched from the database on next access.
 Set to 0 to disable TTL (cache entries will only be evicted by LRU policy).

 Note: After a DAG is updated, the API server may serve the previous version
 until the cached entry expires. Lower values reduce staleness but increase
 database load.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "3600"
 base_url:
 description: |
 The base url of the API server. Airflow cannot guess what domain or CNAME you are using.
Expand Down

152 changes: 115 additions & 37 deletions
airflow-core/src/airflow/models/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -18,12 +18,17 @@
from __future__ import annotations

import hashlib
from collections.abc import MutableMapping
from contextlib import nullcontext
from threading import RLock
from typing import TYPE_CHECKING, Any
from uuid import UUID

from cachetools import LRUCache, TTLCache
from sqlalchemy import String, select
from sqlalchemy.orm import Mapped, joinedload, mapped_column

from airflow._shared.observability.metrics.stats import Stats
from airflow.models.base import Base, StringID
from airflow.models.dag_version import DagVersion

Expand All@@ -39,50 +44,117 @@

class DBDagBag:
 """
 Internal class for retrieving and caching dags in the scheduler.
 Internal class for retrieving dags from the database.

 Optionally supports LRU+TTL caching when cache_size is provided.
 The scheduler uses this without caching, while the API server can
 enable caching via configuration.

 :meta private:
 """

 def __init__(self, load_op_links: bool = True) -> None:
 self._dags: dict[UUID, SerializedDagModel] = {} # dag_version_id to dag
 self.load_op_links = load_op_links
 def __init__(
 self,
 load_op_links: bool = True,
 cache_size: int | None = None,
 cache_ttl: int | None = None,
 ) -> None:
 """
 Initialize DBDagBag.

 def _read_dag(self, serialized_dag_model: SerializedDagModel) -> SerializedDAG | None:
 serialized_dag_model.load_op_links = self.load_op_links
 if dag := serialized_dag_model.dag:
 self._dags[serialized_dag_model.dag_version_id] = serialized_dag_model
 :param load_op_links: Should the extra operator link be loaded when de-serializing the DAG?
 :param cache_size: Size of LRU cache. If None or 0, uses unbounded dict (no eviction).
 :param cache_ttl: Time-to-live for cache entries in seconds. If None or 0, no TTL (LRU only).
 """
 self.load_op_links = load_op_links
 self._dags: MutableMapping[UUID | str, SerializedDAG] = {}
 self._use_cache = False

 # Initialize bounded cache if cache_size is provided and > 0
 if cache_size and cache_size > 0:
Comment thread
kaxil marked this conversation as resolved.

 if cache_ttl and cache_ttl > 0:
 self._dags = TTLCache(maxsize=cache_size, ttl=cache_ttl)
 else:
 self._dags = LRUCache(maxsize=cache_size)
 self._use_cache = True

 # Lock required for bounded caches: cachetools caches are NOT thread-safe
 # (LRU reordering and TTL cleanup mutate internal linked lists).
 # nullcontext for unbounded dict avoids lock overhead in the scheduler path.
 self._lock: RLock | nullcontext = RLock() if self._use_cache else nullcontext()

Comment thread
kaxil marked this conversation as resolved.

 def _read_dag(self, serdag: SerializedDagModel) -> SerializedDAG | None:
 """Read and optionally cache a SerializedDAG from a SerializedDagModel."""
 serdag.load_op_links = self.load_op_links
 dag = serdag.dag
 if not dag:
 return None
 with self._lock:
 self._dags[serdag.dag_version_id] = dag
 cache_size = len(self._dags)
 if self._use_cache:
 Stats.gauge("api_server.dag_bag.cache_size", cache_size, rate=0.1)
 return dag

 def get_serialized_dag_model(self, version_id: UUID, session: Session) -> SerializedDagModel | None:
 def _get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 # Check cache first
 with self._lock:
 dag = self._dags.get(version_id)

 if dag:
 if self._use_cache:
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag

 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version:
 return None
 if not (serdag := dag_version.serialized_dag):
 return None

 # Double-checked locking: another thread may have cached it while we queried DB.
 # Only emit the miss metric after confirming no other thread cached it, to avoid
 # counting a single lookup as both a miss and a hit.
 if self._use_cache:
 with self._lock:
 if dag := self._dags.get(version_id):
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag
 Stats.incr("api_server.dag_bag.cache_miss")
 return self._read_dag(serdag)
Comment thread
kaxil marked this conversation as resolved.


 def get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 """Get a dag by its version id, using cache if enabled."""
 return self._get_dag(version_id=version_id, session=session)

 def get_serialized_dag_model(self, version_id: UUID | str, session: Session) -> SerializedDagModel | None:
 """
 Return the SerializedDagModel for a given dag version id.

 This will first consult the in-memory cache keyed by the dag version id. If the
 model is not cached, the database is queried for a corresponding :class:`DagVersion`
 and its associated :class:`SerializedDagModel`.
 Always queries the database. The triggerer needs the full model
 for ``serialized_dag_model.data``, which cannot be stored in the
 LRU/TTL cache (it stores deserialized SerializedDAG objects).
 """
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serdag := dag_version.serialized_dag):
 return None
 serdag.load_op_links = self.load_op_links
 return serdag

 :param version_id: The UUID of the dag version to look up.
 :param session: SQLAlchemy session used to query the database.
 :return: The serialized DAG model if found either in the cache or the database; ``None``
 is returned when no :class:`DagVersion` exists for the given ``version_id`` or
 when that :class:`DagVersion` does not have an associated :class:`SerializedDagModel`.
 :rtype: SerializedDagModel | None
 def clear_cache(self) -> int:
 """
 Clear all cached DAGs and serialized DAG models.

 Note: If a serialized dag model is found in the database it will be stored in the
 internal cache (``self._dags``) before being returned.
 :return: Number of entries cleared from the DAG cache.
 """
Comment thread
kaxil marked this conversation as resolved.

 if not (serialized_dag_model := self._dags.get(version_id)):
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serialized_dag_model := dag_version.serialized_dag):
 return None
 self._read_dag(serialized_dag_model)
 return serialized_dag_model

 def get_dag(self, version_id: UUID, session: Session) -> SerializedDAG | None:
 if serialized_dag_model := self.get_serialized_dag_model(version_id=version_id, session=session):
 return serialized_dag_model.dag
 return None
 with self._lock:
 count = len(self._dags)
 self._dags.clear()

 if self._use_cache:
 Stats.incr("api_server.dag_bag.cache_clear")
 Stats.gauge("api_server.dag_bag.cache_size", 0)
 return count
Comment thread
kaxil marked this conversation as resolved.


 @staticmethod
 def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:
Expand All@@ -94,24 +166,30 @@ def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:

 def get_dag_for_run(self, dag_run: DagRun, session: Session) -> SerializedDAG | None:
 if version_id := self._version_from_dag_run(dag_run=dag_run, session=session):
 return self.get_dag(version_id=version_id, session=session)
 return self._get_dag(version_id=version_id, session=session)
 return None

 def iter_all_latest_version_dags(self, *, session: Session) -> Generator[SerializedDAG, None, None]:
 """Walk through all latest version dags available in the database."""
 """
 Walk through all latest version dags available in the database.

 Note: This method does NOT cache the DAGs to avoid cache thrashing when
 iterating over many DAGs. Each DAG is deserialized fresh from the database.
 """
 from airflow.models.serialized_dag import SerializedDagModel

 for serialized_dag_model in session.scalars(select(SerializedDagModel)):
 if dag := self._read_dag(serialized_dag_model):
 for sdm in session.scalars(select(SerializedDagModel)):
 sdm.load_op_links = self.load_op_links
 if dag := sdm.dag:
 yield dag

 def get_latest_version_of_dag(self, dag_id: str, *, session: Session) -> SerializedDAG | None:
 """Get the latest version of a dag by its id."""
 from airflow.models.serialized_dag import SerializedDagModel

 if not (serialized_dag_model := SerializedDagModel.get(dag_id, session=session)):
 if not (serdag := SerializedDagModel.get(dag_id, session=session)):
 return None
 return self._read_dag(serialized_dag_model)
 return self._read_dag(serdag)


def generate_md5_hash(context):
Expand Down

28 changes: 28 additions & 0 deletions
airflow-core/tests/unit/api_fastapi/common/test_dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,6 +19,7 @@
from unittest import mock

import pytest
from cachetools import LRUCache, TTLCache

from airflow.api_fastapi.app import purge_cached_app
from airflow.sdk import BaseOperator
Expand DownExpand Up@@ -82,3 +83,30 @@ def test_dagbag_used_as_singleton_in_dependency(self, session, dag_maker, test_c
 assert resp2.status_code == 200

 assert self.dagbag_call_counter["count"] == 1


class TestCreateDagBag:
 """Tests for create_dag_bag() function."""

Comment thread
kaxil marked this conversation as resolved.

 @pytest.mark.parametrize(
 ["cache_size", "cache_ttl", "expected_use_cache", "expected_dags_type"],
 [
 pytest.param(64, 3600, True, TTLCache, id="default_ttl_cache"),
 pytest.param(0, 3600, False, dict, id="size_zero_unbounded"),
 pytest.param(64, 0, True, LRUCache, id="ttl_zero_lru_only"),
 ],
 )
 @mock.patch("airflow.api_fastapi.common.dagbag.conf")
 def test_create_dag_bag_cache_modes(
 self, mock_conf, cache_size, cache_ttl, expected_use_cache, expected_dags_type
 ):
 from airflow.api_fastapi.common.dagbag import create_dag_bag

 mock_conf.getint.side_effect = lambda section, key, fallback: {
 "dag_cache_size": cache_size,
 "dag_cache_ttl": cache_ttl,
 }.get(key, fallback)

 dag_bag = create_dag_bag()
 assert dag_bag._use_cache is expected_use_cache
 assert isinstance(dag_bag._dags, expected_dags_type)
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Add configurable LRU+TTL caching for API server DAG retrieval#60804

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kaxil
merged 7 commits into

from

Apr 17, 2026
 Merged


Add configurable LRU+TTL caching for API server DAG retrieval
#60804
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
9ed6486

Add configurable LRU+TTL caching for API server DAG retrieval


kaxil Apr 11, 2026
d3dc177

Add docs and newsfragment for DAG cache feature


kaxil Apr 11, 2026
99faf78

Add bounded DAG caching section to API server memory FAQ


kaxil Apr 11, 2026
f8b3398

Fix config wording: cache_size=0 means unbounded dict, not disabled


kaxil Apr 11, 2026
03ed0f8

Fix version_added to 3.3.0 and update FAQ version


kaxil Apr 15, 2026
dcb2ab5

Remove unbounded _dag_models dict


kaxil Apr 15, 2026
d31c6ef

Apply suggestions from code review


kaxil Apr 17, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


6 changes: 6 additions & 0 deletions
airflow-core/docs/administration-and-deployment/web-stack.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -142,6 +142,8 @@ The following configuration options are available in the ``[api]`` section:
- ``server_type``: ``uvicorn`` (default) or ``gunicorn``
- ``worker_refresh_interval``: Seconds between worker refresh cycles (0 = disabled, default)
- ``worker_refresh_batch_size``: Number of workers to refresh per cycle (default: 1)
- ``dag_cache_size``: Max cached SerializedDAG versions in the API server (default: 64, 0 = unbounded)
- ``dag_cache_ttl``: TTL in seconds for cached DAGs (default: 3600, 0 = LRU only)
- ``reload_on_plugin_change``: Reload when plugin files change (default: False)

When to Use Gunicorn
Expand DownExpand Up@@ -187,6 +189,10 @@ For example, to trigger a rolling restart of the API server pods:

 kubectl rollout restart deployment airflow-api-server

The API server also supports bounded DAG caching via ``dag_cache_size`` and
``dag_cache_ttl``, which limits memory consumed by cached SerializedDAG objects.
This reduces memory growth from DAG version accumulation regardless of server type.

In many Kubernetes environments, relying solely on Kubernetes OOM kills or
crash restarts is not recommended, as memory growth may not always trigger an
OOM event. For production deployments that require automatic worker recycling
Expand Down

29 changes: 25 additions & 4 deletions
airflow-core/docs/faq.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -677,8 +677,28 @@ How to prevent API server memory growth?
The API server caches serialized Dag objects in memory. Over time, as Dag versions accumulate
(see :ref:`faq:dag-version-inflation`), this cache grows and can consume several gigabytes of memory.

The recommended solution (available since Airflow 3.2.0) is to use **gunicorn** with **rolling worker
restarts**. Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
There are two complementary approaches:

**1. Bounded DAG caching (available since Airflow 3.3.0)**

The API server supports LRU+TTL caching that bounds how many serialized Dag versions are kept
in memory. Configure this in the ``[api]`` section:

.. code-block:: ini

 [api]
 dag_cache_size = 64 ; max cached versions (0 = unbounded, pre-3.2 behavior)
 dag_cache_ttl = 3600 ; seconds before a cached entry expires (0 = LRU only)

The cache is keyed by Dag version ID. After a Dag is updated, the API server may serve the
previous version until the cached entry expires (controlled by ``dag_cache_ttl``).

See :ref:`config:api__dag_cache_size` and :ref:`config:api__dag_cache_ttl` for the full
configuration reference.

**2. Gunicorn with rolling worker restarts (available since Airflow 3.2.0)**

Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
uses ``preload`` + ``fork``, so workers share read-only memory pages via copy-on-write, reducing overall
memory usage by 40-50% compared to uvicorn's multiprocess mode.

Expand All@@ -699,8 +719,9 @@ See :ref:`config:api__server_type`, :ref:`config:api__worker_refresh_interval`,

.. note::

 Worker recycling handles memory growth from *any* source, not just the Dag cache. It is the
 recommended approach for production API server deployments.
 Worker recycling handles memory growth from *any* source, not just the Dag cache.
 For production deployments, using both bounded caching and gunicorn worker recycling
 provides the best results.


MySQL and MySQL variant Databases
Expand Down

1 change: 1 addition & 0 deletions
airflow-core/newsfragments/60804.feature.rst

View file


Open in desktop

Comment thread
kaxil marked this conversation as resolved.

Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
Add configurable LRU+TTL caching for API server DAG retrieval via ``dag_cache_size`` and ``dag_cache_ttl`` config options in the ``[api]`` section. This bounds memory growth from accumulated SerializedDAG objects in long-running API server processes.

1 change: 1 addition & 0 deletions
airflow-core/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,7 @@ version = "3.3.0"

dependencies = [
 "a2wsgi>=1.10.8",
 "cachetools>=6.0.0",
 # aiosqlite 0.22.0 has a problem with hanging pytest sessions and we excluded it
 # See https://github.com/omnilib/aiosqlite/issues/369
 # It seems that while our test issues are fixed in 0.22.1, sqlalchemy 2 itself
Expand Down

25 changes: 23 additions & 2 deletions
airflow-core/src/airflow/api_fastapi/common/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,21 +16,42 @@
# under the License.
from __future__ import annotations

import logging
from typing import TYPE_CHECKING, Annotated

from fastapi import Depends, HTTPException, Request, status
from sqlalchemy.orm import Session

from airflow.configuration import conf
from airflow.models.dagbag import DBDagBag

if TYPE_CHECKING:
 from airflow.models.dagrun import DagRun
 from airflow.serialization.definitions.dag import SerializedDAG

log = logging.getLogger(__name__)


def create_dag_bag() -> DBDagBag:
 """Create DagBag to retrieve DAGs from the database."""
 return DBDagBag()
 """Create DagBag with configurable LRU+TTL caching for API server usage."""
 cache_size = conf.getint("api", "dag_cache_size", fallback=64)
 cache_ttl_config = conf.getint("api", "dag_cache_ttl", fallback=3600)

 if cache_size < 0:
 log.warning("dag_cache_size must be >= 0, using unbounded dict")
 cache_size = 0
 if cache_ttl_config < 0:
 log.warning("dag_cache_ttl must be >= 0, disabling TTL")
 cache_ttl_config = 0

 # Use unbounded dict (no eviction) if cache_size is 0
 if cache_size <= 0:
 return DBDagBag(cache_size=0)

 # Disable TTL if cache_ttl is 0
Comment thread
kaxil marked this conversation as resolved.

 cache_ttl: int | None = cache_ttl_config if cache_ttl_config > 0 else None

 return DBDagBag(cache_size=cache_size, cache_ttl=cache_ttl)


def dag_bag_from_app(request: Request) -> DBDagBag:
Expand Down

24 changes: 24 additions & 0 deletions
airflow-core/src/airflow/config_templates/config.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1606,6 +1606,30 @@
 type: string
 example: ~
 default: "False"
 dag_cache_size:
 description: |
 Size of the LRU cache for SerializedDAG objects in the API server.
 Set to 0 to use an unbounded dict (no eviction, pre-3.2 behavior). The cache is keyed by DAG version ID,

Check failure
on line 1612
in airflow-core/src/airflow/config_templates/config.yml

View workflow job for this annotation
GitHub Actions/ CI image checks / Static checks
1612:111 [line-length] line too long (112 > 110 characters)
 so lookups by DAG ID (e.g., viewing a DAG's details) always query
 the database for the latest version, but the deserialized result is
 cached for subsequent version-specific lookups.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "64"
 dag_cache_ttl:
 description: |
 Time-to-live (seconds) for cached SerializedDAG objects in the API server.
 After this time, cached DAGs will be re-fetched from the database on next access.
 Set to 0 to disable TTL (cache entries will only be evicted by LRU policy).

 Note: After a DAG is updated, the API server may serve the previous version
 until the cached entry expires. Lower values reduce staleness but increase
 database load.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "3600"
 base_url:
 description: |
 The base url of the API server. Airflow cannot guess what domain or CNAME you are using.
Expand Down

152 changes: 115 additions & 37 deletions
airflow-core/src/airflow/models/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -18,12 +18,17 @@
from __future__ import annotations

import hashlib
from collections.abc import MutableMapping
from contextlib import nullcontext
from threading import RLock
from typing import TYPE_CHECKING, Any
from uuid import UUID

from cachetools import LRUCache, TTLCache
from sqlalchemy import String, select
from sqlalchemy.orm import Mapped, joinedload, mapped_column

from airflow._shared.observability.metrics.stats import Stats
from airflow.models.base import Base, StringID
from airflow.models.dag_version import DagVersion

Expand All@@ -39,50 +44,117 @@

class DBDagBag:
 """
 Internal class for retrieving and caching dags in the scheduler.
 Internal class for retrieving dags from the database.

 Optionally supports LRU+TTL caching when cache_size is provided.
 The scheduler uses this without caching, while the API server can
 enable caching via configuration.

 :meta private:
 """

 def __init__(self, load_op_links: bool = True) -> None:
 self._dags: dict[UUID, SerializedDagModel] = {} # dag_version_id to dag
 self.load_op_links = load_op_links
 def __init__(
 self,
 load_op_links: bool = True,
 cache_size: int | None = None,
 cache_ttl: int | None = None,
 ) -> None:
 """
 Initialize DBDagBag.

 def _read_dag(self, serialized_dag_model: SerializedDagModel) -> SerializedDAG | None:
 serialized_dag_model.load_op_links = self.load_op_links
 if dag := serialized_dag_model.dag:
 self._dags[serialized_dag_model.dag_version_id] = serialized_dag_model
 :param load_op_links: Should the extra operator link be loaded when de-serializing the DAG?
 :param cache_size: Size of LRU cache. If None or 0, uses unbounded dict (no eviction).
 :param cache_ttl: Time-to-live for cache entries in seconds. If None or 0, no TTL (LRU only).
 """
 self.load_op_links = load_op_links
 self._dags: MutableMapping[UUID | str, SerializedDAG] = {}
 self._use_cache = False

 # Initialize bounded cache if cache_size is provided and > 0
 if cache_size and cache_size > 0:
Comment thread
kaxil marked this conversation as resolved.

 if cache_ttl and cache_ttl > 0:
 self._dags = TTLCache(maxsize=cache_size, ttl=cache_ttl)
 else:
 self._dags = LRUCache(maxsize=cache_size)
 self._use_cache = True

 # Lock required for bounded caches: cachetools caches are NOT thread-safe
 # (LRU reordering and TTL cleanup mutate internal linked lists).
 # nullcontext for unbounded dict avoids lock overhead in the scheduler path.
 self._lock: RLock | nullcontext = RLock() if self._use_cache else nullcontext()

Comment thread
kaxil marked this conversation as resolved.

 def _read_dag(self, serdag: SerializedDagModel) -> SerializedDAG | None:
 """Read and optionally cache a SerializedDAG from a SerializedDagModel."""
 serdag.load_op_links = self.load_op_links
 dag = serdag.dag
 if not dag:
 return None
 with self._lock:
 self._dags[serdag.dag_version_id] = dag
 cache_size = len(self._dags)
 if self._use_cache:
 Stats.gauge("api_server.dag_bag.cache_size", cache_size, rate=0.1)
 return dag

 def get_serialized_dag_model(self, version_id: UUID, session: Session) -> SerializedDagModel | None:
 def _get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 # Check cache first
 with self._lock:
 dag = self._dags.get(version_id)

 if dag:
 if self._use_cache:
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag

 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version:
 return None
 if not (serdag := dag_version.serialized_dag):
 return None

 # Double-checked locking: another thread may have cached it while we queried DB.
 # Only emit the miss metric after confirming no other thread cached it, to avoid
 # counting a single lookup as both a miss and a hit.
 if self._use_cache:
 with self._lock:
 if dag := self._dags.get(version_id):
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag
 Stats.incr("api_server.dag_bag.cache_miss")
 return self._read_dag(serdag)
Comment thread
kaxil marked this conversation as resolved.


 def get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 """Get a dag by its version id, using cache if enabled."""
 return self._get_dag(version_id=version_id, session=session)

 def get_serialized_dag_model(self, version_id: UUID | str, session: Session) -> SerializedDagModel | None:
 """
 Return the SerializedDagModel for a given dag version id.

 This will first consult the in-memory cache keyed by the dag version id. If the
 model is not cached, the database is queried for a corresponding :class:`DagVersion`
 and its associated :class:`SerializedDagModel`.
 Always queries the database. The triggerer needs the full model
 for ``serialized_dag_model.data``, which cannot be stored in the
 LRU/TTL cache (it stores deserialized SerializedDAG objects).
 """
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serdag := dag_version.serialized_dag):
 return None
 serdag.load_op_links = self.load_op_links
 return serdag

 :param version_id: The UUID of the dag version to look up.
 :param session: SQLAlchemy session used to query the database.
 :return: The serialized DAG model if found either in the cache or the database; ``None``
 is returned when no :class:`DagVersion` exists for the given ``version_id`` or
 when that :class:`DagVersion` does not have an associated :class:`SerializedDagModel`.
 :rtype: SerializedDagModel | None
 def clear_cache(self) -> int:
 """
 Clear all cached DAGs and serialized DAG models.

 Note: If a serialized dag model is found in the database it will be stored in the
 internal cache (``self._dags``) before being returned.
 :return: Number of entries cleared from the DAG cache.
 """
Comment thread
kaxil marked this conversation as resolved.

 if not (serialized_dag_model := self._dags.get(version_id)):
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serialized_dag_model := dag_version.serialized_dag):
 return None
 self._read_dag(serialized_dag_model)
 return serialized_dag_model

 def get_dag(self, version_id: UUID, session: Session) -> SerializedDAG | None:
 if serialized_dag_model := self.get_serialized_dag_model(version_id=version_id, session=session):
 return serialized_dag_model.dag
 return None
 with self._lock:
 count = len(self._dags)
 self._dags.clear()

 if self._use_cache:
 Stats.incr("api_server.dag_bag.cache_clear")
 Stats.gauge("api_server.dag_bag.cache_size", 0)
 return count
Comment thread
kaxil marked this conversation as resolved.


 @staticmethod
 def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:
Expand All@@ -94,24 +166,30 @@ def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:

 def get_dag_for_run(self, dag_run: DagRun, session: Session) -> SerializedDAG | None:
 if version_id := self._version_from_dag_run(dag_run=dag_run, session=session):
 return self.get_dag(version_id=version_id, session=session)
 return self._get_dag(version_id=version_id, session=session)
 return None

 def iter_all_latest_version_dags(self, *, session: Session) -> Generator[SerializedDAG, None, None]:
 """Walk through all latest version dags available in the database."""
 """
 Walk through all latest version dags available in the database.

 Note: This method does NOT cache the DAGs to avoid cache thrashing when
 iterating over many DAGs. Each DAG is deserialized fresh from the database.
 """
 from airflow.models.serialized_dag import SerializedDagModel

 for serialized_dag_model in session.scalars(select(SerializedDagModel)):
 if dag := self._read_dag(serialized_dag_model):
 for sdm in session.scalars(select(SerializedDagModel)):
 sdm.load_op_links = self.load_op_links
 if dag := sdm.dag:
 yield dag

 def get_latest_version_of_dag(self, dag_id: str, *, session: Session) -> SerializedDAG | None:
 """Get the latest version of a dag by its id."""
 from airflow.models.serialized_dag import SerializedDagModel

 if not (serialized_dag_model := SerializedDagModel.get(dag_id, session=session)):
 if not (serdag := SerializedDagModel.get(dag_id, session=session)):
 return None
 return self._read_dag(serialized_dag_model)
 return self._read_dag(serdag)


def generate_md5_hash(context):
Expand Down

28 changes: 28 additions & 0 deletions
airflow-core/tests/unit/api_fastapi/common/test_dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,6 +19,7 @@
from unittest import mock

import pytest
from cachetools import LRUCache, TTLCache

from airflow.api_fastapi.app import purge_cached_app
from airflow.sdk import BaseOperator
Expand DownExpand Up@@ -82,3 +83,30 @@ def test_dagbag_used_as_singleton_in_dependency(self, session, dag_maker, test_c
 assert resp2.status_code == 200

 assert self.dagbag_call_counter["count"] == 1


class TestCreateDagBag:
 """Tests for create_dag_bag() function."""

Comment thread
kaxil marked this conversation as resolved.

 @pytest.mark.parametrize(
 ["cache_size", "cache_ttl", "expected_use_cache", "expected_dags_type"],
 [
 pytest.param(64, 3600, True, TTLCache, id="default_ttl_cache"),
 pytest.param(0, 3600, False, dict, id="size_zero_unbounded"),
 pytest.param(64, 0, True, LRUCache, id="ttl_zero_lru_only"),
 ],
 )
 @mock.patch("airflow.api_fastapi.common.dagbag.conf")
 def test_create_dag_bag_cache_modes(
 self, mock_conf, cache_size, cache_ttl, expected_use_cache, expected_dags_type
 ):
 from airflow.api_fastapi.common.dagbag import create_dag_bag

 mock_conf.getint.side_effect = lambda section, key, fallback: {
 "dag_cache_size": cache_size,
 "dag_cache_ttl": cache_ttl,
 }.get(key, fallback)

 dag_bag = create_dag_bag()
 assert dag_bag._use_cache is expected_use_cache
 assert isinstance(dag_bag._dags, expected_dags_type)
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Add configurable LRU+TTL caching for API server DAG retrieval#60804

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kaxil
merged 7 commits into

from

Apr 17, 2026
 Merged


Add configurable LRU+TTL caching for API server DAG retrieval
#60804
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
9ed6486

Add configurable LRU+TTL caching for API server DAG retrieval


kaxil Apr 11, 2026
d3dc177

Add docs and newsfragment for DAG cache feature


kaxil Apr 11, 2026
99faf78

Add bounded DAG caching section to API server memory FAQ


kaxil Apr 11, 2026
f8b3398

Fix config wording: cache_size=0 means unbounded dict, not disabled


kaxil Apr 11, 2026
03ed0f8

Fix version_added to 3.3.0 and update FAQ version


kaxil Apr 15, 2026
dcb2ab5

Remove unbounded _dag_models dict


kaxil Apr 15, 2026
d31c6ef

Apply suggestions from code review


kaxil Apr 17, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


6 changes: 6 additions & 0 deletions
airflow-core/docs/administration-and-deployment/web-stack.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -142,6 +142,8 @@ The following configuration options are available in the ``[api]`` section:
- ``server_type``: ``uvicorn`` (default) or ``gunicorn``
- ``worker_refresh_interval``: Seconds between worker refresh cycles (0 = disabled, default)
- ``worker_refresh_batch_size``: Number of workers to refresh per cycle (default: 1)
- ``dag_cache_size``: Max cached SerializedDAG versions in the API server (default: 64, 0 = unbounded)
- ``dag_cache_ttl``: TTL in seconds for cached DAGs (default: 3600, 0 = LRU only)
- ``reload_on_plugin_change``: Reload when plugin files change (default: False)

When to Use Gunicorn
Expand DownExpand Up@@ -187,6 +189,10 @@ For example, to trigger a rolling restart of the API server pods:

 kubectl rollout restart deployment airflow-api-server

The API server also supports bounded DAG caching via ``dag_cache_size`` and
``dag_cache_ttl``, which limits memory consumed by cached SerializedDAG objects.
This reduces memory growth from DAG version accumulation regardless of server type.

In many Kubernetes environments, relying solely on Kubernetes OOM kills or
crash restarts is not recommended, as memory growth may not always trigger an
OOM event. For production deployments that require automatic worker recycling
Expand Down

29 changes: 25 additions & 4 deletions
airflow-core/docs/faq.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -677,8 +677,28 @@ How to prevent API server memory growth?
The API server caches serialized Dag objects in memory. Over time, as Dag versions accumulate
(see :ref:`faq:dag-version-inflation`), this cache grows and can consume several gigabytes of memory.

The recommended solution (available since Airflow 3.2.0) is to use **gunicorn** with **rolling worker
restarts**. Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
There are two complementary approaches:

**1. Bounded DAG caching (available since Airflow 3.3.0)**

The API server supports LRU+TTL caching that bounds how many serialized Dag versions are kept
in memory. Configure this in the ``[api]`` section:

.. code-block:: ini

 [api]
 dag_cache_size = 64 ; max cached versions (0 = unbounded, pre-3.2 behavior)
 dag_cache_ttl = 3600 ; seconds before a cached entry expires (0 = LRU only)

The cache is keyed by Dag version ID. After a Dag is updated, the API server may serve the
previous version until the cached entry expires (controlled by ``dag_cache_ttl``).

See :ref:`config:api__dag_cache_size` and :ref:`config:api__dag_cache_ttl` for the full
configuration reference.

**2. Gunicorn with rolling worker restarts (available since Airflow 3.2.0)**

Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
uses ``preload`` + ``fork``, so workers share read-only memory pages via copy-on-write, reducing overall
memory usage by 40-50% compared to uvicorn's multiprocess mode.

Expand All@@ -699,8 +719,9 @@ See :ref:`config:api__server_type`, :ref:`config:api__worker_refresh_interval`,

.. note::

 Worker recycling handles memory growth from *any* source, not just the Dag cache. It is the
 recommended approach for production API server deployments.
 Worker recycling handles memory growth from *any* source, not just the Dag cache.
 For production deployments, using both bounded caching and gunicorn worker recycling
 provides the best results.


MySQL and MySQL variant Databases
Expand Down

1 change: 1 addition & 0 deletions
airflow-core/newsfragments/60804.feature.rst

View file


Open in desktop

Comment thread
kaxil marked this conversation as resolved.

Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
Add configurable LRU+TTL caching for API server DAG retrieval via ``dag_cache_size`` and ``dag_cache_ttl`` config options in the ``[api]`` section. This bounds memory growth from accumulated SerializedDAG objects in long-running API server processes.

1 change: 1 addition & 0 deletions
airflow-core/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,7 @@ version = "3.3.0"

dependencies = [
 "a2wsgi>=1.10.8",
 "cachetools>=6.0.0",
 # aiosqlite 0.22.0 has a problem with hanging pytest sessions and we excluded it
 # See https://github.com/omnilib/aiosqlite/issues/369
 # It seems that while our test issues are fixed in 0.22.1, sqlalchemy 2 itself
Expand Down

25 changes: 23 additions & 2 deletions
airflow-core/src/airflow/api_fastapi/common/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,21 +16,42 @@
# under the License.
from __future__ import annotations

import logging
from typing import TYPE_CHECKING, Annotated

from fastapi import Depends, HTTPException, Request, status
from sqlalchemy.orm import Session

from airflow.configuration import conf
from airflow.models.dagbag import DBDagBag

if TYPE_CHECKING:
 from airflow.models.dagrun import DagRun
 from airflow.serialization.definitions.dag import SerializedDAG

log = logging.getLogger(__name__)


def create_dag_bag() -> DBDagBag:
 """Create DagBag to retrieve DAGs from the database."""
 return DBDagBag()
 """Create DagBag with configurable LRU+TTL caching for API server usage."""
 cache_size = conf.getint("api", "dag_cache_size", fallback=64)
 cache_ttl_config = conf.getint("api", "dag_cache_ttl", fallback=3600)

 if cache_size < 0:
 log.warning("dag_cache_size must be >= 0, using unbounded dict")
 cache_size = 0
 if cache_ttl_config < 0:
 log.warning("dag_cache_ttl must be >= 0, disabling TTL")
 cache_ttl_config = 0

 # Use unbounded dict (no eviction) if cache_size is 0
 if cache_size <= 0:
 return DBDagBag(cache_size=0)

 # Disable TTL if cache_ttl is 0
Comment thread
kaxil marked this conversation as resolved.

 cache_ttl: int | None = cache_ttl_config if cache_ttl_config > 0 else None

 return DBDagBag(cache_size=cache_size, cache_ttl=cache_ttl)


def dag_bag_from_app(request: Request) -> DBDagBag:
Expand Down

24 changes: 24 additions & 0 deletions
airflow-core/src/airflow/config_templates/config.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1606,6 +1606,30 @@
 type: string
 example: ~
 default: "False"
 dag_cache_size:
 description: |
 Size of the LRU cache for SerializedDAG objects in the API server.
 Set to 0 to use an unbounded dict (no eviction, pre-3.2 behavior). The cache is keyed by DAG version ID,

Check failure
on line 1612
in airflow-core/src/airflow/config_templates/config.yml

View workflow job for this annotation
GitHub Actions/ CI image checks / Static checks
1612:111 [line-length] line too long (112 > 110 characters)
 so lookups by DAG ID (e.g., viewing a DAG's details) always query
 the database for the latest version, but the deserialized result is
 cached for subsequent version-specific lookups.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "64"
 dag_cache_ttl:
 description: |
 Time-to-live (seconds) for cached SerializedDAG objects in the API server.
 After this time, cached DAGs will be re-fetched from the database on next access.
 Set to 0 to disable TTL (cache entries will only be evicted by LRU policy).

 Note: After a DAG is updated, the API server may serve the previous version
 until the cached entry expires. Lower values reduce staleness but increase
 database load.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "3600"
 base_url:
 description: |
 The base url of the API server. Airflow cannot guess what domain or CNAME you are using.
Expand Down

152 changes: 115 additions & 37 deletions
airflow-core/src/airflow/models/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -18,12 +18,17 @@
from __future__ import annotations

import hashlib
from collections.abc import MutableMapping
from contextlib import nullcontext
from threading import RLock
from typing import TYPE_CHECKING, Any
from uuid import UUID

from cachetools import LRUCache, TTLCache
from sqlalchemy import String, select
from sqlalchemy.orm import Mapped, joinedload, mapped_column

from airflow._shared.observability.metrics.stats import Stats
from airflow.models.base import Base, StringID
from airflow.models.dag_version import DagVersion

Expand All@@ -39,50 +44,117 @@

class DBDagBag:
 """
 Internal class for retrieving and caching dags in the scheduler.
 Internal class for retrieving dags from the database.

 Optionally supports LRU+TTL caching when cache_size is provided.
 The scheduler uses this without caching, while the API server can
 enable caching via configuration.

 :meta private:
 """

 def __init__(self, load_op_links: bool = True) -> None:
 self._dags: dict[UUID, SerializedDagModel] = {} # dag_version_id to dag
 self.load_op_links = load_op_links
 def __init__(
 self,
 load_op_links: bool = True,
 cache_size: int | None = None,
 cache_ttl: int | None = None,
 ) -> None:
 """
 Initialize DBDagBag.

 def _read_dag(self, serialized_dag_model: SerializedDagModel) -> SerializedDAG | None:
 serialized_dag_model.load_op_links = self.load_op_links
 if dag := serialized_dag_model.dag:
 self._dags[serialized_dag_model.dag_version_id] = serialized_dag_model
 :param load_op_links: Should the extra operator link be loaded when de-serializing the DAG?
 :param cache_size: Size of LRU cache. If None or 0, uses unbounded dict (no eviction).
 :param cache_ttl: Time-to-live for cache entries in seconds. If None or 0, no TTL (LRU only).
 """
 self.load_op_links = load_op_links
 self._dags: MutableMapping[UUID | str, SerializedDAG] = {}
 self._use_cache = False

 # Initialize bounded cache if cache_size is provided and > 0
 if cache_size and cache_size > 0:
Comment thread
kaxil marked this conversation as resolved.

 if cache_ttl and cache_ttl > 0:
 self._dags = TTLCache(maxsize=cache_size, ttl=cache_ttl)
 else:
 self._dags = LRUCache(maxsize=cache_size)
 self._use_cache = True

 # Lock required for bounded caches: cachetools caches are NOT thread-safe
 # (LRU reordering and TTL cleanup mutate internal linked lists).
 # nullcontext for unbounded dict avoids lock overhead in the scheduler path.
 self._lock: RLock | nullcontext = RLock() if self._use_cache else nullcontext()

Comment thread
kaxil marked this conversation as resolved.

 def _read_dag(self, serdag: SerializedDagModel) -> SerializedDAG | None:
 """Read and optionally cache a SerializedDAG from a SerializedDagModel."""
 serdag.load_op_links = self.load_op_links
 dag = serdag.dag
 if not dag:
 return None
 with self._lock:
 self._dags[serdag.dag_version_id] = dag
 cache_size = len(self._dags)
 if self._use_cache:
 Stats.gauge("api_server.dag_bag.cache_size", cache_size, rate=0.1)
 return dag

 def get_serialized_dag_model(self, version_id: UUID, session: Session) -> SerializedDagModel | None:
 def _get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 # Check cache first
 with self._lock:
 dag = self._dags.get(version_id)

 if dag:
 if self._use_cache:
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag

 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version:
 return None
 if not (serdag := dag_version.serialized_dag):
 return None

 # Double-checked locking: another thread may have cached it while we queried DB.
 # Only emit the miss metric after confirming no other thread cached it, to avoid
 # counting a single lookup as both a miss and a hit.
 if self._use_cache:
 with self._lock:
 if dag := self._dags.get(version_id):
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag
 Stats.incr("api_server.dag_bag.cache_miss")
 return self._read_dag(serdag)
Comment thread
kaxil marked this conversation as resolved.


 def get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 """Get a dag by its version id, using cache if enabled."""
 return self._get_dag(version_id=version_id, session=session)

 def get_serialized_dag_model(self, version_id: UUID | str, session: Session) -> SerializedDagModel | None:
 """
 Return the SerializedDagModel for a given dag version id.

 This will first consult the in-memory cache keyed by the dag version id. If the
 model is not cached, the database is queried for a corresponding :class:`DagVersion`
 and its associated :class:`SerializedDagModel`.
 Always queries the database. The triggerer needs the full model
 for ``serialized_dag_model.data``, which cannot be stored in the
 LRU/TTL cache (it stores deserialized SerializedDAG objects).
 """
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serdag := dag_version.serialized_dag):
 return None
 serdag.load_op_links = self.load_op_links
 return serdag

 :param version_id: The UUID of the dag version to look up.
 :param session: SQLAlchemy session used to query the database.
 :return: The serialized DAG model if found either in the cache or the database; ``None``
 is returned when no :class:`DagVersion` exists for the given ``version_id`` or
 when that :class:`DagVersion` does not have an associated :class:`SerializedDagModel`.
 :rtype: SerializedDagModel | None
 def clear_cache(self) -> int:
 """
 Clear all cached DAGs and serialized DAG models.

 Note: If a serialized dag model is found in the database it will be stored in the
 internal cache (``self._dags``) before being returned.
 :return: Number of entries cleared from the DAG cache.
 """
Comment thread
kaxil marked this conversation as resolved.

 if not (serialized_dag_model := self._dags.get(version_id)):
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serialized_dag_model := dag_version.serialized_dag):
 return None
 self._read_dag(serialized_dag_model)
 return serialized_dag_model

 def get_dag(self, version_id: UUID, session: Session) -> SerializedDAG | None:
 if serialized_dag_model := self.get_serialized_dag_model(version_id=version_id, session=session):
 return serialized_dag_model.dag
 return None
 with self._lock:
 count = len(self._dags)
 self._dags.clear()

 if self._use_cache:
 Stats.incr("api_server.dag_bag.cache_clear")
 Stats.gauge("api_server.dag_bag.cache_size", 0)
 return count
Comment thread
kaxil marked this conversation as resolved.


 @staticmethod
 def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:
Expand All@@ -94,24 +166,30 @@ def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:

 def get_dag_for_run(self, dag_run: DagRun, session: Session) -> SerializedDAG | None:
 if version_id := self._version_from_dag_run(dag_run=dag_run, session=session):
 return self.get_dag(version_id=version_id, session=session)
 return self._get_dag(version_id=version_id, session=session)
 return None

 def iter_all_latest_version_dags(self, *, session: Session) -> Generator[SerializedDAG, None, None]:
 """Walk through all latest version dags available in the database."""
 """
 Walk through all latest version dags available in the database.

 Note: This method does NOT cache the DAGs to avoid cache thrashing when
 iterating over many DAGs. Each DAG is deserialized fresh from the database.
 """
 from airflow.models.serialized_dag import SerializedDagModel

 for serialized_dag_model in session.scalars(select(SerializedDagModel)):
 if dag := self._read_dag(serialized_dag_model):
 for sdm in session.scalars(select(SerializedDagModel)):
 sdm.load_op_links = self.load_op_links
 if dag := sdm.dag:
 yield dag

 def get_latest_version_of_dag(self, dag_id: str, *, session: Session) -> SerializedDAG | None:
 """Get the latest version of a dag by its id."""
 from airflow.models.serialized_dag import SerializedDagModel

 if not (serialized_dag_model := SerializedDagModel.get(dag_id, session=session)):
 if not (serdag := SerializedDagModel.get(dag_id, session=session)):
 return None
 return self._read_dag(serialized_dag_model)
 return self._read_dag(serdag)


def generate_md5_hash(context):
Expand Down

28 changes: 28 additions & 0 deletions
airflow-core/tests/unit/api_fastapi/common/test_dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,6 +19,7 @@
from unittest import mock

import pytest
from cachetools import LRUCache, TTLCache

from airflow.api_fastapi.app import purge_cached_app
from airflow.sdk import BaseOperator
Expand DownExpand Up@@ -82,3 +83,30 @@ def test_dagbag_used_as_singleton_in_dependency(self, session, dag_maker, test_c
 assert resp2.status_code == 200

 assert self.dagbag_call_counter["count"] == 1


class TestCreateDagBag:
 """Tests for create_dag_bag() function."""

Comment thread
kaxil marked this conversation as resolved.

 @pytest.mark.parametrize(
 ["cache_size", "cache_ttl", "expected_use_cache", "expected_dags_type"],
 [
 pytest.param(64, 3600, True, TTLCache, id="default_ttl_cache"),
 pytest.param(0, 3600, False, dict, id="size_zero_unbounded"),
 pytest.param(64, 0, True, LRUCache, id="ttl_zero_lru_only"),
 ],
 )
 @mock.patch("airflow.api_fastapi.common.dagbag.conf")
 def test_create_dag_bag_cache_modes(
 self, mock_conf, cache_size, cache_ttl, expected_use_cache, expected_dags_type
 ):
 from airflow.api_fastapi.common.dagbag import create_dag_bag

 mock_conf.getint.side_effect = lambda section, key, fallback: {
 "dag_cache_size": cache_size,
 "dag_cache_ttl": cache_ttl,
 }.get(key, fallback)

 dag_bag = create_dag_bag()
 assert dag_bag._use_cache is expected_use_cache
 assert isinstance(dag_bag._dags, expected_dags_type)
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Add configurable LRU+TTL caching for API server DAG retrieval#60804

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kaxil
merged 7 commits into

from

Apr 17, 2026
 Merged


Add configurable LRU+TTL caching for API server DAG retrieval
#60804
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
9ed6486

Add configurable LRU+TTL caching for API server DAG retrieval


kaxil Apr 11, 2026
d3dc177

Add docs and newsfragment for DAG cache feature


kaxil Apr 11, 2026
99faf78

Add bounded DAG caching section to API server memory FAQ


kaxil Apr 11, 2026
f8b3398

Fix config wording: cache_size=0 means unbounded dict, not disabled


kaxil Apr 11, 2026
03ed0f8

Fix version_added to 3.3.0 and update FAQ version


kaxil Apr 15, 2026
dcb2ab5

Remove unbounded _dag_models dict


kaxil Apr 15, 2026
d31c6ef

Apply suggestions from code review


kaxil Apr 17, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


6 changes: 6 additions & 0 deletions
airflow-core/docs/administration-and-deployment/web-stack.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -142,6 +142,8 @@ The following configuration options are available in the ``[api]`` section:
- ``server_type``: ``uvicorn`` (default) or ``gunicorn``
- ``worker_refresh_interval``: Seconds between worker refresh cycles (0 = disabled, default)
- ``worker_refresh_batch_size``: Number of workers to refresh per cycle (default: 1)
- ``dag_cache_size``: Max cached SerializedDAG versions in the API server (default: 64, 0 = unbounded)
- ``dag_cache_ttl``: TTL in seconds for cached DAGs (default: 3600, 0 = LRU only)
- ``reload_on_plugin_change``: Reload when plugin files change (default: False)

When to Use Gunicorn
Expand DownExpand Up@@ -187,6 +189,10 @@ For example, to trigger a rolling restart of the API server pods:

 kubectl rollout restart deployment airflow-api-server

The API server also supports bounded DAG caching via ``dag_cache_size`` and
``dag_cache_ttl``, which limits memory consumed by cached SerializedDAG objects.
This reduces memory growth from DAG version accumulation regardless of server type.

In many Kubernetes environments, relying solely on Kubernetes OOM kills or
crash restarts is not recommended, as memory growth may not always trigger an
OOM event. For production deployments that require automatic worker recycling
Expand Down

29 changes: 25 additions & 4 deletions
airflow-core/docs/faq.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -677,8 +677,28 @@ How to prevent API server memory growth?
The API server caches serialized Dag objects in memory. Over time, as Dag versions accumulate
(see :ref:`faq:dag-version-inflation`), this cache grows and can consume several gigabytes of memory.

The recommended solution (available since Airflow 3.2.0) is to use **gunicorn** with **rolling worker
restarts**. Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
There are two complementary approaches:

**1. Bounded DAG caching (available since Airflow 3.3.0)**

The API server supports LRU+TTL caching that bounds how many serialized Dag versions are kept
in memory. Configure this in the ``[api]`` section:

.. code-block:: ini

 [api]
 dag_cache_size = 64 ; max cached versions (0 = unbounded, pre-3.2 behavior)
 dag_cache_ttl = 3600 ; seconds before a cached entry expires (0 = LRU only)

The cache is keyed by Dag version ID. After a Dag is updated, the API server may serve the
previous version until the cached entry expires (controlled by ``dag_cache_ttl``).

See :ref:`config:api__dag_cache_size` and :ref:`config:api__dag_cache_ttl` for the full
configuration reference.

**2. Gunicorn with rolling worker restarts (available since Airflow 3.2.0)**

Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
uses ``preload`` + ``fork``, so workers share read-only memory pages via copy-on-write, reducing overall
memory usage by 40-50% compared to uvicorn's multiprocess mode.

Expand All@@ -699,8 +719,9 @@ See :ref:`config:api__server_type`, :ref:`config:api__worker_refresh_interval`,

.. note::

 Worker recycling handles memory growth from *any* source, not just the Dag cache. It is the
 recommended approach for production API server deployments.
 Worker recycling handles memory growth from *any* source, not just the Dag cache.
 For production deployments, using both bounded caching and gunicorn worker recycling
 provides the best results.


MySQL and MySQL variant Databases
Expand Down

1 change: 1 addition & 0 deletions
airflow-core/newsfragments/60804.feature.rst

View file


Open in desktop

Comment thread
kaxil marked this conversation as resolved.

Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
Add configurable LRU+TTL caching for API server DAG retrieval via ``dag_cache_size`` and ``dag_cache_ttl`` config options in the ``[api]`` section. This bounds memory growth from accumulated SerializedDAG objects in long-running API server processes.

1 change: 1 addition & 0 deletions
airflow-core/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,7 @@ version = "3.3.0"

dependencies = [
 "a2wsgi>=1.10.8",
 "cachetools>=6.0.0",
 # aiosqlite 0.22.0 has a problem with hanging pytest sessions and we excluded it
 # See https://github.com/omnilib/aiosqlite/issues/369
 # It seems that while our test issues are fixed in 0.22.1, sqlalchemy 2 itself
Expand Down

25 changes: 23 additions & 2 deletions
airflow-core/src/airflow/api_fastapi/common/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,21 +16,42 @@
# under the License.
from __future__ import annotations

import logging
from typing import TYPE_CHECKING, Annotated

from fastapi import Depends, HTTPException, Request, status
from sqlalchemy.orm import Session

from airflow.configuration import conf
from airflow.models.dagbag import DBDagBag

if TYPE_CHECKING:
 from airflow.models.dagrun import DagRun
 from airflow.serialization.definitions.dag import SerializedDAG

log = logging.getLogger(__name__)


def create_dag_bag() -> DBDagBag:
 """Create DagBag to retrieve DAGs from the database."""
 return DBDagBag()
 """Create DagBag with configurable LRU+TTL caching for API server usage."""
 cache_size = conf.getint("api", "dag_cache_size", fallback=64)
 cache_ttl_config = conf.getint("api", "dag_cache_ttl", fallback=3600)

 if cache_size < 0:
 log.warning("dag_cache_size must be >= 0, using unbounded dict")
 cache_size = 0
 if cache_ttl_config < 0:
 log.warning("dag_cache_ttl must be >= 0, disabling TTL")
 cache_ttl_config = 0

 # Use unbounded dict (no eviction) if cache_size is 0
 if cache_size <= 0:
 return DBDagBag(cache_size=0)

 # Disable TTL if cache_ttl is 0
Comment thread
kaxil marked this conversation as resolved.

 cache_ttl: int | None = cache_ttl_config if cache_ttl_config > 0 else None

 return DBDagBag(cache_size=cache_size, cache_ttl=cache_ttl)


def dag_bag_from_app(request: Request) -> DBDagBag:
Expand Down

24 changes: 24 additions & 0 deletions
airflow-core/src/airflow/config_templates/config.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1606,6 +1606,30 @@
 type: string
 example: ~
 default: "False"
 dag_cache_size:
 description: |
 Size of the LRU cache for SerializedDAG objects in the API server.
 Set to 0 to use an unbounded dict (no eviction, pre-3.2 behavior). The cache is keyed by DAG version ID,

Check failure
on line 1612
in airflow-core/src/airflow/config_templates/config.yml

View workflow job for this annotation
GitHub Actions/ CI image checks / Static checks
1612:111 [line-length] line too long (112 > 110 characters)
 so lookups by DAG ID (e.g., viewing a DAG's details) always query
 the database for the latest version, but the deserialized result is
 cached for subsequent version-specific lookups.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "64"
 dag_cache_ttl:
 description: |
 Time-to-live (seconds) for cached SerializedDAG objects in the API server.
 After this time, cached DAGs will be re-fetched from the database on next access.
 Set to 0 to disable TTL (cache entries will only be evicted by LRU policy).

 Note: After a DAG is updated, the API server may serve the previous version
 until the cached entry expires. Lower values reduce staleness but increase
 database load.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "3600"
 base_url:
 description: |
 The base url of the API server. Airflow cannot guess what domain or CNAME you are using.
Expand Down

152 changes: 115 additions & 37 deletions
airflow-core/src/airflow/models/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -18,12 +18,17 @@
from __future__ import annotations

import hashlib
from collections.abc import MutableMapping
from contextlib import nullcontext
from threading import RLock
from typing import TYPE_CHECKING, Any
from uuid import UUID

from cachetools import LRUCache, TTLCache
from sqlalchemy import String, select
from sqlalchemy.orm import Mapped, joinedload, mapped_column

from airflow._shared.observability.metrics.stats import Stats
from airflow.models.base import Base, StringID
from airflow.models.dag_version import DagVersion

Expand All@@ -39,50 +44,117 @@

class DBDagBag:
 """
 Internal class for retrieving and caching dags in the scheduler.
 Internal class for retrieving dags from the database.

 Optionally supports LRU+TTL caching when cache_size is provided.
 The scheduler uses this without caching, while the API server can
 enable caching via configuration.

 :meta private:
 """

 def __init__(self, load_op_links: bool = True) -> None:
 self._dags: dict[UUID, SerializedDagModel] = {} # dag_version_id to dag
 self.load_op_links = load_op_links
 def __init__(
 self,
 load_op_links: bool = True,
 cache_size: int | None = None,
 cache_ttl: int | None = None,
 ) -> None:
 """
 Initialize DBDagBag.

 def _read_dag(self, serialized_dag_model: SerializedDagModel) -> SerializedDAG | None:
 serialized_dag_model.load_op_links = self.load_op_links
 if dag := serialized_dag_model.dag:
 self._dags[serialized_dag_model.dag_version_id] = serialized_dag_model
 :param load_op_links: Should the extra operator link be loaded when de-serializing the DAG?
 :param cache_size: Size of LRU cache. If None or 0, uses unbounded dict (no eviction).
 :param cache_ttl: Time-to-live for cache entries in seconds. If None or 0, no TTL (LRU only).
 """
 self.load_op_links = load_op_links
 self._dags: MutableMapping[UUID | str, SerializedDAG] = {}
 self._use_cache = False

 # Initialize bounded cache if cache_size is provided and > 0
 if cache_size and cache_size > 0:
Comment thread
kaxil marked this conversation as resolved.

 if cache_ttl and cache_ttl > 0:
 self._dags = TTLCache(maxsize=cache_size, ttl=cache_ttl)
 else:
 self._dags = LRUCache(maxsize=cache_size)
 self._use_cache = True

 # Lock required for bounded caches: cachetools caches are NOT thread-safe
 # (LRU reordering and TTL cleanup mutate internal linked lists).
 # nullcontext for unbounded dict avoids lock overhead in the scheduler path.
 self._lock: RLock | nullcontext = RLock() if self._use_cache else nullcontext()

Comment thread
kaxil marked this conversation as resolved.

 def _read_dag(self, serdag: SerializedDagModel) -> SerializedDAG | None:
 """Read and optionally cache a SerializedDAG from a SerializedDagModel."""
 serdag.load_op_links = self.load_op_links
 dag = serdag.dag
 if not dag:
 return None
 with self._lock:
 self._dags[serdag.dag_version_id] = dag
 cache_size = len(self._dags)
 if self._use_cache:
 Stats.gauge("api_server.dag_bag.cache_size", cache_size, rate=0.1)
 return dag

 def get_serialized_dag_model(self, version_id: UUID, session: Session) -> SerializedDagModel | None:
 def _get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 # Check cache first
 with self._lock:
 dag = self._dags.get(version_id)

 if dag:
 if self._use_cache:
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag

 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version:
 return None
 if not (serdag := dag_version.serialized_dag):
 return None

 # Double-checked locking: another thread may have cached it while we queried DB.
 # Only emit the miss metric after confirming no other thread cached it, to avoid
 # counting a single lookup as both a miss and a hit.
 if self._use_cache:
 with self._lock:
 if dag := self._dags.get(version_id):
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag
 Stats.incr("api_server.dag_bag.cache_miss")
 return self._read_dag(serdag)
Comment thread
kaxil marked this conversation as resolved.


 def get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 """Get a dag by its version id, using cache if enabled."""
 return self._get_dag(version_id=version_id, session=session)

 def get_serialized_dag_model(self, version_id: UUID | str, session: Session) -> SerializedDagModel | None:
 """
 Return the SerializedDagModel for a given dag version id.

 This will first consult the in-memory cache keyed by the dag version id. If the
 model is not cached, the database is queried for a corresponding :class:`DagVersion`
 and its associated :class:`SerializedDagModel`.
 Always queries the database. The triggerer needs the full model
 for ``serialized_dag_model.data``, which cannot be stored in the
 LRU/TTL cache (it stores deserialized SerializedDAG objects).
 """
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serdag := dag_version.serialized_dag):
 return None
 serdag.load_op_links = self.load_op_links
 return serdag

 :param version_id: The UUID of the dag version to look up.
 :param session: SQLAlchemy session used to query the database.
 :return: The serialized DAG model if found either in the cache or the database; ``None``
 is returned when no :class:`DagVersion` exists for the given ``version_id`` or
 when that :class:`DagVersion` does not have an associated :class:`SerializedDagModel`.
 :rtype: SerializedDagModel | None
 def clear_cache(self) -> int:
 """
 Clear all cached DAGs and serialized DAG models.

 Note: If a serialized dag model is found in the database it will be stored in the
 internal cache (``self._dags``) before being returned.
 :return: Number of entries cleared from the DAG cache.
 """
Comment thread
kaxil marked this conversation as resolved.

 if not (serialized_dag_model := self._dags.get(version_id)):
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serialized_dag_model := dag_version.serialized_dag):
 return None
 self._read_dag(serialized_dag_model)
 return serialized_dag_model

 def get_dag(self, version_id: UUID, session: Session) -> SerializedDAG | None:
 if serialized_dag_model := self.get_serialized_dag_model(version_id=version_id, session=session):
 return serialized_dag_model.dag
 return None
 with self._lock:
 count = len(self._dags)
 self._dags.clear()

 if self._use_cache:
 Stats.incr("api_server.dag_bag.cache_clear")
 Stats.gauge("api_server.dag_bag.cache_size", 0)
 return count
Comment thread
kaxil marked this conversation as resolved.


 @staticmethod
 def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:
Expand All@@ -94,24 +166,30 @@ def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:

 def get_dag_for_run(self, dag_run: DagRun, session: Session) -> SerializedDAG | None:
 if version_id := self._version_from_dag_run(dag_run=dag_run, session=session):
 return self.get_dag(version_id=version_id, session=session)
 return self._get_dag(version_id=version_id, session=session)
 return None

 def iter_all_latest_version_dags(self, *, session: Session) -> Generator[SerializedDAG, None, None]:
 """Walk through all latest version dags available in the database."""
 """
 Walk through all latest version dags available in the database.

 Note: This method does NOT cache the DAGs to avoid cache thrashing when
 iterating over many DAGs. Each DAG is deserialized fresh from the database.
 """
 from airflow.models.serialized_dag import SerializedDagModel

 for serialized_dag_model in session.scalars(select(SerializedDagModel)):
 if dag := self._read_dag(serialized_dag_model):
 for sdm in session.scalars(select(SerializedDagModel)):
 sdm.load_op_links = self.load_op_links
 if dag := sdm.dag:
 yield dag

 def get_latest_version_of_dag(self, dag_id: str, *, session: Session) -> SerializedDAG | None:
 """Get the latest version of a dag by its id."""
 from airflow.models.serialized_dag import SerializedDagModel

 if not (serialized_dag_model := SerializedDagModel.get(dag_id, session=session)):
 if not (serdag := SerializedDagModel.get(dag_id, session=session)):
 return None
 return self._read_dag(serialized_dag_model)
 return self._read_dag(serdag)


def generate_md5_hash(context):
Expand Down

28 changes: 28 additions & 0 deletions
airflow-core/tests/unit/api_fastapi/common/test_dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,6 +19,7 @@
from unittest import mock

import pytest
from cachetools import LRUCache, TTLCache

from airflow.api_fastapi.app import purge_cached_app
from airflow.sdk import BaseOperator
Expand DownExpand Up@@ -82,3 +83,30 @@ def test_dagbag_used_as_singleton_in_dependency(self, session, dag_maker, test_c
 assert resp2.status_code == 200

 assert self.dagbag_call_counter["count"] == 1


class TestCreateDagBag:
 """Tests for create_dag_bag() function."""

Comment thread
kaxil marked this conversation as resolved.

 @pytest.mark.parametrize(
 ["cache_size", "cache_ttl", "expected_use_cache", "expected_dags_type"],
 [
 pytest.param(64, 3600, True, TTLCache, id="default_ttl_cache"),
 pytest.param(0, 3600, False, dict, id="size_zero_unbounded"),
 pytest.param(64, 0, True, LRUCache, id="ttl_zero_lru_only"),
 ],
 )
 @mock.patch("airflow.api_fastapi.common.dagbag.conf")
 def test_create_dag_bag_cache_modes(
 self, mock_conf, cache_size, cache_ttl, expected_use_cache, expected_dags_type
 ):
 from airflow.api_fastapi.common.dagbag import create_dag_bag

 mock_conf.getint.side_effect = lambda section, key, fallback: {
 "dag_cache_size": cache_size,
 "dag_cache_ttl": cache_ttl,
 }.get(key, fallback)

 dag_bag = create_dag_bag()
 assert dag_bag._use_cache is expected_use_cache
 assert isinstance(dag_bag._dags, expected_dags_type)
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Add configurable LRU+TTL caching for API server DAG retrieval#60804

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

kaxil
merged 7 commits into

from

Apr 17, 2026
 Merged


Add configurable LRU+TTL caching for API server DAG retrieval
#60804
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
9ed6486

Add configurable LRU+TTL caching for API server DAG retrieval


kaxil Apr 11, 2026
d3dc177

Add docs and newsfragment for DAG cache feature


kaxil Apr 11, 2026
99faf78

Add bounded DAG caching section to API server memory FAQ


kaxil Apr 11, 2026
f8b3398

Fix config wording: cache_size=0 means unbounded dict, not disabled


kaxil Apr 11, 2026
03ed0f8

Fix version_added to 3.3.0 and update FAQ version


kaxil Apr 15, 2026
dcb2ab5

Remove unbounded _dag_models dict


kaxil Apr 15, 2026
d31c6ef

Apply suggestions from code review


kaxil Apr 17, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


6 changes: 6 additions & 0 deletions
airflow-core/docs/administration-and-deployment/web-stack.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -142,6 +142,8 @@ The following configuration options are available in the ``[api]`` section:
- ``server_type``: ``uvicorn`` (default) or ``gunicorn``
- ``worker_refresh_interval``: Seconds between worker refresh cycles (0 = disabled, default)
- ``worker_refresh_batch_size``: Number of workers to refresh per cycle (default: 1)
- ``dag_cache_size``: Max cached SerializedDAG versions in the API server (default: 64, 0 = unbounded)
- ``dag_cache_ttl``: TTL in seconds for cached DAGs (default: 3600, 0 = LRU only)
- ``reload_on_plugin_change``: Reload when plugin files change (default: False)

When to Use Gunicorn
Expand DownExpand Up@@ -187,6 +189,10 @@ For example, to trigger a rolling restart of the API server pods:

 kubectl rollout restart deployment airflow-api-server

The API server also supports bounded DAG caching via ``dag_cache_size`` and
``dag_cache_ttl``, which limits memory consumed by cached SerializedDAG objects.
This reduces memory growth from DAG version accumulation regardless of server type.

In many Kubernetes environments, relying solely on Kubernetes OOM kills or
crash restarts is not recommended, as memory growth may not always trigger an
OOM event. For production deployments that require automatic worker recycling
Expand Down

29 changes: 25 additions & 4 deletions
airflow-core/docs/faq.rst

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -677,8 +677,28 @@ How to prevent API server memory growth?
The API server caches serialized Dag objects in memory. Over time, as Dag versions accumulate
(see :ref:`faq:dag-version-inflation`), this cache grows and can consume several gigabytes of memory.

The recommended solution (available since Airflow 3.2.0) is to use **gunicorn** with **rolling worker
restarts**. Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
There are two complementary approaches:

**1. Bounded DAG caching (available since Airflow 3.3.0)**

The API server supports LRU+TTL caching that bounds how many serialized Dag versions are kept
in memory. Configure this in the ``[api]`` section:

.. code-block:: ini

 [api]
 dag_cache_size = 64 ; max cached versions (0 = unbounded, pre-3.2 behavior)
 dag_cache_ttl = 3600 ; seconds before a cached entry expires (0 = LRU only)

The cache is keyed by Dag version ID. After a Dag is updated, the API server may serve the
previous version until the cached entry expires (controlled by ``dag_cache_ttl``).

See :ref:`config:api__dag_cache_size` and :ref:`config:api__dag_cache_ttl` for the full
configuration reference.

**2. Gunicorn with rolling worker restarts (available since Airflow 3.2.0)**

Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also
uses ``preload`` + ``fork``, so workers share read-only memory pages via copy-on-write, reducing overall
memory usage by 40-50% compared to uvicorn's multiprocess mode.

Expand All@@ -699,8 +719,9 @@ See :ref:`config:api__server_type`, :ref:`config:api__worker_refresh_interval`,

.. note::

 Worker recycling handles memory growth from *any* source, not just the Dag cache. It is the
 recommended approach for production API server deployments.
 Worker recycling handles memory growth from *any* source, not just the Dag cache.
 For production deployments, using both bounded caching and gunicorn worker recycling
 provides the best results.


MySQL and MySQL variant Databases
Expand Down

1 change: 1 addition & 0 deletions
airflow-core/newsfragments/60804.feature.rst

View file


Open in desktop

Comment thread
kaxil marked this conversation as resolved.

Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
Add configurable LRU+TTL caching for API server DAG retrieval via ``dag_cache_size`` and ``dag_cache_ttl`` config options in the ``[api]`` section. This bounds memory growth from accumulated SerializedDAG objects in long-running API server processes.

1 change: 1 addition & 0 deletions
airflow-core/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -68,6 +68,7 @@ version = "3.3.0"

dependencies = [
 "a2wsgi>=1.10.8",
 "cachetools>=6.0.0",
 # aiosqlite 0.22.0 has a problem with hanging pytest sessions and we excluded it
 # See https://github.com/omnilib/aiosqlite/issues/369
 # It seems that while our test issues are fixed in 0.22.1, sqlalchemy 2 itself
Expand Down

25 changes: 23 additions & 2 deletions
airflow-core/src/airflow/api_fastapi/common/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,21 +16,42 @@
# under the License.
from __future__ import annotations

import logging
from typing import TYPE_CHECKING, Annotated

from fastapi import Depends, HTTPException, Request, status
from sqlalchemy.orm import Session

from airflow.configuration import conf
from airflow.models.dagbag import DBDagBag

if TYPE_CHECKING:
 from airflow.models.dagrun import DagRun
 from airflow.serialization.definitions.dag import SerializedDAG

log = logging.getLogger(__name__)


def create_dag_bag() -> DBDagBag:
 """Create DagBag to retrieve DAGs from the database."""
 return DBDagBag()
 """Create DagBag with configurable LRU+TTL caching for API server usage."""
 cache_size = conf.getint("api", "dag_cache_size", fallback=64)
 cache_ttl_config = conf.getint("api", "dag_cache_ttl", fallback=3600)

 if cache_size < 0:
 log.warning("dag_cache_size must be >= 0, using unbounded dict")
 cache_size = 0
 if cache_ttl_config < 0:
 log.warning("dag_cache_ttl must be >= 0, disabling TTL")
 cache_ttl_config = 0

 # Use unbounded dict (no eviction) if cache_size is 0
 if cache_size <= 0:
 return DBDagBag(cache_size=0)

 # Disable TTL if cache_ttl is 0
Comment thread
kaxil marked this conversation as resolved.

 cache_ttl: int | None = cache_ttl_config if cache_ttl_config > 0 else None

 return DBDagBag(cache_size=cache_size, cache_ttl=cache_ttl)


def dag_bag_from_app(request: Request) -> DBDagBag:
Expand Down

24 changes: 24 additions & 0 deletions
airflow-core/src/airflow/config_templates/config.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -1606,6 +1606,30 @@
 type: string
 example: ~
 default: "False"
 dag_cache_size:
 description: |
 Size of the LRU cache for SerializedDAG objects in the API server.
 Set to 0 to use an unbounded dict (no eviction, pre-3.2 behavior). The cache is keyed by DAG version ID,

Check failure
on line 1612
in airflow-core/src/airflow/config_templates/config.yml

View workflow job for this annotation
GitHub Actions/ CI image checks / Static checks
1612:111 [line-length] line too long (112 > 110 characters)
 so lookups by DAG ID (e.g., viewing a DAG's details) always query
 the database for the latest version, but the deserialized result is
 cached for subsequent version-specific lookups.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "64"
 dag_cache_ttl:
 description: |
 Time-to-live (seconds) for cached SerializedDAG objects in the API server.
 After this time, cached DAGs will be re-fetched from the database on next access.
 Set to 0 to disable TTL (cache entries will only be evicted by LRU policy).

 Note: After a DAG is updated, the API server may serve the previous version
 until the cached entry expires. Lower values reduce staleness but increase
 database load.
 version_added: 3.3.0
 type: integer
 example: ~
 default: "3600"
 base_url:
 description: |
 The base url of the API server. Airflow cannot guess what domain or CNAME you are using.
Expand Down

152 changes: 115 additions & 37 deletions
airflow-core/src/airflow/models/dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -18,12 +18,17 @@
from __future__ import annotations

import hashlib
from collections.abc import MutableMapping
from contextlib import nullcontext
from threading import RLock
from typing import TYPE_CHECKING, Any
from uuid import UUID

from cachetools import LRUCache, TTLCache
from sqlalchemy import String, select
from sqlalchemy.orm import Mapped, joinedload, mapped_column

from airflow._shared.observability.metrics.stats import Stats
from airflow.models.base import Base, StringID
from airflow.models.dag_version import DagVersion

Expand All@@ -39,50 +44,117 @@

class DBDagBag:
 """
 Internal class for retrieving and caching dags in the scheduler.
 Internal class for retrieving dags from the database.

 Optionally supports LRU+TTL caching when cache_size is provided.
 The scheduler uses this without caching, while the API server can
 enable caching via configuration.

 :meta private:
 """

 def __init__(self, load_op_links: bool = True) -> None:
 self._dags: dict[UUID, SerializedDagModel] = {} # dag_version_id to dag
 self.load_op_links = load_op_links
 def __init__(
 self,
 load_op_links: bool = True,
 cache_size: int | None = None,
 cache_ttl: int | None = None,
 ) -> None:
 """
 Initialize DBDagBag.

 def _read_dag(self, serialized_dag_model: SerializedDagModel) -> SerializedDAG | None:
 serialized_dag_model.load_op_links = self.load_op_links
 if dag := serialized_dag_model.dag:
 self._dags[serialized_dag_model.dag_version_id] = serialized_dag_model
 :param load_op_links: Should the extra operator link be loaded when de-serializing the DAG?
 :param cache_size: Size of LRU cache. If None or 0, uses unbounded dict (no eviction).
 :param cache_ttl: Time-to-live for cache entries in seconds. If None or 0, no TTL (LRU only).
 """
 self.load_op_links = load_op_links
 self._dags: MutableMapping[UUID | str, SerializedDAG] = {}
 self._use_cache = False

 # Initialize bounded cache if cache_size is provided and > 0
 if cache_size and cache_size > 0:
Comment thread
kaxil marked this conversation as resolved.

 if cache_ttl and cache_ttl > 0:
 self._dags = TTLCache(maxsize=cache_size, ttl=cache_ttl)
 else:
 self._dags = LRUCache(maxsize=cache_size)
 self._use_cache = True

 # Lock required for bounded caches: cachetools caches are NOT thread-safe
 # (LRU reordering and TTL cleanup mutate internal linked lists).
 # nullcontext for unbounded dict avoids lock overhead in the scheduler path.
 self._lock: RLock | nullcontext = RLock() if self._use_cache else nullcontext()

Comment thread
kaxil marked this conversation as resolved.

 def _read_dag(self, serdag: SerializedDagModel) -> SerializedDAG | None:
 """Read and optionally cache a SerializedDAG from a SerializedDagModel."""
 serdag.load_op_links = self.load_op_links
 dag = serdag.dag
 if not dag:
 return None
 with self._lock:
 self._dags[serdag.dag_version_id] = dag
 cache_size = len(self._dags)
 if self._use_cache:
 Stats.gauge("api_server.dag_bag.cache_size", cache_size, rate=0.1)
 return dag

 def get_serialized_dag_model(self, version_id: UUID, session: Session) -> SerializedDagModel | None:
 def _get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 # Check cache first
 with self._lock:
 dag = self._dags.get(version_id)

 if dag:
 if self._use_cache:
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag

 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version:
 return None
 if not (serdag := dag_version.serialized_dag):
 return None

 # Double-checked locking: another thread may have cached it while we queried DB.
 # Only emit the miss metric after confirming no other thread cached it, to avoid
 # counting a single lookup as both a miss and a hit.
 if self._use_cache:
 with self._lock:
 if dag := self._dags.get(version_id):
Comment thread
kaxil marked this conversation as resolved.

 Stats.incr("api_server.dag_bag.cache_hit")
 return dag
 Stats.incr("api_server.dag_bag.cache_miss")
 return self._read_dag(serdag)
Comment thread
kaxil marked this conversation as resolved.


 def get_dag(self, version_id: UUID | str, session: Session) -> SerializedDAG | None:
 """Get a dag by its version id, using cache if enabled."""
 return self._get_dag(version_id=version_id, session=session)

 def get_serialized_dag_model(self, version_id: UUID | str, session: Session) -> SerializedDagModel | None:
 """
 Return the SerializedDagModel for a given dag version id.

 This will first consult the in-memory cache keyed by the dag version id. If the
 model is not cached, the database is queried for a corresponding :class:`DagVersion`
 and its associated :class:`SerializedDagModel`.
 Always queries the database. The triggerer needs the full model
 for ``serialized_dag_model.data``, which cannot be stored in the
 LRU/TTL cache (it stores deserialized SerializedDAG objects).
 """
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serdag := dag_version.serialized_dag):
 return None
 serdag.load_op_links = self.load_op_links
 return serdag

 :param version_id: The UUID of the dag version to look up.
 :param session: SQLAlchemy session used to query the database.
 :return: The serialized DAG model if found either in the cache or the database; ``None``
 is returned when no :class:`DagVersion` exists for the given ``version_id`` or
 when that :class:`DagVersion` does not have an associated :class:`SerializedDagModel`.
 :rtype: SerializedDagModel | None
 def clear_cache(self) -> int:
 """
 Clear all cached DAGs and serialized DAG models.

 Note: If a serialized dag model is found in the database it will be stored in the
 internal cache (``self._dags``) before being returned.
 :return: Number of entries cleared from the DAG cache.
 """
Comment thread
kaxil marked this conversation as resolved.

 if not (serialized_dag_model := self._dags.get(version_id)):
 dag_version = session.get(DagVersion, version_id, options=[joinedload(DagVersion.serialized_dag)])
 if not dag_version or not (serialized_dag_model := dag_version.serialized_dag):
 return None
 self._read_dag(serialized_dag_model)
 return serialized_dag_model

 def get_dag(self, version_id: UUID, session: Session) -> SerializedDAG | None:
 if serialized_dag_model := self.get_serialized_dag_model(version_id=version_id, session=session):
 return serialized_dag_model.dag
 return None
 with self._lock:
 count = len(self._dags)
 self._dags.clear()

 if self._use_cache:
 Stats.incr("api_server.dag_bag.cache_clear")
 Stats.gauge("api_server.dag_bag.cache_size", 0)
 return count
Comment thread
kaxil marked this conversation as resolved.


 @staticmethod
 def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:
Expand All@@ -94,24 +166,30 @@ def _version_from_dag_run(dag_run: DagRun, *, session: Session) -> UUID | None:

 def get_dag_for_run(self, dag_run: DagRun, session: Session) -> SerializedDAG | None:
 if version_id := self._version_from_dag_run(dag_run=dag_run, session=session):
 return self.get_dag(version_id=version_id, session=session)
 return self._get_dag(version_id=version_id, session=session)
 return None

 def iter_all_latest_version_dags(self, *, session: Session) -> Generator[SerializedDAG, None, None]:
 """Walk through all latest version dags available in the database."""
 """
 Walk through all latest version dags available in the database.

 Note: This method does NOT cache the DAGs to avoid cache thrashing when
 iterating over many DAGs. Each DAG is deserialized fresh from the database.
 """
 from airflow.models.serialized_dag import SerializedDagModel

 for serialized_dag_model in session.scalars(select(SerializedDagModel)):
 if dag := self._read_dag(serialized_dag_model):
 for sdm in session.scalars(select(SerializedDagModel)):
 sdm.load_op_links = self.load_op_links
 if dag := sdm.dag:
 yield dag

 def get_latest_version_of_dag(self, dag_id: str, *, session: Session) -> SerializedDAG | None:
 """Get the latest version of a dag by its id."""
 from airflow.models.serialized_dag import SerializedDagModel

 if not (serialized_dag_model := SerializedDagModel.get(dag_id, session=session)):
 if not (serdag := SerializedDagModel.get(dag_id, session=session)):
 return None
 return self._read_dag(serialized_dag_model)
 return self._read_dag(serdag)


def generate_md5_hash(context):
Expand Down

28 changes: 28 additions & 0 deletions
airflow-core/tests/unit/api_fastapi/common/test_dagbag.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -19,6 +19,7 @@
from unittest import mock

import pytest
from cachetools import LRUCache, TTLCache

from airflow.api_fastapi.app import purge_cached_app
from airflow.sdk import BaseOperator
Expand DownExpand Up@@ -82,3 +83,30 @@ def test_dagbag_used_as_singleton_in_dependency(self, session, dag_maker, test_c
 assert resp2.status_code == 200

 assert self.dagbag_call_counter["count"] == 1


class TestCreateDagBag:
 """Tests for create_dag_bag() function."""

Comment thread
kaxil marked this conversation as resolved.

 @pytest.mark.parametrize(
 ["cache_size", "cache_ttl", "expected_use_cache", "expected_dags_type"],
 [
 pytest.param(64, 3600, True, TTLCache, id="default_ttl_cache"),
 pytest.param(0, 3600, False, dict, id="size_zero_unbounded"),
 pytest.param(64, 0, True, LRUCache, id="ttl_zero_lru_only"),
 ],
 )
 @mock.patch("airflow.api_fastapi.common.dagbag.conf")
 def test_create_dag_bag_cache_modes(
 self, mock_conf, cache_size, cache_ttl, expected_use_cache, expected_dags_type
 ):
 from airflow.api_fastapi.common.dagbag import create_dag_bag

 mock_conf.getint.side_effect = lambda section, key, fallback: {
 "dag_cache_size": cache_size,
 "dag_cache_ttl": cache_ttl,
 }.get(key, fallback)

 dag_bag = create_dag_bag()
 assert dag_bag._use_cache is expected_use_cache
 assert isinstance(dag_bag._dags, expected_dags_type)
Loading
Loading