Fix race condition in auth manager initialization - #62214

Merged
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init
Feb 22, 2026
Merged

Fix race condition in auth manager initialization#62214
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init

Conversation

@kimyoungi99

Copy link
Copy Markdown
Contributor

Closes#61108

Problem

When sending concurrent requests to /auth/token, intermittent 500 errors occur:

AttributeError: 'AirflowAppBuilder' object has no attribute 'sm'

create_auth_manager() creates a new instance on every call without checking for an existing one. Under concurrent requests, one thread can overwrite _AuthManagerState.instance while another thread's instance is still being initialized via init(), resulting in an uninitialized auth manager being served.

Fix

Apply double-checked locking in create_auth_manager() so the auth manager is created exactly once:

defcreate_auth_manager() ->BaseAuthManager:
if_AuthManagerState.instanceisNone:
with_AuthManagerState._lock:
if_AuthManagerState.instanceisNone:
auth_manager_cls=get_auth_manager_cls()
_AuthManagerState.instance=auth_manager_cls()
return_AuthManagerState.instance

The first check avoids lock overhead on subsequent calls. The second check (inside the lock) prevents duplicate creation when multiple threads pass the first check simultaneously.

What's changed

  • airflow-core/src/airflow/api_fastapi/app.py: Added threading.Lock to _AuthManagerState and guarded instance creation with double-checked locking
  • airflow-core/tests/unit/api_fastapi/test_app.py: Added test_create_auth_manager_thread_safety — spawns 10 concurrent threads and verifies singleton behavior

Testing

  • All 9 tests in test_app.py pass locally (including the new one)
  • prek, ruff check, ruff format all clean

@boring-cyborgboring-cyborgBot added the area:API Airflow's REST/HTTP API label Feb 20, 2026
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 4b355ba to 231ee38CompareFebruary 20, 2026 10:42

@vincbeckvincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

@kimyoungi99

Copy link
Copy Markdown
ContributorAuthor

Fixed the CI failure in test_upgradedb[auth1-2].

The initial singleton implementation cached the auth manager instance unconditionally, but upgradedb() switches between auth manager classes via config (e.g. SimpleAuthManager → FabAuthManager). The cached instance from a previous config was being returned instead of creating a new one for the updated config.

Added an isinstance check so the singleton is invalidated when the configured auth manager class changes. The thread-safety guarantee (double-checked locking) remains intact.

  • test_upgradedb[auth0-1]
  • test_upgradedb[auth1-2] ✅ (was failing)
  • test_create_auth_manager_thread_safety

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for helping out, I just approved the CI.

Just FYI, we might need to update the provider tests if necessary for this version (I’m not sure yet). For the previous PR that tried to resolve the issue (#61310), we need to update the provider tests.

Comment threadairflow-core/src/airflow/api_fastapi/app.py Outdated
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from dad0c4e to 6792939CompareFebruary 20, 2026 18:18
@kimyoungi99

kimyoungi99 commented Feb 20, 2026

Copy link
Copy Markdown
ContributorAuthor

Looked into the CI failures — I think the provider DB tests (MySQL, Postgres, Sqlite) are failing because the singleton caching returns a stale FabAuthManager instance bound to a previous Flask app context (KeyError: 'AUTH_USER_REGISTRATION').

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

Thanks for updating the context.

I just approved the CI again, let's see whether current PR are able to pass all CI.
If current changes already able to pass the CI, I think we're good to go and there is no need to change other areas just to pass the unit test.

Additionally, the auth manager will initialized in FastAPI instead of Flask context

@auth_router.get(
"/logout",
status_code=status.HTTP_307_TEMPORARY_REDIRECT,
)
deflogout(request: Request) ->RedirectResponse:
"""Generate a new API token."""
withget_application_builder():
login_url=get_auth_manager().get_url_login()
secure=request.base_url.scheme=="https"orbool(conf.get("api", "ssl_cert", fallback=""))
response=RedirectResponse(login_url)
response.delete_cookie(
key="session",
secure=secure,
httponly=True,
)
response.delete_cookie(
key=COOKIE_NAME_JWT_TOKEN,
secure=secure,
httponly=True,
)
returnresponse

Calling purge_cached_app to clear the auth manager instance for every API call might not be efficient IMHO.

Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
@potiuk
potiukforce-pushed the fix/thread-safe-auth-manager-init branch from 6875a29 to ab08975CompareFebruary 21, 2026 20:23
@kimyoungi99

kimyoungi99 commented Feb 21, 2026

Copy link
Copy Markdown
ContributorAuthor

Sorry for the extra CI round — my Breeze setup was slightly off so I thought the previous push would pass.
First-time contributor here, bear with me :)

The singleton's @cached_property on security_manager was causing AUTH_USER_REGISTRATION KeyErrors when multiple Flask apps are created in the same process (e.g., CLI commands calling get_application_builder() internally). Fixed by:

  • Clearing the auth manager singleton in get_application_builder() before creating a new Flask app
  • Adding purge_cached_app() to test fixtures that call application.create_app() with a different auth manager config

Verified locally using Breeze (--backend postgres --python 3.10).
All green.

Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 881bdbe to c49d3daCompareFebruary 22, 2026 00:01

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kimyoungi99, LGTM!

@jason810496
jason810496 merged commit 5c9171a into apache:mainFeb 22, 2026
129 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-1-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 5c9171a v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

jason810496 pushed a commit to jason810496/airflow that referenced this pull request Feb 22, 2026
…#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
@jason810496

Copy link
Copy Markdown
Member

I manually backported in #62326.

@kimyoungi99
kimyoungi99 deleted the fix/thread-safe-auth-manager-init branch February 22, 2026 22:08
jason810496 added a commit that referenced this pull request Feb 24, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
pierrejeambrun added a commit that referenced this pull request Feb 24, 2026
@pierrejeambrun

Copy link
Copy Markdown
Member

@kimyoungi99 we will revert this PR because it's not ready to merge actually. It's causing some trouble. Do you mind re-opening a PR so we can keep improving and fixing before we merge it again?

@kimyoungi99

kimyoungi99 commented Feb 24, 2026

Copy link
Copy Markdown
ContributorAuthor

@pierrejeambrun Thanks for catching this. I'll re-open a PR with the production code change in get_application_builder() removed.

vatsrahul1001 pushed a commit that referenced this pull request Mar 4, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Race condition causes "AirflowAppBuilder has no attribute 'sm'" on concurrent auth requests

4 participants

@kimyoungi99@jason810496@pierrejeambrun@vincbeck
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Fix race condition in auth manager initialization - #62214

Merged
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init
Feb 22, 2026
Merged

Fix race condition in auth manager initialization#62214
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init

Conversation

@kimyoungi99

Copy link
Copy Markdown
Contributor

Closes#61108

Problem

When sending concurrent requests to /auth/token, intermittent 500 errors occur:

AttributeError: 'AirflowAppBuilder' object has no attribute 'sm'

create_auth_manager() creates a new instance on every call without checking for an existing one. Under concurrent requests, one thread can overwrite _AuthManagerState.instance while another thread's instance is still being initialized via init(), resulting in an uninitialized auth manager being served.

Fix

Apply double-checked locking in create_auth_manager() so the auth manager is created exactly once:

defcreate_auth_manager() ->BaseAuthManager:
if_AuthManagerState.instanceisNone:
with_AuthManagerState._lock:
if_AuthManagerState.instanceisNone:
auth_manager_cls=get_auth_manager_cls()
_AuthManagerState.instance=auth_manager_cls()
return_AuthManagerState.instance

The first check avoids lock overhead on subsequent calls. The second check (inside the lock) prevents duplicate creation when multiple threads pass the first check simultaneously.

What's changed

  • airflow-core/src/airflow/api_fastapi/app.py: Added threading.Lock to _AuthManagerState and guarded instance creation with double-checked locking
  • airflow-core/tests/unit/api_fastapi/test_app.py: Added test_create_auth_manager_thread_safety — spawns 10 concurrent threads and verifies singleton behavior

Testing

  • All 9 tests in test_app.py pass locally (including the new one)
  • prek, ruff check, ruff format all clean

@boring-cyborgboring-cyborgBot added the area:API Airflow's REST/HTTP API label Feb 20, 2026
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 4b355ba to 231ee38CompareFebruary 20, 2026 10:42

@vincbeckvincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

@kimyoungi99

Copy link
Copy Markdown
ContributorAuthor

Fixed the CI failure in test_upgradedb[auth1-2].

The initial singleton implementation cached the auth manager instance unconditionally, but upgradedb() switches between auth manager classes via config (e.g. SimpleAuthManager → FabAuthManager). The cached instance from a previous config was being returned instead of creating a new one for the updated config.

Added an isinstance check so the singleton is invalidated when the configured auth manager class changes. The thread-safety guarantee (double-checked locking) remains intact.

  • test_upgradedb[auth0-1]
  • test_upgradedb[auth1-2] ✅ (was failing)
  • test_create_auth_manager_thread_safety

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for helping out, I just approved the CI.

Just FYI, we might need to update the provider tests if necessary for this version (I’m not sure yet). For the previous PR that tried to resolve the issue (#61310), we need to update the provider tests.

Comment threadairflow-core/src/airflow/api_fastapi/app.py Outdated
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from dad0c4e to 6792939CompareFebruary 20, 2026 18:18
@kimyoungi99

kimyoungi99 commented Feb 20, 2026

Copy link
Copy Markdown
ContributorAuthor

Looked into the CI failures — I think the provider DB tests (MySQL, Postgres, Sqlite) are failing because the singleton caching returns a stale FabAuthManager instance bound to a previous Flask app context (KeyError: 'AUTH_USER_REGISTRATION').

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

Thanks for updating the context.

I just approved the CI again, let's see whether current PR are able to pass all CI.
If current changes already able to pass the CI, I think we're good to go and there is no need to change other areas just to pass the unit test.

Additionally, the auth manager will initialized in FastAPI instead of Flask context

@auth_router.get(
"/logout",
status_code=status.HTTP_307_TEMPORARY_REDIRECT,
)
deflogout(request: Request) ->RedirectResponse:
"""Generate a new API token."""
withget_application_builder():
login_url=get_auth_manager().get_url_login()
secure=request.base_url.scheme=="https"orbool(conf.get("api", "ssl_cert", fallback=""))
response=RedirectResponse(login_url)
response.delete_cookie(
key="session",
secure=secure,
httponly=True,
)
response.delete_cookie(
key=COOKIE_NAME_JWT_TOKEN,
secure=secure,
httponly=True,
)
returnresponse

Calling purge_cached_app to clear the auth manager instance for every API call might not be efficient IMHO.

Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
@potiuk
potiukforce-pushed the fix/thread-safe-auth-manager-init branch from 6875a29 to ab08975CompareFebruary 21, 2026 20:23
@kimyoungi99

kimyoungi99 commented Feb 21, 2026

Copy link
Copy Markdown
ContributorAuthor

Sorry for the extra CI round — my Breeze setup was slightly off so I thought the previous push would pass.
First-time contributor here, bear with me :)

The singleton's @cached_property on security_manager was causing AUTH_USER_REGISTRATION KeyErrors when multiple Flask apps are created in the same process (e.g., CLI commands calling get_application_builder() internally). Fixed by:

  • Clearing the auth manager singleton in get_application_builder() before creating a new Flask app
  • Adding purge_cached_app() to test fixtures that call application.create_app() with a different auth manager config

Verified locally using Breeze (--backend postgres --python 3.10).
All green.

Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 881bdbe to c49d3daCompareFebruary 22, 2026 00:01

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kimyoungi99, LGTM!

@jason810496
jason810496 merged commit 5c9171a into apache:mainFeb 22, 2026
129 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-1-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 5c9171a v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

jason810496 pushed a commit to jason810496/airflow that referenced this pull request Feb 22, 2026
…#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
@jason810496

Copy link
Copy Markdown
Member

I manually backported in #62326.

@kimyoungi99
kimyoungi99 deleted the fix/thread-safe-auth-manager-init branch February 22, 2026 22:08
jason810496 added a commit that referenced this pull request Feb 24, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
pierrejeambrun added a commit that referenced this pull request Feb 24, 2026
@pierrejeambrun

Copy link
Copy Markdown
Member

@kimyoungi99 we will revert this PR because it's not ready to merge actually. It's causing some trouble. Do you mind re-opening a PR so we can keep improving and fixing before we merge it again?

@kimyoungi99

kimyoungi99 commented Feb 24, 2026

Copy link
Copy Markdown
ContributorAuthor

@pierrejeambrun Thanks for catching this. I'll re-open a PR with the production code change in get_application_builder() removed.

vatsrahul1001 pushed a commit that referenced this pull request Mar 4, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Race condition causes "AirflowAppBuilder has no attribute 'sm'" on concurrent auth requests

4 participants

@kimyoungi99@jason810496@pierrejeambrun@vincbeck
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix race condition in auth manager initialization - #62214

Merged
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init
Feb 22, 2026
Merged

Fix race condition in auth manager initialization#62214
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init

Conversation

@kimyoungi99

Copy link
Copy Markdown
Contributor

Closes#61108

Problem

When sending concurrent requests to /auth/token, intermittent 500 errors occur:

AttributeError: 'AirflowAppBuilder' object has no attribute 'sm'

create_auth_manager() creates a new instance on every call without checking for an existing one. Under concurrent requests, one thread can overwrite _AuthManagerState.instance while another thread's instance is still being initialized via init(), resulting in an uninitialized auth manager being served.

Fix

Apply double-checked locking in create_auth_manager() so the auth manager is created exactly once:

defcreate_auth_manager() ->BaseAuthManager:
if_AuthManagerState.instanceisNone:
with_AuthManagerState._lock:
if_AuthManagerState.instanceisNone:
auth_manager_cls=get_auth_manager_cls()
_AuthManagerState.instance=auth_manager_cls()
return_AuthManagerState.instance

The first check avoids lock overhead on subsequent calls. The second check (inside the lock) prevents duplicate creation when multiple threads pass the first check simultaneously.

What's changed

  • airflow-core/src/airflow/api_fastapi/app.py: Added threading.Lock to _AuthManagerState and guarded instance creation with double-checked locking
  • airflow-core/tests/unit/api_fastapi/test_app.py: Added test_create_auth_manager_thread_safety — spawns 10 concurrent threads and verifies singleton behavior

Testing

  • All 9 tests in test_app.py pass locally (including the new one)
  • prek, ruff check, ruff format all clean

@boring-cyborgboring-cyborgBot added the area:API Airflow's REST/HTTP API label Feb 20, 2026
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 4b355ba to 231ee38CompareFebruary 20, 2026 10:42

@vincbeckvincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

@kimyoungi99

Copy link
Copy Markdown
ContributorAuthor

Fixed the CI failure in test_upgradedb[auth1-2].

The initial singleton implementation cached the auth manager instance unconditionally, but upgradedb() switches between auth manager classes via config (e.g. SimpleAuthManager → FabAuthManager). The cached instance from a previous config was being returned instead of creating a new one for the updated config.

Added an isinstance check so the singleton is invalidated when the configured auth manager class changes. The thread-safety guarantee (double-checked locking) remains intact.

  • test_upgradedb[auth0-1]
  • test_upgradedb[auth1-2] ✅ (was failing)
  • test_create_auth_manager_thread_safety

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for helping out, I just approved the CI.

Just FYI, we might need to update the provider tests if necessary for this version (I’m not sure yet). For the previous PR that tried to resolve the issue (#61310), we need to update the provider tests.

Comment threadairflow-core/src/airflow/api_fastapi/app.py Outdated
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from dad0c4e to 6792939CompareFebruary 20, 2026 18:18
@kimyoungi99

kimyoungi99 commented Feb 20, 2026

Copy link
Copy Markdown
ContributorAuthor

Looked into the CI failures — I think the provider DB tests (MySQL, Postgres, Sqlite) are failing because the singleton caching returns a stale FabAuthManager instance bound to a previous Flask app context (KeyError: 'AUTH_USER_REGISTRATION').

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

Thanks for updating the context.

I just approved the CI again, let's see whether current PR are able to pass all CI.
If current changes already able to pass the CI, I think we're good to go and there is no need to change other areas just to pass the unit test.

Additionally, the auth manager will initialized in FastAPI instead of Flask context

@auth_router.get(
"/logout",
status_code=status.HTTP_307_TEMPORARY_REDIRECT,
)
deflogout(request: Request) ->RedirectResponse:
"""Generate a new API token."""
withget_application_builder():
login_url=get_auth_manager().get_url_login()
secure=request.base_url.scheme=="https"orbool(conf.get("api", "ssl_cert", fallback=""))
response=RedirectResponse(login_url)
response.delete_cookie(
key="session",
secure=secure,
httponly=True,
)
response.delete_cookie(
key=COOKIE_NAME_JWT_TOKEN,
secure=secure,
httponly=True,
)
returnresponse

Calling purge_cached_app to clear the auth manager instance for every API call might not be efficient IMHO.

Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
@potiuk
potiukforce-pushed the fix/thread-safe-auth-manager-init branch from 6875a29 to ab08975CompareFebruary 21, 2026 20:23
@kimyoungi99

kimyoungi99 commented Feb 21, 2026

Copy link
Copy Markdown
ContributorAuthor

Sorry for the extra CI round — my Breeze setup was slightly off so I thought the previous push would pass.
First-time contributor here, bear with me :)

The singleton's @cached_property on security_manager was causing AUTH_USER_REGISTRATION KeyErrors when multiple Flask apps are created in the same process (e.g., CLI commands calling get_application_builder() internally). Fixed by:

  • Clearing the auth manager singleton in get_application_builder() before creating a new Flask app
  • Adding purge_cached_app() to test fixtures that call application.create_app() with a different auth manager config

Verified locally using Breeze (--backend postgres --python 3.10).
All green.

Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 881bdbe to c49d3daCompareFebruary 22, 2026 00:01

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kimyoungi99, LGTM!

@jason810496
jason810496 merged commit 5c9171a into apache:mainFeb 22, 2026
129 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-1-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 5c9171a v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

jason810496 pushed a commit to jason810496/airflow that referenced this pull request Feb 22, 2026
…#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
@jason810496

Copy link
Copy Markdown
Member

I manually backported in #62326.

@kimyoungi99
kimyoungi99 deleted the fix/thread-safe-auth-manager-init branch February 22, 2026 22:08
jason810496 added a commit that referenced this pull request Feb 24, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
pierrejeambrun added a commit that referenced this pull request Feb 24, 2026
@pierrejeambrun

Copy link
Copy Markdown
Member

@kimyoungi99 we will revert this PR because it's not ready to merge actually. It's causing some trouble. Do you mind re-opening a PR so we can keep improving and fixing before we merge it again?

@kimyoungi99

kimyoungi99 commented Feb 24, 2026

Copy link
Copy Markdown
ContributorAuthor

@pierrejeambrun Thanks for catching this. I'll re-open a PR with the production code change in get_application_builder() removed.

vatsrahul1001 pushed a commit that referenced this pull request Mar 4, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Race condition causes "AirflowAppBuilder has no attribute 'sm'" on concurrent auth requests

4 participants

@kimyoungi99@jason810496@pierrejeambrun@vincbeck
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix race condition in auth manager initialization - #62214

Merged
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init
Feb 22, 2026
Merged

Fix race condition in auth manager initialization#62214
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init

Conversation

@kimyoungi99

Copy link
Copy Markdown
Contributor

Closes#61108

Problem

When sending concurrent requests to /auth/token, intermittent 500 errors occur:

AttributeError: 'AirflowAppBuilder' object has no attribute 'sm'

create_auth_manager() creates a new instance on every call without checking for an existing one. Under concurrent requests, one thread can overwrite _AuthManagerState.instance while another thread's instance is still being initialized via init(), resulting in an uninitialized auth manager being served.

Fix

Apply double-checked locking in create_auth_manager() so the auth manager is created exactly once:

defcreate_auth_manager() ->BaseAuthManager:
if_AuthManagerState.instanceisNone:
with_AuthManagerState._lock:
if_AuthManagerState.instanceisNone:
auth_manager_cls=get_auth_manager_cls()
_AuthManagerState.instance=auth_manager_cls()
return_AuthManagerState.instance

The first check avoids lock overhead on subsequent calls. The second check (inside the lock) prevents duplicate creation when multiple threads pass the first check simultaneously.

What's changed

  • airflow-core/src/airflow/api_fastapi/app.py: Added threading.Lock to _AuthManagerState and guarded instance creation with double-checked locking
  • airflow-core/tests/unit/api_fastapi/test_app.py: Added test_create_auth_manager_thread_safety — spawns 10 concurrent threads and verifies singleton behavior

Testing

  • All 9 tests in test_app.py pass locally (including the new one)
  • prek, ruff check, ruff format all clean

@boring-cyborgboring-cyborgBot added the area:API Airflow's REST/HTTP API label Feb 20, 2026
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 4b355ba to 231ee38CompareFebruary 20, 2026 10:42

@vincbeckvincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

@kimyoungi99

Copy link
Copy Markdown
ContributorAuthor

Fixed the CI failure in test_upgradedb[auth1-2].

The initial singleton implementation cached the auth manager instance unconditionally, but upgradedb() switches between auth manager classes via config (e.g. SimpleAuthManager → FabAuthManager). The cached instance from a previous config was being returned instead of creating a new one for the updated config.

Added an isinstance check so the singleton is invalidated when the configured auth manager class changes. The thread-safety guarantee (double-checked locking) remains intact.

  • test_upgradedb[auth0-1]
  • test_upgradedb[auth1-2] ✅ (was failing)
  • test_create_auth_manager_thread_safety

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for helping out, I just approved the CI.

Just FYI, we might need to update the provider tests if necessary for this version (I’m not sure yet). For the previous PR that tried to resolve the issue (#61310), we need to update the provider tests.

Comment threadairflow-core/src/airflow/api_fastapi/app.py Outdated
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from dad0c4e to 6792939CompareFebruary 20, 2026 18:18
@kimyoungi99

kimyoungi99 commented Feb 20, 2026

Copy link
Copy Markdown
ContributorAuthor

Looked into the CI failures — I think the provider DB tests (MySQL, Postgres, Sqlite) are failing because the singleton caching returns a stale FabAuthManager instance bound to a previous Flask app context (KeyError: 'AUTH_USER_REGISTRATION').

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

Thanks for updating the context.

I just approved the CI again, let's see whether current PR are able to pass all CI.
If current changes already able to pass the CI, I think we're good to go and there is no need to change other areas just to pass the unit test.

Additionally, the auth manager will initialized in FastAPI instead of Flask context

@auth_router.get(
"/logout",
status_code=status.HTTP_307_TEMPORARY_REDIRECT,
)
deflogout(request: Request) ->RedirectResponse:
"""Generate a new API token."""
withget_application_builder():
login_url=get_auth_manager().get_url_login()
secure=request.base_url.scheme=="https"orbool(conf.get("api", "ssl_cert", fallback=""))
response=RedirectResponse(login_url)
response.delete_cookie(
key="session",
secure=secure,
httponly=True,
)
response.delete_cookie(
key=COOKIE_NAME_JWT_TOKEN,
secure=secure,
httponly=True,
)
returnresponse

Calling purge_cached_app to clear the auth manager instance for every API call might not be efficient IMHO.

Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
@potiuk
potiukforce-pushed the fix/thread-safe-auth-manager-init branch from 6875a29 to ab08975CompareFebruary 21, 2026 20:23
@kimyoungi99

kimyoungi99 commented Feb 21, 2026

Copy link
Copy Markdown
ContributorAuthor

Sorry for the extra CI round — my Breeze setup was slightly off so I thought the previous push would pass.
First-time contributor here, bear with me :)

The singleton's @cached_property on security_manager was causing AUTH_USER_REGISTRATION KeyErrors when multiple Flask apps are created in the same process (e.g., CLI commands calling get_application_builder() internally). Fixed by:

  • Clearing the auth manager singleton in get_application_builder() before creating a new Flask app
  • Adding purge_cached_app() to test fixtures that call application.create_app() with a different auth manager config

Verified locally using Breeze (--backend postgres --python 3.10).
All green.

Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 881bdbe to c49d3daCompareFebruary 22, 2026 00:01

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kimyoungi99, LGTM!

@jason810496
jason810496 merged commit 5c9171a into apache:mainFeb 22, 2026
129 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-1-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 5c9171a v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

jason810496 pushed a commit to jason810496/airflow that referenced this pull request Feb 22, 2026
…#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
@jason810496

Copy link
Copy Markdown
Member

I manually backported in #62326.

@kimyoungi99
kimyoungi99 deleted the fix/thread-safe-auth-manager-init branch February 22, 2026 22:08
jason810496 added a commit that referenced this pull request Feb 24, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
pierrejeambrun added a commit that referenced this pull request Feb 24, 2026
@pierrejeambrun

Copy link
Copy Markdown
Member

@kimyoungi99 we will revert this PR because it's not ready to merge actually. It's causing some trouble. Do you mind re-opening a PR so we can keep improving and fixing before we merge it again?

@kimyoungi99

kimyoungi99 commented Feb 24, 2026

Copy link
Copy Markdown
ContributorAuthor

@pierrejeambrun Thanks for catching this. I'll re-open a PR with the production code change in get_application_builder() removed.

vatsrahul1001 pushed a commit that referenced this pull request Mar 4, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Race condition causes "AirflowAppBuilder has no attribute 'sm'" on concurrent auth requests

4 participants

@kimyoungi99@jason810496@pierrejeambrun@vincbeck
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Fix race condition in auth manager initialization - #62214

Merged
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init
Feb 22, 2026
Merged

Fix race condition in auth manager initialization#62214
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init

Conversation

@kimyoungi99

Copy link
Copy Markdown
Contributor

Closes#61108

Problem

When sending concurrent requests to /auth/token, intermittent 500 errors occur:

AttributeError: 'AirflowAppBuilder' object has no attribute 'sm'

create_auth_manager() creates a new instance on every call without checking for an existing one. Under concurrent requests, one thread can overwrite _AuthManagerState.instance while another thread's instance is still being initialized via init(), resulting in an uninitialized auth manager being served.

Fix

Apply double-checked locking in create_auth_manager() so the auth manager is created exactly once:

defcreate_auth_manager() ->BaseAuthManager:
if_AuthManagerState.instanceisNone:
with_AuthManagerState._lock:
if_AuthManagerState.instanceisNone:
auth_manager_cls=get_auth_manager_cls()
_AuthManagerState.instance=auth_manager_cls()
return_AuthManagerState.instance

The first check avoids lock overhead on subsequent calls. The second check (inside the lock) prevents duplicate creation when multiple threads pass the first check simultaneously.

What's changed

  • airflow-core/src/airflow/api_fastapi/app.py: Added threading.Lock to _AuthManagerState and guarded instance creation with double-checked locking
  • airflow-core/tests/unit/api_fastapi/test_app.py: Added test_create_auth_manager_thread_safety — spawns 10 concurrent threads and verifies singleton behavior

Testing

  • All 9 tests in test_app.py pass locally (including the new one)
  • prek, ruff check, ruff format all clean

@boring-cyborgboring-cyborgBot added the area:API Airflow's REST/HTTP API label Feb 20, 2026
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 4b355ba to 231ee38CompareFebruary 20, 2026 10:42

@vincbeckvincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

@kimyoungi99

Copy link
Copy Markdown
ContributorAuthor

Fixed the CI failure in test_upgradedb[auth1-2].

The initial singleton implementation cached the auth manager instance unconditionally, but upgradedb() switches between auth manager classes via config (e.g. SimpleAuthManager → FabAuthManager). The cached instance from a previous config was being returned instead of creating a new one for the updated config.

Added an isinstance check so the singleton is invalidated when the configured auth manager class changes. The thread-safety guarantee (double-checked locking) remains intact.

  • test_upgradedb[auth0-1]
  • test_upgradedb[auth1-2] ✅ (was failing)
  • test_create_auth_manager_thread_safety

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for helping out, I just approved the CI.

Just FYI, we might need to update the provider tests if necessary for this version (I’m not sure yet). For the previous PR that tried to resolve the issue (#61310), we need to update the provider tests.

Comment threadairflow-core/src/airflow/api_fastapi/app.py Outdated
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from dad0c4e to 6792939CompareFebruary 20, 2026 18:18
@kimyoungi99

kimyoungi99 commented Feb 20, 2026

Copy link
Copy Markdown
ContributorAuthor

Looked into the CI failures — I think the provider DB tests (MySQL, Postgres, Sqlite) are failing because the singleton caching returns a stale FabAuthManager instance bound to a previous Flask app context (KeyError: 'AUTH_USER_REGISTRATION').

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

Thanks for updating the context.

I just approved the CI again, let's see whether current PR are able to pass all CI.
If current changes already able to pass the CI, I think we're good to go and there is no need to change other areas just to pass the unit test.

Additionally, the auth manager will initialized in FastAPI instead of Flask context

@auth_router.get(
"/logout",
status_code=status.HTTP_307_TEMPORARY_REDIRECT,
)
deflogout(request: Request) ->RedirectResponse:
"""Generate a new API token."""
withget_application_builder():
login_url=get_auth_manager().get_url_login()
secure=request.base_url.scheme=="https"orbool(conf.get("api", "ssl_cert", fallback=""))
response=RedirectResponse(login_url)
response.delete_cookie(
key="session",
secure=secure,
httponly=True,
)
response.delete_cookie(
key=COOKIE_NAME_JWT_TOKEN,
secure=secure,
httponly=True,
)
returnresponse

Calling purge_cached_app to clear the auth manager instance for every API call might not be efficient IMHO.

Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
@potiuk
potiukforce-pushed the fix/thread-safe-auth-manager-init branch from 6875a29 to ab08975CompareFebruary 21, 2026 20:23
@kimyoungi99

kimyoungi99 commented Feb 21, 2026

Copy link
Copy Markdown
ContributorAuthor

Sorry for the extra CI round — my Breeze setup was slightly off so I thought the previous push would pass.
First-time contributor here, bear with me :)

The singleton's @cached_property on security_manager was causing AUTH_USER_REGISTRATION KeyErrors when multiple Flask apps are created in the same process (e.g., CLI commands calling get_application_builder() internally). Fixed by:

  • Clearing the auth manager singleton in get_application_builder() before creating a new Flask app
  • Adding purge_cached_app() to test fixtures that call application.create_app() with a different auth manager config

Verified locally using Breeze (--backend postgres --python 3.10).
All green.

Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 881bdbe to c49d3daCompareFebruary 22, 2026 00:01

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kimyoungi99, LGTM!

@jason810496
jason810496 merged commit 5c9171a into apache:mainFeb 22, 2026
129 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-1-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 5c9171a v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

jason810496 pushed a commit to jason810496/airflow that referenced this pull request Feb 22, 2026
…#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
@jason810496

Copy link
Copy Markdown
Member

I manually backported in #62326.

@kimyoungi99
kimyoungi99 deleted the fix/thread-safe-auth-manager-init branch February 22, 2026 22:08
jason810496 added a commit that referenced this pull request Feb 24, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
pierrejeambrun added a commit that referenced this pull request Feb 24, 2026
@pierrejeambrun

Copy link
Copy Markdown
Member

@kimyoungi99 we will revert this PR because it's not ready to merge actually. It's causing some trouble. Do you mind re-opening a PR so we can keep improving and fixing before we merge it again?

@kimyoungi99

kimyoungi99 commented Feb 24, 2026

Copy link
Copy Markdown
ContributorAuthor

@pierrejeambrun Thanks for catching this. I'll re-open a PR with the production code change in get_application_builder() removed.

vatsrahul1001 pushed a commit that referenced this pull request Mar 4, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Race condition causes "AirflowAppBuilder has no attribute 'sm'" on concurrent auth requests

4 participants

@kimyoungi99@jason810496@pierrejeambrun@vincbeck
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix race condition in auth manager initialization - #62214

Merged
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init
Feb 22, 2026
Merged

Fix race condition in auth manager initialization#62214
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init

Conversation

@kimyoungi99

Copy link
Copy Markdown
Contributor

Closes#61108

Problem

When sending concurrent requests to /auth/token, intermittent 500 errors occur:

AttributeError: 'AirflowAppBuilder' object has no attribute 'sm'

create_auth_manager() creates a new instance on every call without checking for an existing one. Under concurrent requests, one thread can overwrite _AuthManagerState.instance while another thread's instance is still being initialized via init(), resulting in an uninitialized auth manager being served.

Fix

Apply double-checked locking in create_auth_manager() so the auth manager is created exactly once:

defcreate_auth_manager() ->BaseAuthManager:
if_AuthManagerState.instanceisNone:
with_AuthManagerState._lock:
if_AuthManagerState.instanceisNone:
auth_manager_cls=get_auth_manager_cls()
_AuthManagerState.instance=auth_manager_cls()
return_AuthManagerState.instance

The first check avoids lock overhead on subsequent calls. The second check (inside the lock) prevents duplicate creation when multiple threads pass the first check simultaneously.

What's changed

  • airflow-core/src/airflow/api_fastapi/app.py: Added threading.Lock to _AuthManagerState and guarded instance creation with double-checked locking
  • airflow-core/tests/unit/api_fastapi/test_app.py: Added test_create_auth_manager_thread_safety — spawns 10 concurrent threads and verifies singleton behavior

Testing

  • All 9 tests in test_app.py pass locally (including the new one)
  • prek, ruff check, ruff format all clean

@boring-cyborgboring-cyborgBot added the area:API Airflow's REST/HTTP API label Feb 20, 2026
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 4b355ba to 231ee38CompareFebruary 20, 2026 10:42

@vincbeckvincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

@kimyoungi99

Copy link
Copy Markdown
ContributorAuthor

Fixed the CI failure in test_upgradedb[auth1-2].

The initial singleton implementation cached the auth manager instance unconditionally, but upgradedb() switches between auth manager classes via config (e.g. SimpleAuthManager → FabAuthManager). The cached instance from a previous config was being returned instead of creating a new one for the updated config.

Added an isinstance check so the singleton is invalidated when the configured auth manager class changes. The thread-safety guarantee (double-checked locking) remains intact.

  • test_upgradedb[auth0-1]
  • test_upgradedb[auth1-2] ✅ (was failing)
  • test_create_auth_manager_thread_safety

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for helping out, I just approved the CI.

Just FYI, we might need to update the provider tests if necessary for this version (I’m not sure yet). For the previous PR that tried to resolve the issue (#61310), we need to update the provider tests.

Comment threadairflow-core/src/airflow/api_fastapi/app.py Outdated
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from dad0c4e to 6792939CompareFebruary 20, 2026 18:18
@kimyoungi99

kimyoungi99 commented Feb 20, 2026

Copy link
Copy Markdown
ContributorAuthor

Looked into the CI failures — I think the provider DB tests (MySQL, Postgres, Sqlite) are failing because the singleton caching returns a stale FabAuthManager instance bound to a previous Flask app context (KeyError: 'AUTH_USER_REGISTRATION').

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

Thanks for updating the context.

I just approved the CI again, let's see whether current PR are able to pass all CI.
If current changes already able to pass the CI, I think we're good to go and there is no need to change other areas just to pass the unit test.

Additionally, the auth manager will initialized in FastAPI instead of Flask context

@auth_router.get(
"/logout",
status_code=status.HTTP_307_TEMPORARY_REDIRECT,
)
deflogout(request: Request) ->RedirectResponse:
"""Generate a new API token."""
withget_application_builder():
login_url=get_auth_manager().get_url_login()
secure=request.base_url.scheme=="https"orbool(conf.get("api", "ssl_cert", fallback=""))
response=RedirectResponse(login_url)
response.delete_cookie(
key="session",
secure=secure,
httponly=True,
)
response.delete_cookie(
key=COOKIE_NAME_JWT_TOKEN,
secure=secure,
httponly=True,
)
returnresponse

Calling purge_cached_app to clear the auth manager instance for every API call might not be efficient IMHO.

Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
@potiuk
potiukforce-pushed the fix/thread-safe-auth-manager-init branch from 6875a29 to ab08975CompareFebruary 21, 2026 20:23
@kimyoungi99

kimyoungi99 commented Feb 21, 2026

Copy link
Copy Markdown
ContributorAuthor

Sorry for the extra CI round — my Breeze setup was slightly off so I thought the previous push would pass.
First-time contributor here, bear with me :)

The singleton's @cached_property on security_manager was causing AUTH_USER_REGISTRATION KeyErrors when multiple Flask apps are created in the same process (e.g., CLI commands calling get_application_builder() internally). Fixed by:

  • Clearing the auth manager singleton in get_application_builder() before creating a new Flask app
  • Adding purge_cached_app() to test fixtures that call application.create_app() with a different auth manager config

Verified locally using Breeze (--backend postgres --python 3.10).
All green.

Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 881bdbe to c49d3daCompareFebruary 22, 2026 00:01

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kimyoungi99, LGTM!

@jason810496
jason810496 merged commit 5c9171a into apache:mainFeb 22, 2026
129 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-1-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 5c9171a v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

jason810496 pushed a commit to jason810496/airflow that referenced this pull request Feb 22, 2026
…#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
@jason810496

Copy link
Copy Markdown
Member

I manually backported in #62326.

@kimyoungi99
kimyoungi99 deleted the fix/thread-safe-auth-manager-init branch February 22, 2026 22:08
jason810496 added a commit that referenced this pull request Feb 24, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
pierrejeambrun added a commit that referenced this pull request Feb 24, 2026
@pierrejeambrun

Copy link
Copy Markdown
Member

@kimyoungi99 we will revert this PR because it's not ready to merge actually. It's causing some trouble. Do you mind re-opening a PR so we can keep improving and fixing before we merge it again?

@kimyoungi99

kimyoungi99 commented Feb 24, 2026

Copy link
Copy Markdown
ContributorAuthor

@pierrejeambrun Thanks for catching this. I'll re-open a PR with the production code change in get_application_builder() removed.

vatsrahul1001 pushed a commit that referenced this pull request Mar 4, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Race condition causes "AirflowAppBuilder has no attribute 'sm'" on concurrent auth requests

4 participants

@kimyoungi99@jason810496@pierrejeambrun@vincbeck
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix race condition in auth manager initialization - #62214

Merged
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init
Feb 22, 2026
Merged

Fix race condition in auth manager initialization#62214
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init

Conversation

@kimyoungi99

Copy link
Copy Markdown
Contributor

Closes#61108

Problem

When sending concurrent requests to /auth/token, intermittent 500 errors occur:

AttributeError: 'AirflowAppBuilder' object has no attribute 'sm'

create_auth_manager() creates a new instance on every call without checking for an existing one. Under concurrent requests, one thread can overwrite _AuthManagerState.instance while another thread's instance is still being initialized via init(), resulting in an uninitialized auth manager being served.

Fix

Apply double-checked locking in create_auth_manager() so the auth manager is created exactly once:

defcreate_auth_manager() ->BaseAuthManager:
if_AuthManagerState.instanceisNone:
with_AuthManagerState._lock:
if_AuthManagerState.instanceisNone:
auth_manager_cls=get_auth_manager_cls()
_AuthManagerState.instance=auth_manager_cls()
return_AuthManagerState.instance

The first check avoids lock overhead on subsequent calls. The second check (inside the lock) prevents duplicate creation when multiple threads pass the first check simultaneously.

What's changed

  • airflow-core/src/airflow/api_fastapi/app.py: Added threading.Lock to _AuthManagerState and guarded instance creation with double-checked locking
  • airflow-core/tests/unit/api_fastapi/test_app.py: Added test_create_auth_manager_thread_safety — spawns 10 concurrent threads and verifies singleton behavior

Testing

  • All 9 tests in test_app.py pass locally (including the new one)
  • prek, ruff check, ruff format all clean

@boring-cyborgboring-cyborgBot added the area:API Airflow's REST/HTTP API label Feb 20, 2026
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 4b355ba to 231ee38CompareFebruary 20, 2026 10:42

@vincbeckvincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

@kimyoungi99

Copy link
Copy Markdown
ContributorAuthor

Fixed the CI failure in test_upgradedb[auth1-2].

The initial singleton implementation cached the auth manager instance unconditionally, but upgradedb() switches between auth manager classes via config (e.g. SimpleAuthManager → FabAuthManager). The cached instance from a previous config was being returned instead of creating a new one for the updated config.

Added an isinstance check so the singleton is invalidated when the configured auth manager class changes. The thread-safety guarantee (double-checked locking) remains intact.

  • test_upgradedb[auth0-1]
  • test_upgradedb[auth1-2] ✅ (was failing)
  • test_create_auth_manager_thread_safety

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for helping out, I just approved the CI.

Just FYI, we might need to update the provider tests if necessary for this version (I’m not sure yet). For the previous PR that tried to resolve the issue (#61310), we need to update the provider tests.

Comment threadairflow-core/src/airflow/api_fastapi/app.py Outdated
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from dad0c4e to 6792939CompareFebruary 20, 2026 18:18
@kimyoungi99

kimyoungi99 commented Feb 20, 2026

Copy link
Copy Markdown
ContributorAuthor

Looked into the CI failures — I think the provider DB tests (MySQL, Postgres, Sqlite) are failing because the singleton caching returns a stale FabAuthManager instance bound to a previous Flask app context (KeyError: 'AUTH_USER_REGISTRATION').

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

Thanks for updating the context.

I just approved the CI again, let's see whether current PR are able to pass all CI.
If current changes already able to pass the CI, I think we're good to go and there is no need to change other areas just to pass the unit test.

Additionally, the auth manager will initialized in FastAPI instead of Flask context

@auth_router.get(
"/logout",
status_code=status.HTTP_307_TEMPORARY_REDIRECT,
)
deflogout(request: Request) ->RedirectResponse:
"""Generate a new API token."""
withget_application_builder():
login_url=get_auth_manager().get_url_login()
secure=request.base_url.scheme=="https"orbool(conf.get("api", "ssl_cert", fallback=""))
response=RedirectResponse(login_url)
response.delete_cookie(
key="session",
secure=secure,
httponly=True,
)
response.delete_cookie(
key=COOKIE_NAME_JWT_TOKEN,
secure=secure,
httponly=True,
)
returnresponse

Calling purge_cached_app to clear the auth manager instance for every API call might not be efficient IMHO.

Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
@potiuk
potiukforce-pushed the fix/thread-safe-auth-manager-init branch from 6875a29 to ab08975CompareFebruary 21, 2026 20:23
@kimyoungi99

kimyoungi99 commented Feb 21, 2026

Copy link
Copy Markdown
ContributorAuthor

Sorry for the extra CI round — my Breeze setup was slightly off so I thought the previous push would pass.
First-time contributor here, bear with me :)

The singleton's @cached_property on security_manager was causing AUTH_USER_REGISTRATION KeyErrors when multiple Flask apps are created in the same process (e.g., CLI commands calling get_application_builder() internally). Fixed by:

  • Clearing the auth manager singleton in get_application_builder() before creating a new Flask app
  • Adding purge_cached_app() to test fixtures that call application.create_app() with a different auth manager config

Verified locally using Breeze (--backend postgres --python 3.10).
All green.

Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 881bdbe to c49d3daCompareFebruary 22, 2026 00:01

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kimyoungi99, LGTM!

@jason810496
jason810496 merged commit 5c9171a into apache:mainFeb 22, 2026
129 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-1-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 5c9171a v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

jason810496 pushed a commit to jason810496/airflow that referenced this pull request Feb 22, 2026
…#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
@jason810496

Copy link
Copy Markdown
Member

I manually backported in #62326.

@kimyoungi99
kimyoungi99 deleted the fix/thread-safe-auth-manager-init branch February 22, 2026 22:08
jason810496 added a commit that referenced this pull request Feb 24, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
pierrejeambrun added a commit that referenced this pull request Feb 24, 2026
@pierrejeambrun

Copy link
Copy Markdown
Member

@kimyoungi99 we will revert this PR because it's not ready to merge actually. It's causing some trouble. Do you mind re-opening a PR so we can keep improving and fixing before we merge it again?

@kimyoungi99

kimyoungi99 commented Feb 24, 2026

Copy link
Copy Markdown
ContributorAuthor

@pierrejeambrun Thanks for catching this. I'll re-open a PR with the production code change in get_application_builder() removed.

vatsrahul1001 pushed a commit that referenced this pull request Mar 4, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Race condition causes "AirflowAppBuilder has no attribute 'sm'" on concurrent auth requests

4 participants

@kimyoungi99@jason810496@pierrejeambrun@vincbeck
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Fix race condition in auth manager initialization - #62214

Merged
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init
Feb 22, 2026
Merged

Fix race condition in auth manager initialization#62214
jason810496 merged 5 commits into
apache:mainfrom
kimyoungi99:fix/thread-safe-auth-manager-init

Conversation

@kimyoungi99

Copy link
Copy Markdown
Contributor

Closes#61108

Problem

When sending concurrent requests to /auth/token, intermittent 500 errors occur:

AttributeError: 'AirflowAppBuilder' object has no attribute 'sm'

create_auth_manager() creates a new instance on every call without checking for an existing one. Under concurrent requests, one thread can overwrite _AuthManagerState.instance while another thread's instance is still being initialized via init(), resulting in an uninitialized auth manager being served.

Fix

Apply double-checked locking in create_auth_manager() so the auth manager is created exactly once:

defcreate_auth_manager() ->BaseAuthManager:
if_AuthManagerState.instanceisNone:
with_AuthManagerState._lock:
if_AuthManagerState.instanceisNone:
auth_manager_cls=get_auth_manager_cls()
_AuthManagerState.instance=auth_manager_cls()
return_AuthManagerState.instance

The first check avoids lock overhead on subsequent calls. The second check (inside the lock) prevents duplicate creation when multiple threads pass the first check simultaneously.

What's changed

  • airflow-core/src/airflow/api_fastapi/app.py: Added threading.Lock to _AuthManagerState and guarded instance creation with double-checked locking
  • airflow-core/tests/unit/api_fastapi/test_app.py: Added test_create_auth_manager_thread_safety — spawns 10 concurrent threads and verifies singleton behavior

Testing

  • All 9 tests in test_app.py pass locally (including the new one)
  • prek, ruff check, ruff format all clean

@boring-cyborgboring-cyborgBot added the area:API Airflow's REST/HTTP API label Feb 20, 2026
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 4b355ba to 231ee38CompareFebruary 20, 2026 10:42

@vincbeckvincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

@kimyoungi99

Copy link
Copy Markdown
ContributorAuthor

Fixed the CI failure in test_upgradedb[auth1-2].

The initial singleton implementation cached the auth manager instance unconditionally, but upgradedb() switches between auth manager classes via config (e.g. SimpleAuthManager → FabAuthManager). The cached instance from a previous config was being returned instead of creating a new one for the updated config.

Added an isinstance check so the singleton is invalidated when the configured auth manager class changes. The thread-safety guarantee (double-checked locking) remains intact.

  • test_upgradedb[auth0-1]
  • test_upgradedb[auth1-2] ✅ (was failing)
  • test_create_auth_manager_thread_safety

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for helping out, I just approved the CI.

Just FYI, we might need to update the provider tests if necessary for this version (I’m not sure yet). For the previous PR that tried to resolve the issue (#61310), we need to update the provider tests.

Comment threadairflow-core/src/airflow/api_fastapi/app.py Outdated
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from dad0c4e to 6792939CompareFebruary 20, 2026 18:18
@kimyoungi99

kimyoungi99 commented Feb 20, 2026

Copy link
Copy Markdown
ContributorAuthor

Looked into the CI failures — I think the provider DB tests (MySQL, Postgres, Sqlite) are failing because the singleton caching returns a stale FabAuthManager instance bound to a previous Flask app context (KeyError: 'AUTH_USER_REGISTRATION').

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking rather than adding cleanup to individual test fixtures, it might make more sense to call purge_cached_app() directly in get_application_builder() since it creates a fresh Flask app each time. What do you think?

Thanks for updating the context.

I just approved the CI again, let's see whether current PR are able to pass all CI.
If current changes already able to pass the CI, I think we're good to go and there is no need to change other areas just to pass the unit test.

Additionally, the auth manager will initialized in FastAPI instead of Flask context

@auth_router.get(
"/logout",
status_code=status.HTTP_307_TEMPORARY_REDIRECT,
)
deflogout(request: Request) ->RedirectResponse:
"""Generate a new API token."""
withget_application_builder():
login_url=get_auth_manager().get_url_login()
secure=request.base_url.scheme=="https"orbool(conf.get("api", "ssl_cert", fallback=""))
response=RedirectResponse(login_url)
response.delete_cookie(
key="session",
secure=secure,
httponly=True,
)
response.delete_cookie(
key=COOKIE_NAME_JWT_TOKEN,
secure=secure,
httponly=True,
)
returnresponse

Calling purge_cached_app to clear the auth manager instance for every API call might not be efficient IMHO.

Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
@potiuk
potiukforce-pushed the fix/thread-safe-auth-manager-init branch from 6875a29 to ab08975CompareFebruary 21, 2026 20:23
@kimyoungi99

kimyoungi99 commented Feb 21, 2026

Copy link
Copy Markdown
ContributorAuthor

Sorry for the extra CI round — my Breeze setup was slightly off so I thought the previous push would pass.
First-time contributor here, bear with me :)

The singleton's @cached_property on security_manager was causing AUTH_USER_REGISTRATION KeyErrors when multiple Flask apps are created in the same process (e.g., CLI commands calling get_application_builder() internally). Fixed by:

  • Clearing the auth manager singleton in get_application_builder() before creating a new Flask app
  • Adding purge_cached_app() to test fixtures that call application.create_app() with a different auth manager config

Verified locally using Breeze (--backend postgres --python 3.10).
All green.

Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
@kimyoungi99
kimyoungi99force-pushed the fix/thread-safe-auth-manager-init branch from 881bdbe to c49d3daCompareFebruary 22, 2026 00:01

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kimyoungi99, LGTM!

@jason810496
jason810496 merged commit 5c9171a into apache:mainFeb 22, 2026
129 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-1-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 5c9171a v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

jason810496 pushed a commit to jason810496/airflow that referenced this pull request Feb 22, 2026
…#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
@jason810496

Copy link
Copy Markdown
Member

I manually backported in #62326.

@kimyoungi99
kimyoungi99 deleted the fix/thread-safe-auth-manager-init branch February 22, 2026 22:08
jason810496 added a commit that referenced this pull request Feb 24, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
pierrejeambrun added a commit that referenced this pull request Feb 24, 2026
@pierrejeambrun

Copy link
Copy Markdown
Member

@kimyoungi99 we will revert this PR because it's not ready to merge actually. It's causing some trouble. Do you mind re-opening a PR so we can keep improving and fixing before we merge it again?

@kimyoungi99

kimyoungi99 commented Feb 24, 2026

Copy link
Copy Markdown
ContributorAuthor

@pierrejeambrun Thanks for catching this. I'll re-open a PR with the production code change in get_application_builder() removed.

vatsrahul1001 pushed a commit that referenced this pull request Mar 4, 2026
#62326)
* [v3-1-test] Fix race condition in auth manager initialization (#62214)
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: #61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
(cherry picked from commit 5c9171a)
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
* Fix CI error
---------
Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com>
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
* Fix race condition in auth manager initialization
Make create_auth_manager() thread-safe using double-checked locking
to prevent concurrent requests from creating multiple auth manager
instances. This fixes intermittent 500 errors on /auth/token when
multiple requests arrive simultaneously.
Closes: apache#61108
* Handle auth manager class change in singleton cache
The singleton check now also verifies the cached instance matches
the currently configured auth manager class. This prevents stale
instances when the config changes (e.g. switching between
SimpleAuthManager and FabAuthManager during db upgrade).
* Avoid calling get_auth_manager_cls on every create_auth_manager call
Move get_auth_manager_cls() inside the lock so the fast path is just a None check. Add purge_cached_app() in test_upgradedb to ensure clean state between parametrized cases with different auth manager configs.
* Reset auth manager singleton in get_application_builder
Since get_application_builder creates a fresh Flask app each time, the cached auth manager singleton from a previous app context must be cleared to avoid stale state (e.g. KeyError on AUTH_USER_REGISTRATION).
* Clear auth manager singleton in test fixtures using create_app
Test fixtures that call application.create_app() or get_application_builder()
can receive a stale auth manager singleton from a previous test, causing
AirflowSecurityManagerV2 to be used instead of FabAirflowSecurityManagerOverride.
Add purge_cached_app() calls to test fixtures across fab, google, and keycloak
providers to ensure each test gets a fresh auth manager instance.
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:APIAirflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Race condition causes "AirflowAppBuilder has no attribute 'sm'" on concurrent auth requests

4 participants

@kimyoungi99@jason810496@pierrejeambrun@vincbeck