Cache requires_access_dag checks and other auth manager redundant checks

[tirkarthi]

Apache Airflow version

main (development)

If "Other Airflow 3 version" selected, which one?

No response

What happened?

In Airflow 2 when a dag page is loaded the grid is loaded along with the details tab. Loading a dag page had only 4-5 requests for grid data, details, asset events etc. In Airflow 3 since the dag page now uses an API call per dagrun it results in 10 API calls along with API calls for other parts of the page like structure_data, plugins, config, backfills, hitl, dag warnings, recent dag runs, recent task instances, asset events, latest run resulting in total of 25-30 API calls. Along with the fact that auto refresh is 3 seconds many of these API calls are made again once the initial API call finishes.

Each API call has a set of dependencies and we noticed that requires_access_dag is checked per request for a ti_summary. For 10 requests to load the grid this results in 10 checks though this never changes in between the API calls. We also noticed that per requires_access_dag also fetches the team for the dag which also doesn't change often, We enabled caching these checks with an in-memory cache and this helped in noticeable performance improvements. This also reduces the db load with the requests served faster helping workers to process other requests. This also helps with other endpoints where require_access_dag check is performed redundantly.

This involves caching the is_authorize_dag check which users can do by customizing the auth manager but I opened this up for discussion. Git branch with patches : https://github.com/tirkarthi/airflow/tree/cache-requires-check

time hey -c 4 -H 'Cookie: session=07b3594d-405a-489b-ae0c-02ee02c9bbdb.uq376NuwgAarpTdjToOPTJuO2a0; _token=eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxIiwianRpIjoiZWVhN2RhODBkZjFjNGRmMjkxZWRmMzE1YzBlNTZmNDQiLCJhdWQiOiJhcGFjaGUtYWlyZmxvdyIsIm5iZiI6MTc3MDI3NDU0MCwiZXhwIjoxNzcwMjc4MTQwLCJpYXQiOjE3NzAyNzQ1NDB9.gVjrIncHUIswAGwQ4IHR-nbmfrLWzQFWPSl1M1wex_Vm9De8DOjAI8bM1v5mOGOfXil-Y5jp0bGGVp4rSlwwSw' 'http://localhost:8000/ui/grid/ti_summaries/asset_produces_2/manual__2026-01-31T05:00:20.973352+00:00'

Summary:
 Total:	0.6636 secs
 Slowest:	0.0171 secs
 Fastest:	0.0098 secs
 Average:	0.0132 secs
 Requests/sec:	301.3933
 

Response time histogram:
 0.010 [1]	|■
 0.011 [2]	|■
 0.011 [3]	|■■
 0.012 [17]	|■■■■■■■■■■■■■
 0.013 [46]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
 0.013 [54]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
 0.014 [37]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■
 0.015 [25]	|■■■■■■■■■■■■■■■■■■■
 0.016 [8]	|■■■■■■
 0.016 [5]	|■■■■
 0.017 [2]	|■


Latency distribution:
 10% in 0.0119 secs
 25% in 0.0124 secs
 50% in 0.0131 secs
 75% in 0.0139 secs
 90% in 0.0146 secs
 95% in 0.0154 secs
 99% in 0.0170 secs

Details (average, fastest, slowest):
 DNS+dialup:	0.0000 secs, 0.0098 secs, 0.0171 secs
 DNS-lookup:	0.0000 secs, 0.0000 secs, 0.0004 secs
 req write:	0.0000 secs, 0.0000 secs, 0.0002 secs
 resp wait:	0.0130 secs, 0.0095 secs, 0.0168 secs
 resp read:	0.0002 secs, 0.0000 secs, 0.0008 secs

Status code distribution:
 [200]	200 responses



hey -c 4 -H 0.08s user 0.05s system 19% cpu 0.672 total

After caching requires_access_dag, teams and serialized dag retrieval

time hey -c 4 -H 'Cookie: session=07b3594d-405a-489b-ae0c-02ee02c9bbdb.uq376NuwgAarpTdjToOPTJuO2a0; _token=eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxIiwianRpIjoiZWVhN2RhODBkZjFjNGRmMjkxZWRmMzE1YzBlNTZmNDQiLCJhdWQiOiJhcGFjaGUtYWlyZmxvdyIsIm5iZiI6MTc3MDI3NDU0MCwiZXhwIjoxNzcwMjc4MTQwLCJpYXQiOjE3NzAyNzQ1NDB9.gVjrIncHUIswAGwQ4IHR-nbmfrLWzQFWPSl1M1wex_Vm9De8DOjAI8bM1v5mOGOfXil-Y5jp0bGGVp4rSlwwSw' 'http://localhost:8000/ui/grid/ti_summaries/asset_produces_2/manual__2026-01-31T05:00:20.973352+00:00'

Summary:
 Total:	0.3493 secs
 Slowest:	0.0283 secs
 Fastest:	0.0047 secs
 Average:	0.0069 secs
 Requests/sec:	572.6096
 

Response time histogram:
 0.005 [1]	|
 0.007 [168]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
 0.009 [24]	|■■■■■■
 0.012 [3]	|■
 0.014 [0]	|
 0.017 [0]	|
 0.019 [0]	|
 0.021 [1]	|
 0.024 [0]	|
 0.026 [1]	|
 0.028 [2]	|


Latency distribution:
 10% in 0.0057 secs
 25% in 0.0061 secs
 50% in 0.0064 secs
 75% in 0.0068 secs
 90% in 0.0074 secs
 95% in 0.0084 secs
 99% in 0.0283 secs

Details (average, fastest, slowest):
 DNS+dialup:	0.0000 secs, 0.0047 secs, 0.0283 secs
 DNS-lookup:	0.0000 secs, 0.0000 secs, 0.0006 secs
 req write:	0.0002 secs, 0.0000 secs, 0.0205 secs
 resp wait:	0.0063 secs, 0.0043 secs, 0.0093 secs
 resp read:	0.0003 secs, 0.0000 secs, 0.0182 secs

Status code distribution:
 [200]	200 responses

hey -c 4 -H 0.07s user 0.05s system 31% cpu 0.354 total

What you think should happen instead?

No response

How to reproduce

1. Load the dag page with 10 dagruns.
2. Add a print statement to requires_access_dag function which is printed per API call though access to dag in one instance indicates this check is not required for future ti_summary API calls.

Operating System

Ubuntu 20.04

Versions of Apache Airflow Providers

No response

Deployment

Virtualenv installation

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[tirkarthi]

It will be also helpful to identify and remove redundant info present from different API calls to see if the API calls can be reduced or deferred when the component actually needs to be loaded.

cc: @vincbeck @pierrejeambrun @bbovenzi

[abhijeets25012-tech]

i would like to contribute

[vincbeck]

Since @tirkarthi you seem to have a fix? Could you maybe create a PR and let's discuss there?

[pierrejeambrun]

Yes I thought about this before and improving auth manager with some caching could be a great addition. We just need to think about the invalidation policy. (updating roles/users etc... should invalidate), but that would help a lot I think.

Ideally having a cache decorator that also adds .invalidate() .invalidate_all() to functions so we can easilly handle this.

[abhijeets25012-tech]

Hi @tirkarthi, I’d like to work on raising a PR to cache the requires_access_dag checks for better performance on the DAG page. I will ensure proper invalidation for user/role changes.

[tirkarthi]

@abhijeets25012-tech I have a patch that I tested internally. I will create it in a couple of days so that you can review and discuss if something differs from my approach. Thanks.