Docs: refresh JWT and security model for v3.2 with mermaid diagrams - #67435

Merged
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup
May 25, 2026
Merged

Docs: refresh JWT and security model for v3.2 with mermaid diagrams#67435
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup

Conversation

@potiuk

Copy link
Copy Markdown
Member

Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
and re-aligns prose with the current code.

What changed in the docs

airflow-core/docs/security/jwt_token_authentication.rst:

airflow-core/docs/security/security_model.rst:

Tooling change

Registers sphinxcontrib-mermaid>=1.0.0 as a new dependency in
devel-common[docs] and adds sphinxcontrib.mermaid to
BASIC_SPHINX_EXTENSIONS so every Airflow Sphinx build (core,
providers, chart, docker-stack) can use .. mermaid:: directives.
uv.lock is regenerated.

Verification

  • breeze build-docs --package-filter apache-airflow
    "Documentation build is successful", 0 build errors, 0 spelling errors.
  • All six mermaid containers render in the produced HTML.
  • prek run --from-ref upstream/main --stage pre-commit and
    --stage manual both pass.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

@potiuk

Copy link
Copy Markdown
MemberAuthor

@potiuk

Copy link
Copy Markdown
MemberAuthor

Also cc: @vatsrahul1001 -> we should include it in rc2

@potiuk

potiuk commented May 24, 2026

Copy link
Copy Markdown
MemberAuthor

Below are the six mermaid diagrams introduced in this PR (updated for higher contrast and an easier-to-read credential matrix), rendered inline via GitHub's native mermaid support. They are identical to what breeze build-docs produces in the published HTML.


jwt_token_authentication.rst — overview of components and flows

flowchart LR
subgraph Clients
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph Internal["Internal Airflow components"]
WORKER[Worker / Task]
DFP[Dag File Processor]
TRG[Triggerer]
end
APISVR[API Server]
EXECAPI[Execution API]
UI -->|JWT cookie / Bearer| APISVR
CLI -->|Bearer| APISVR
EXT -->|Bearer| APISVR
WORKER -->|Bearer<br/>workload &rarr; execution| EXECAPI
DFP -. in-process<br/>JWT bypassed .-> EXECAPI
TRG -. in-process<br/>JWT bypassed .-> EXECAPI
classDef internal fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WORKER,DFP,TRG internal
Loading

jwt_token_authentication.rst — symmetric vs asymmetric signing

flowchart TB
subgraph Sym["Symmetric (HS512)"]
direction LR
S1[Scheduler / API Server]
S2[Shared secret<br/>jwt_secret]
S3[Token validator]
S1 -->|sign| S2 -->|same secret<br/>also validates| S3
end
subgraph Asym["Asymmetric (RS256 / EdDSA)"]
direction LR
A1[Scheduler / API Server]
A2[Private key<br/>jwt_private_key_path]
A3[Public key /<br/>JWKS endpoint]
A4[Token validator]
A1 -->|sign| A2
A2 -. derives or<br/>publishes .-> A3
A3 -->|verify only| A4
end
classDef secret fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pub fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class S2 secret
class A2 secret
class A3 pub
Loading

jwt_token_authentication.rst — two-token sequence (workload → execution)

sequenceDiagram
autonumber
participant SCH as Scheduler
participant EXE as Executor<br/>(Celery / K8s / Local)
participant WRK as Worker
participant API as Execution API
Note over SCH: Task ready to dispatch
SCH->>SCH: generate workload token<br/>scope=workload<br/>exp = task_queued_timeout
SCH->>EXE: workload JSON<br/>(includes token)
Note over EXE: Task waits in queue<br/>(can be minutes)
EXE->>WRK: dispatch (workload JSON)
WRK->>API: POST /run<br/>Bearer: workload token
Note over API: validates workload scope<br/>checks TI in QUEUED/RESTARTING<br/>409 if not
API-->>WRK: 200 OK<br/>Refreshed-API-Token: execution token<br/>(scope=execution, ~10 min)
WRK->>WRK: BearerAuth swaps to<br/>execution token
loop For all subsequent calls (heartbeats, XComs, ...)
WRK->>API: Bearer: execution token
alt token expiring (less than 20% left)
API-->>WRK: 200 OK<br/>Refreshed-API-Token: new execution token
WRK->>WRK: BearerAuth swaps again
end
end
Loading

jwt_token_authentication.rst — Execution API request-time validation pipeline

flowchart TD
REQ([Incoming request<br/>Authorization: Bearer ...])
REQ --> CACHE{Cached on<br/>request.scope?}
CACHE -->|yes| RET([Return cached TIToken])
CACHE -->|no| SIG[JWTValidator:<br/>verify signature]
SIG -->|fail| F1([403 Forbidden])
SIG -->|ok| STD[Verify exp / iat / nbf<br/>aud / iss]
STD -->|fail| F1
STD -->|ok| SCOPE[Default scope to<br/>'execution' if absent]
SCOPE --> SCHEMA[TIClaims:<br/>typed Pydantic schema]
SCHEMA -->|ValidationError| F1
SCHEMA -->|ok| TYP{require_auth:<br/>scope in<br/>route.allowed_token_types?}
TYP -->|no| F1
TYP -->|yes| SELF{ti:self scope<br/>declared?}
SELF -->|no| OK([Return TIToken])
SELF -->|yes| MATCH{token.sub ==<br/>task_instance_id?}
MATCH -->|no| F1
MATCH -->|yes| OK
classDef fail fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pass fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class F1 fail
class OK,RET pass
Loading

security_model.rst — component trust boundaries

flowchart LR
subgraph users["Users (untrusted by default)"]
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph dataplane["Worker plane (no metadata DB access)"]
WRK[Worker / Task]
end
subgraph controlplane["Control plane (metadata DB access)"]
APISVR[API Server]
SCH[Scheduler]
DFP[Dag File Processor]
TRG[Triggerer]
end
DB[(Metadata DB)]
UI -->|JWT| APISVR
CLI -->|JWT| APISVR
EXT -->|JWT| APISVR
WRK -->|JWT<br/>Execution API| APISVR
APISVR -. SQL .-> DB
SCH -. SQL .-> DB
DFP -. SQL .-> DB
TRG -. SQL .-> DB
DFP -. in-process<br/>JWT bypassed .-> APISVR
TRG -. in-process<br/>JWT bypassed .-> APISVR
classDef untrusted fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef trusted fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef data fill:#fff9c4,stroke:#f57f17,stroke-width:2px,color:#000
class UI,CLI,EXT,WRK untrusted
class APISVR,SCH,DFP,TRG trusted
class DB data
Loading

security_model.rst — credential-distribution (least → most privileged)

flowchart LR
subgraph WRK["Worker (least privileged)"]
direction TB
W1[Fernet key]
W2[Worker secrets backend credentials]
W3[Remote log handler kwargs]
end
subgraph TRG["Triggerer"]
direction TB
T1[DB connection]
T2[Fernet key]
T3[Non-worker secrets backend credentials]
T4[Remote log handler kwargs]
end
subgraph DFP["Dag File Processor"]
direction TB
D1[DB connection]
D2[Fernet key]
D3[Non-worker secrets backend credentials]
end
subgraph SCH["Scheduler"]
direction TB
S1[DB connection]
S2[JWT signing key]
S3[Fernet key]
S4[Non-worker secrets backend credentials]
S5[Remote log handler kwargs]
end
subgraph API["API Server (most privileged)"]
direction TB
A1[DB connection]
A2[JWT signing key]
A3[Fernet key]
end
classDef wrk fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef ctrl fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WRK wrk
class API,SCH,DFP,TRG ctrl
Loading

The doc also includes an explicit ✓/— table for true matrix lookup (which secret does which component need?) — see the rst source.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

Comment threadairflow-core/docs/security/jwt_token_authentication.rst Outdated
potiuk added 3 commits May 25, 2026 08:40
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of apache#67435.
@potiuk
potiukforce-pushed the docs/security-model-v3.2-catchup branch from e6a0a7c to 5c989d2CompareMay 25, 2026 06:40
@vatsrahul1001
vatsrahul1001 merged commit 0a506b1 into apache:mainMay 25, 2026
145 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-2-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-2-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 0a506b1 v3-2-test

This should apply the commit to the v3-2-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@potiuk@kaxil@vatsrahul1001
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Docs: refresh JWT and security model for v3.2 with mermaid diagrams - #67435

Merged
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup
May 25, 2026
Merged

Docs: refresh JWT and security model for v3.2 with mermaid diagrams#67435
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup

Conversation

@potiuk

Copy link
Copy Markdown
Member

Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
and re-aligns prose with the current code.

What changed in the docs

airflow-core/docs/security/jwt_token_authentication.rst:

airflow-core/docs/security/security_model.rst:

Tooling change

Registers sphinxcontrib-mermaid>=1.0.0 as a new dependency in
devel-common[docs] and adds sphinxcontrib.mermaid to
BASIC_SPHINX_EXTENSIONS so every Airflow Sphinx build (core,
providers, chart, docker-stack) can use .. mermaid:: directives.
uv.lock is regenerated.

Verification

  • breeze build-docs --package-filter apache-airflow
    "Documentation build is successful", 0 build errors, 0 spelling errors.
  • All six mermaid containers render in the produced HTML.
  • prek run --from-ref upstream/main --stage pre-commit and
    --stage manual both pass.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

@potiuk

Copy link
Copy Markdown
MemberAuthor

@potiuk

Copy link
Copy Markdown
MemberAuthor

Also cc: @vatsrahul1001 -> we should include it in rc2

@potiuk

potiuk commented May 24, 2026

Copy link
Copy Markdown
MemberAuthor

Below are the six mermaid diagrams introduced in this PR (updated for higher contrast and an easier-to-read credential matrix), rendered inline via GitHub's native mermaid support. They are identical to what breeze build-docs produces in the published HTML.


jwt_token_authentication.rst — overview of components and flows

flowchart LR
subgraph Clients
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph Internal["Internal Airflow components"]
WORKER[Worker / Task]
DFP[Dag File Processor]
TRG[Triggerer]
end
APISVR[API Server]
EXECAPI[Execution API]
UI -->|JWT cookie / Bearer| APISVR
CLI -->|Bearer| APISVR
EXT -->|Bearer| APISVR
WORKER -->|Bearer<br/>workload &rarr; execution| EXECAPI
DFP -. in-process<br/>JWT bypassed .-> EXECAPI
TRG -. in-process<br/>JWT bypassed .-> EXECAPI
classDef internal fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WORKER,DFP,TRG internal
Loading

jwt_token_authentication.rst — symmetric vs asymmetric signing

flowchart TB
subgraph Sym["Symmetric (HS512)"]
direction LR
S1[Scheduler / API Server]
S2[Shared secret<br/>jwt_secret]
S3[Token validator]
S1 -->|sign| S2 -->|same secret<br/>also validates| S3
end
subgraph Asym["Asymmetric (RS256 / EdDSA)"]
direction LR
A1[Scheduler / API Server]
A2[Private key<br/>jwt_private_key_path]
A3[Public key /<br/>JWKS endpoint]
A4[Token validator]
A1 -->|sign| A2
A2 -. derives or<br/>publishes .-> A3
A3 -->|verify only| A4
end
classDef secret fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pub fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class S2 secret
class A2 secret
class A3 pub
Loading

jwt_token_authentication.rst — two-token sequence (workload → execution)

sequenceDiagram
autonumber
participant SCH as Scheduler
participant EXE as Executor<br/>(Celery / K8s / Local)
participant WRK as Worker
participant API as Execution API
Note over SCH: Task ready to dispatch
SCH->>SCH: generate workload token<br/>scope=workload<br/>exp = task_queued_timeout
SCH->>EXE: workload JSON<br/>(includes token)
Note over EXE: Task waits in queue<br/>(can be minutes)
EXE->>WRK: dispatch (workload JSON)
WRK->>API: POST /run<br/>Bearer: workload token
Note over API: validates workload scope<br/>checks TI in QUEUED/RESTARTING<br/>409 if not
API-->>WRK: 200 OK<br/>Refreshed-API-Token: execution token<br/>(scope=execution, ~10 min)
WRK->>WRK: BearerAuth swaps to<br/>execution token
loop For all subsequent calls (heartbeats, XComs, ...)
WRK->>API: Bearer: execution token
alt token expiring (less than 20% left)
API-->>WRK: 200 OK<br/>Refreshed-API-Token: new execution token
WRK->>WRK: BearerAuth swaps again
end
end
Loading

jwt_token_authentication.rst — Execution API request-time validation pipeline

flowchart TD
REQ([Incoming request<br/>Authorization: Bearer ...])
REQ --> CACHE{Cached on<br/>request.scope?}
CACHE -->|yes| RET([Return cached TIToken])
CACHE -->|no| SIG[JWTValidator:<br/>verify signature]
SIG -->|fail| F1([403 Forbidden])
SIG -->|ok| STD[Verify exp / iat / nbf<br/>aud / iss]
STD -->|fail| F1
STD -->|ok| SCOPE[Default scope to<br/>'execution' if absent]
SCOPE --> SCHEMA[TIClaims:<br/>typed Pydantic schema]
SCHEMA -->|ValidationError| F1
SCHEMA -->|ok| TYP{require_auth:<br/>scope in<br/>route.allowed_token_types?}
TYP -->|no| F1
TYP -->|yes| SELF{ti:self scope<br/>declared?}
SELF -->|no| OK([Return TIToken])
SELF -->|yes| MATCH{token.sub ==<br/>task_instance_id?}
MATCH -->|no| F1
MATCH -->|yes| OK
classDef fail fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pass fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class F1 fail
class OK,RET pass
Loading

security_model.rst — component trust boundaries

flowchart LR
subgraph users["Users (untrusted by default)"]
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph dataplane["Worker plane (no metadata DB access)"]
WRK[Worker / Task]
end
subgraph controlplane["Control plane (metadata DB access)"]
APISVR[API Server]
SCH[Scheduler]
DFP[Dag File Processor]
TRG[Triggerer]
end
DB[(Metadata DB)]
UI -->|JWT| APISVR
CLI -->|JWT| APISVR
EXT -->|JWT| APISVR
WRK -->|JWT<br/>Execution API| APISVR
APISVR -. SQL .-> DB
SCH -. SQL .-> DB
DFP -. SQL .-> DB
TRG -. SQL .-> DB
DFP -. in-process<br/>JWT bypassed .-> APISVR
TRG -. in-process<br/>JWT bypassed .-> APISVR
classDef untrusted fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef trusted fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef data fill:#fff9c4,stroke:#f57f17,stroke-width:2px,color:#000
class UI,CLI,EXT,WRK untrusted
class APISVR,SCH,DFP,TRG trusted
class DB data
Loading

security_model.rst — credential-distribution (least → most privileged)

flowchart LR
subgraph WRK["Worker (least privileged)"]
direction TB
W1[Fernet key]
W2[Worker secrets backend credentials]
W3[Remote log handler kwargs]
end
subgraph TRG["Triggerer"]
direction TB
T1[DB connection]
T2[Fernet key]
T3[Non-worker secrets backend credentials]
T4[Remote log handler kwargs]
end
subgraph DFP["Dag File Processor"]
direction TB
D1[DB connection]
D2[Fernet key]
D3[Non-worker secrets backend credentials]
end
subgraph SCH["Scheduler"]
direction TB
S1[DB connection]
S2[JWT signing key]
S3[Fernet key]
S4[Non-worker secrets backend credentials]
S5[Remote log handler kwargs]
end
subgraph API["API Server (most privileged)"]
direction TB
A1[DB connection]
A2[JWT signing key]
A3[Fernet key]
end
classDef wrk fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef ctrl fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WRK wrk
class API,SCH,DFP,TRG ctrl
Loading

The doc also includes an explicit ✓/— table for true matrix lookup (which secret does which component need?) — see the rst source.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

Comment threadairflow-core/docs/security/jwt_token_authentication.rst Outdated
potiuk added 3 commits May 25, 2026 08:40
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of apache#67435.
@potiuk
potiukforce-pushed the docs/security-model-v3.2-catchup branch from e6a0a7c to 5c989d2CompareMay 25, 2026 06:40
@vatsrahul1001
vatsrahul1001 merged commit 0a506b1 into apache:mainMay 25, 2026
145 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-2-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-2-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 0a506b1 v3-2-test

This should apply the commit to the v3-2-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@potiuk@kaxil@vatsrahul1001
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Docs: refresh JWT and security model for v3.2 with mermaid diagrams - #67435

Merged
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup
May 25, 2026
Merged

Docs: refresh JWT and security model for v3.2 with mermaid diagrams#67435
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup

Conversation

@potiuk

Copy link
Copy Markdown
Member

Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
and re-aligns prose with the current code.

What changed in the docs

airflow-core/docs/security/jwt_token_authentication.rst:

airflow-core/docs/security/security_model.rst:

Tooling change

Registers sphinxcontrib-mermaid>=1.0.0 as a new dependency in
devel-common[docs] and adds sphinxcontrib.mermaid to
BASIC_SPHINX_EXTENSIONS so every Airflow Sphinx build (core,
providers, chart, docker-stack) can use .. mermaid:: directives.
uv.lock is regenerated.

Verification

  • breeze build-docs --package-filter apache-airflow
    "Documentation build is successful", 0 build errors, 0 spelling errors.
  • All six mermaid containers render in the produced HTML.
  • prek run --from-ref upstream/main --stage pre-commit and
    --stage manual both pass.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

@potiuk

Copy link
Copy Markdown
MemberAuthor

@potiuk

Copy link
Copy Markdown
MemberAuthor

Also cc: @vatsrahul1001 -> we should include it in rc2

@potiuk

potiuk commented May 24, 2026

Copy link
Copy Markdown
MemberAuthor

Below are the six mermaid diagrams introduced in this PR (updated for higher contrast and an easier-to-read credential matrix), rendered inline via GitHub's native mermaid support. They are identical to what breeze build-docs produces in the published HTML.


jwt_token_authentication.rst — overview of components and flows

flowchart LR
subgraph Clients
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph Internal["Internal Airflow components"]
WORKER[Worker / Task]
DFP[Dag File Processor]
TRG[Triggerer]
end
APISVR[API Server]
EXECAPI[Execution API]
UI -->|JWT cookie / Bearer| APISVR
CLI -->|Bearer| APISVR
EXT -->|Bearer| APISVR
WORKER -->|Bearer<br/>workload &rarr; execution| EXECAPI
DFP -. in-process<br/>JWT bypassed .-> EXECAPI
TRG -. in-process<br/>JWT bypassed .-> EXECAPI
classDef internal fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WORKER,DFP,TRG internal
Loading

jwt_token_authentication.rst — symmetric vs asymmetric signing

flowchart TB
subgraph Sym["Symmetric (HS512)"]
direction LR
S1[Scheduler / API Server]
S2[Shared secret<br/>jwt_secret]
S3[Token validator]
S1 -->|sign| S2 -->|same secret<br/>also validates| S3
end
subgraph Asym["Asymmetric (RS256 / EdDSA)"]
direction LR
A1[Scheduler / API Server]
A2[Private key<br/>jwt_private_key_path]
A3[Public key /<br/>JWKS endpoint]
A4[Token validator]
A1 -->|sign| A2
A2 -. derives or<br/>publishes .-> A3
A3 -->|verify only| A4
end
classDef secret fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pub fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class S2 secret
class A2 secret
class A3 pub
Loading

jwt_token_authentication.rst — two-token sequence (workload → execution)

sequenceDiagram
autonumber
participant SCH as Scheduler
participant EXE as Executor<br/>(Celery / K8s / Local)
participant WRK as Worker
participant API as Execution API
Note over SCH: Task ready to dispatch
SCH->>SCH: generate workload token<br/>scope=workload<br/>exp = task_queued_timeout
SCH->>EXE: workload JSON<br/>(includes token)
Note over EXE: Task waits in queue<br/>(can be minutes)
EXE->>WRK: dispatch (workload JSON)
WRK->>API: POST /run<br/>Bearer: workload token
Note over API: validates workload scope<br/>checks TI in QUEUED/RESTARTING<br/>409 if not
API-->>WRK: 200 OK<br/>Refreshed-API-Token: execution token<br/>(scope=execution, ~10 min)
WRK->>WRK: BearerAuth swaps to<br/>execution token
loop For all subsequent calls (heartbeats, XComs, ...)
WRK->>API: Bearer: execution token
alt token expiring (less than 20% left)
API-->>WRK: 200 OK<br/>Refreshed-API-Token: new execution token
WRK->>WRK: BearerAuth swaps again
end
end
Loading

jwt_token_authentication.rst — Execution API request-time validation pipeline

flowchart TD
REQ([Incoming request<br/>Authorization: Bearer ...])
REQ --> CACHE{Cached on<br/>request.scope?}
CACHE -->|yes| RET([Return cached TIToken])
CACHE -->|no| SIG[JWTValidator:<br/>verify signature]
SIG -->|fail| F1([403 Forbidden])
SIG -->|ok| STD[Verify exp / iat / nbf<br/>aud / iss]
STD -->|fail| F1
STD -->|ok| SCOPE[Default scope to<br/>'execution' if absent]
SCOPE --> SCHEMA[TIClaims:<br/>typed Pydantic schema]
SCHEMA -->|ValidationError| F1
SCHEMA -->|ok| TYP{require_auth:<br/>scope in<br/>route.allowed_token_types?}
TYP -->|no| F1
TYP -->|yes| SELF{ti:self scope<br/>declared?}
SELF -->|no| OK([Return TIToken])
SELF -->|yes| MATCH{token.sub ==<br/>task_instance_id?}
MATCH -->|no| F1
MATCH -->|yes| OK
classDef fail fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pass fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class F1 fail
class OK,RET pass
Loading

security_model.rst — component trust boundaries

flowchart LR
subgraph users["Users (untrusted by default)"]
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph dataplane["Worker plane (no metadata DB access)"]
WRK[Worker / Task]
end
subgraph controlplane["Control plane (metadata DB access)"]
APISVR[API Server]
SCH[Scheduler]
DFP[Dag File Processor]
TRG[Triggerer]
end
DB[(Metadata DB)]
UI -->|JWT| APISVR
CLI -->|JWT| APISVR
EXT -->|JWT| APISVR
WRK -->|JWT<br/>Execution API| APISVR
APISVR -. SQL .-> DB
SCH -. SQL .-> DB
DFP -. SQL .-> DB
TRG -. SQL .-> DB
DFP -. in-process<br/>JWT bypassed .-> APISVR
TRG -. in-process<br/>JWT bypassed .-> APISVR
classDef untrusted fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef trusted fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef data fill:#fff9c4,stroke:#f57f17,stroke-width:2px,color:#000
class UI,CLI,EXT,WRK untrusted
class APISVR,SCH,DFP,TRG trusted
class DB data
Loading

security_model.rst — credential-distribution (least → most privileged)

flowchart LR
subgraph WRK["Worker (least privileged)"]
direction TB
W1[Fernet key]
W2[Worker secrets backend credentials]
W3[Remote log handler kwargs]
end
subgraph TRG["Triggerer"]
direction TB
T1[DB connection]
T2[Fernet key]
T3[Non-worker secrets backend credentials]
T4[Remote log handler kwargs]
end
subgraph DFP["Dag File Processor"]
direction TB
D1[DB connection]
D2[Fernet key]
D3[Non-worker secrets backend credentials]
end
subgraph SCH["Scheduler"]
direction TB
S1[DB connection]
S2[JWT signing key]
S3[Fernet key]
S4[Non-worker secrets backend credentials]
S5[Remote log handler kwargs]
end
subgraph API["API Server (most privileged)"]
direction TB
A1[DB connection]
A2[JWT signing key]
A3[Fernet key]
end
classDef wrk fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef ctrl fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WRK wrk
class API,SCH,DFP,TRG ctrl
Loading

The doc also includes an explicit ✓/— table for true matrix lookup (which secret does which component need?) — see the rst source.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

Comment threadairflow-core/docs/security/jwt_token_authentication.rst Outdated
potiuk added 3 commits May 25, 2026 08:40
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of apache#67435.
@potiuk
potiukforce-pushed the docs/security-model-v3.2-catchup branch from e6a0a7c to 5c989d2CompareMay 25, 2026 06:40
@vatsrahul1001
vatsrahul1001 merged commit 0a506b1 into apache:mainMay 25, 2026
145 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-2-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-2-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 0a506b1 v3-2-test

This should apply the commit to the v3-2-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@potiuk@kaxil@vatsrahul1001
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Docs: refresh JWT and security model for v3.2 with mermaid diagrams - #67435

Merged
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup
May 25, 2026
Merged

Docs: refresh JWT and security model for v3.2 with mermaid diagrams#67435
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup

Conversation

@potiuk

Copy link
Copy Markdown
Member

Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
and re-aligns prose with the current code.

What changed in the docs

airflow-core/docs/security/jwt_token_authentication.rst:

airflow-core/docs/security/security_model.rst:

Tooling change

Registers sphinxcontrib-mermaid>=1.0.0 as a new dependency in
devel-common[docs] and adds sphinxcontrib.mermaid to
BASIC_SPHINX_EXTENSIONS so every Airflow Sphinx build (core,
providers, chart, docker-stack) can use .. mermaid:: directives.
uv.lock is regenerated.

Verification

  • breeze build-docs --package-filter apache-airflow
    "Documentation build is successful", 0 build errors, 0 spelling errors.
  • All six mermaid containers render in the produced HTML.
  • prek run --from-ref upstream/main --stage pre-commit and
    --stage manual both pass.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

@potiuk

Copy link
Copy Markdown
MemberAuthor

@potiuk

Copy link
Copy Markdown
MemberAuthor

Also cc: @vatsrahul1001 -> we should include it in rc2

@potiuk

potiuk commented May 24, 2026

Copy link
Copy Markdown
MemberAuthor

Below are the six mermaid diagrams introduced in this PR (updated for higher contrast and an easier-to-read credential matrix), rendered inline via GitHub's native mermaid support. They are identical to what breeze build-docs produces in the published HTML.


jwt_token_authentication.rst — overview of components and flows

flowchart LR
subgraph Clients
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph Internal["Internal Airflow components"]
WORKER[Worker / Task]
DFP[Dag File Processor]
TRG[Triggerer]
end
APISVR[API Server]
EXECAPI[Execution API]
UI -->|JWT cookie / Bearer| APISVR
CLI -->|Bearer| APISVR
EXT -->|Bearer| APISVR
WORKER -->|Bearer<br/>workload &rarr; execution| EXECAPI
DFP -. in-process<br/>JWT bypassed .-> EXECAPI
TRG -. in-process<br/>JWT bypassed .-> EXECAPI
classDef internal fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WORKER,DFP,TRG internal
Loading

jwt_token_authentication.rst — symmetric vs asymmetric signing

flowchart TB
subgraph Sym["Symmetric (HS512)"]
direction LR
S1[Scheduler / API Server]
S2[Shared secret<br/>jwt_secret]
S3[Token validator]
S1 -->|sign| S2 -->|same secret<br/>also validates| S3
end
subgraph Asym["Asymmetric (RS256 / EdDSA)"]
direction LR
A1[Scheduler / API Server]
A2[Private key<br/>jwt_private_key_path]
A3[Public key /<br/>JWKS endpoint]
A4[Token validator]
A1 -->|sign| A2
A2 -. derives or<br/>publishes .-> A3
A3 -->|verify only| A4
end
classDef secret fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pub fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class S2 secret
class A2 secret
class A3 pub
Loading

jwt_token_authentication.rst — two-token sequence (workload → execution)

sequenceDiagram
autonumber
participant SCH as Scheduler
participant EXE as Executor<br/>(Celery / K8s / Local)
participant WRK as Worker
participant API as Execution API
Note over SCH: Task ready to dispatch
SCH->>SCH: generate workload token<br/>scope=workload<br/>exp = task_queued_timeout
SCH->>EXE: workload JSON<br/>(includes token)
Note over EXE: Task waits in queue<br/>(can be minutes)
EXE->>WRK: dispatch (workload JSON)
WRK->>API: POST /run<br/>Bearer: workload token
Note over API: validates workload scope<br/>checks TI in QUEUED/RESTARTING<br/>409 if not
API-->>WRK: 200 OK<br/>Refreshed-API-Token: execution token<br/>(scope=execution, ~10 min)
WRK->>WRK: BearerAuth swaps to<br/>execution token
loop For all subsequent calls (heartbeats, XComs, ...)
WRK->>API: Bearer: execution token
alt token expiring (less than 20% left)
API-->>WRK: 200 OK<br/>Refreshed-API-Token: new execution token
WRK->>WRK: BearerAuth swaps again
end
end
Loading

jwt_token_authentication.rst — Execution API request-time validation pipeline

flowchart TD
REQ([Incoming request<br/>Authorization: Bearer ...])
REQ --> CACHE{Cached on<br/>request.scope?}
CACHE -->|yes| RET([Return cached TIToken])
CACHE -->|no| SIG[JWTValidator:<br/>verify signature]
SIG -->|fail| F1([403 Forbidden])
SIG -->|ok| STD[Verify exp / iat / nbf<br/>aud / iss]
STD -->|fail| F1
STD -->|ok| SCOPE[Default scope to<br/>'execution' if absent]
SCOPE --> SCHEMA[TIClaims:<br/>typed Pydantic schema]
SCHEMA -->|ValidationError| F1
SCHEMA -->|ok| TYP{require_auth:<br/>scope in<br/>route.allowed_token_types?}
TYP -->|no| F1
TYP -->|yes| SELF{ti:self scope<br/>declared?}
SELF -->|no| OK([Return TIToken])
SELF -->|yes| MATCH{token.sub ==<br/>task_instance_id?}
MATCH -->|no| F1
MATCH -->|yes| OK
classDef fail fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pass fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class F1 fail
class OK,RET pass
Loading

security_model.rst — component trust boundaries

flowchart LR
subgraph users["Users (untrusted by default)"]
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph dataplane["Worker plane (no metadata DB access)"]
WRK[Worker / Task]
end
subgraph controlplane["Control plane (metadata DB access)"]
APISVR[API Server]
SCH[Scheduler]
DFP[Dag File Processor]
TRG[Triggerer]
end
DB[(Metadata DB)]
UI -->|JWT| APISVR
CLI -->|JWT| APISVR
EXT -->|JWT| APISVR
WRK -->|JWT<br/>Execution API| APISVR
APISVR -. SQL .-> DB
SCH -. SQL .-> DB
DFP -. SQL .-> DB
TRG -. SQL .-> DB
DFP -. in-process<br/>JWT bypassed .-> APISVR
TRG -. in-process<br/>JWT bypassed .-> APISVR
classDef untrusted fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef trusted fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef data fill:#fff9c4,stroke:#f57f17,stroke-width:2px,color:#000
class UI,CLI,EXT,WRK untrusted
class APISVR,SCH,DFP,TRG trusted
class DB data
Loading

security_model.rst — credential-distribution (least → most privileged)

flowchart LR
subgraph WRK["Worker (least privileged)"]
direction TB
W1[Fernet key]
W2[Worker secrets backend credentials]
W3[Remote log handler kwargs]
end
subgraph TRG["Triggerer"]
direction TB
T1[DB connection]
T2[Fernet key]
T3[Non-worker secrets backend credentials]
T4[Remote log handler kwargs]
end
subgraph DFP["Dag File Processor"]
direction TB
D1[DB connection]
D2[Fernet key]
D3[Non-worker secrets backend credentials]
end
subgraph SCH["Scheduler"]
direction TB
S1[DB connection]
S2[JWT signing key]
S3[Fernet key]
S4[Non-worker secrets backend credentials]
S5[Remote log handler kwargs]
end
subgraph API["API Server (most privileged)"]
direction TB
A1[DB connection]
A2[JWT signing key]
A3[Fernet key]
end
classDef wrk fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef ctrl fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WRK wrk
class API,SCH,DFP,TRG ctrl
Loading

The doc also includes an explicit ✓/— table for true matrix lookup (which secret does which component need?) — see the rst source.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

Comment threadairflow-core/docs/security/jwt_token_authentication.rst Outdated
potiuk added 3 commits May 25, 2026 08:40
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of apache#67435.
@potiuk
potiukforce-pushed the docs/security-model-v3.2-catchup branch from e6a0a7c to 5c989d2CompareMay 25, 2026 06:40
@vatsrahul1001
vatsrahul1001 merged commit 0a506b1 into apache:mainMay 25, 2026
145 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-2-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-2-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 0a506b1 v3-2-test

This should apply the commit to the v3-2-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@potiuk@kaxil@vatsrahul1001
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Docs: refresh JWT and security model for v3.2 with mermaid diagrams - #67435

Merged
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup
May 25, 2026
Merged

Docs: refresh JWT and security model for v3.2 with mermaid diagrams#67435
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup

Conversation

@potiuk

Copy link
Copy Markdown
Member

Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
and re-aligns prose with the current code.

What changed in the docs

airflow-core/docs/security/jwt_token_authentication.rst:

airflow-core/docs/security/security_model.rst:

Tooling change

Registers sphinxcontrib-mermaid>=1.0.0 as a new dependency in
devel-common[docs] and adds sphinxcontrib.mermaid to
BASIC_SPHINX_EXTENSIONS so every Airflow Sphinx build (core,
providers, chart, docker-stack) can use .. mermaid:: directives.
uv.lock is regenerated.

Verification

  • breeze build-docs --package-filter apache-airflow
    "Documentation build is successful", 0 build errors, 0 spelling errors.
  • All six mermaid containers render in the produced HTML.
  • prek run --from-ref upstream/main --stage pre-commit and
    --stage manual both pass.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

@potiuk

Copy link
Copy Markdown
MemberAuthor

@potiuk

Copy link
Copy Markdown
MemberAuthor

Also cc: @vatsrahul1001 -> we should include it in rc2

@potiuk

potiuk commented May 24, 2026

Copy link
Copy Markdown
MemberAuthor

Below are the six mermaid diagrams introduced in this PR (updated for higher contrast and an easier-to-read credential matrix), rendered inline via GitHub's native mermaid support. They are identical to what breeze build-docs produces in the published HTML.


jwt_token_authentication.rst — overview of components and flows

flowchart LR
subgraph Clients
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph Internal["Internal Airflow components"]
WORKER[Worker / Task]
DFP[Dag File Processor]
TRG[Triggerer]
end
APISVR[API Server]
EXECAPI[Execution API]
UI -->|JWT cookie / Bearer| APISVR
CLI -->|Bearer| APISVR
EXT -->|Bearer| APISVR
WORKER -->|Bearer<br/>workload &rarr; execution| EXECAPI
DFP -. in-process<br/>JWT bypassed .-> EXECAPI
TRG -. in-process<br/>JWT bypassed .-> EXECAPI
classDef internal fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WORKER,DFP,TRG internal
Loading

jwt_token_authentication.rst — symmetric vs asymmetric signing

flowchart TB
subgraph Sym["Symmetric (HS512)"]
direction LR
S1[Scheduler / API Server]
S2[Shared secret<br/>jwt_secret]
S3[Token validator]
S1 -->|sign| S2 -->|same secret<br/>also validates| S3
end
subgraph Asym["Asymmetric (RS256 / EdDSA)"]
direction LR
A1[Scheduler / API Server]
A2[Private key<br/>jwt_private_key_path]
A3[Public key /<br/>JWKS endpoint]
A4[Token validator]
A1 -->|sign| A2
A2 -. derives or<br/>publishes .-> A3
A3 -->|verify only| A4
end
classDef secret fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pub fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class S2 secret
class A2 secret
class A3 pub
Loading

jwt_token_authentication.rst — two-token sequence (workload → execution)

sequenceDiagram
autonumber
participant SCH as Scheduler
participant EXE as Executor<br/>(Celery / K8s / Local)
participant WRK as Worker
participant API as Execution API
Note over SCH: Task ready to dispatch
SCH->>SCH: generate workload token<br/>scope=workload<br/>exp = task_queued_timeout
SCH->>EXE: workload JSON<br/>(includes token)
Note over EXE: Task waits in queue<br/>(can be minutes)
EXE->>WRK: dispatch (workload JSON)
WRK->>API: POST /run<br/>Bearer: workload token
Note over API: validates workload scope<br/>checks TI in QUEUED/RESTARTING<br/>409 if not
API-->>WRK: 200 OK<br/>Refreshed-API-Token: execution token<br/>(scope=execution, ~10 min)
WRK->>WRK: BearerAuth swaps to<br/>execution token
loop For all subsequent calls (heartbeats, XComs, ...)
WRK->>API: Bearer: execution token
alt token expiring (less than 20% left)
API-->>WRK: 200 OK<br/>Refreshed-API-Token: new execution token
WRK->>WRK: BearerAuth swaps again
end
end
Loading

jwt_token_authentication.rst — Execution API request-time validation pipeline

flowchart TD
REQ([Incoming request<br/>Authorization: Bearer ...])
REQ --> CACHE{Cached on<br/>request.scope?}
CACHE -->|yes| RET([Return cached TIToken])
CACHE -->|no| SIG[JWTValidator:<br/>verify signature]
SIG -->|fail| F1([403 Forbidden])
SIG -->|ok| STD[Verify exp / iat / nbf<br/>aud / iss]
STD -->|fail| F1
STD -->|ok| SCOPE[Default scope to<br/>'execution' if absent]
SCOPE --> SCHEMA[TIClaims:<br/>typed Pydantic schema]
SCHEMA -->|ValidationError| F1
SCHEMA -->|ok| TYP{require_auth:<br/>scope in<br/>route.allowed_token_types?}
TYP -->|no| F1
TYP -->|yes| SELF{ti:self scope<br/>declared?}
SELF -->|no| OK([Return TIToken])
SELF -->|yes| MATCH{token.sub ==<br/>task_instance_id?}
MATCH -->|no| F1
MATCH -->|yes| OK
classDef fail fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pass fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class F1 fail
class OK,RET pass
Loading

security_model.rst — component trust boundaries

flowchart LR
subgraph users["Users (untrusted by default)"]
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph dataplane["Worker plane (no metadata DB access)"]
WRK[Worker / Task]
end
subgraph controlplane["Control plane (metadata DB access)"]
APISVR[API Server]
SCH[Scheduler]
DFP[Dag File Processor]
TRG[Triggerer]
end
DB[(Metadata DB)]
UI -->|JWT| APISVR
CLI -->|JWT| APISVR
EXT -->|JWT| APISVR
WRK -->|JWT<br/>Execution API| APISVR
APISVR -. SQL .-> DB
SCH -. SQL .-> DB
DFP -. SQL .-> DB
TRG -. SQL .-> DB
DFP -. in-process<br/>JWT bypassed .-> APISVR
TRG -. in-process<br/>JWT bypassed .-> APISVR
classDef untrusted fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef trusted fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef data fill:#fff9c4,stroke:#f57f17,stroke-width:2px,color:#000
class UI,CLI,EXT,WRK untrusted
class APISVR,SCH,DFP,TRG trusted
class DB data
Loading

security_model.rst — credential-distribution (least → most privileged)

flowchart LR
subgraph WRK["Worker (least privileged)"]
direction TB
W1[Fernet key]
W2[Worker secrets backend credentials]
W3[Remote log handler kwargs]
end
subgraph TRG["Triggerer"]
direction TB
T1[DB connection]
T2[Fernet key]
T3[Non-worker secrets backend credentials]
T4[Remote log handler kwargs]
end
subgraph DFP["Dag File Processor"]
direction TB
D1[DB connection]
D2[Fernet key]
D3[Non-worker secrets backend credentials]
end
subgraph SCH["Scheduler"]
direction TB
S1[DB connection]
S2[JWT signing key]
S3[Fernet key]
S4[Non-worker secrets backend credentials]
S5[Remote log handler kwargs]
end
subgraph API["API Server (most privileged)"]
direction TB
A1[DB connection]
A2[JWT signing key]
A3[Fernet key]
end
classDef wrk fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef ctrl fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WRK wrk
class API,SCH,DFP,TRG ctrl
Loading

The doc also includes an explicit ✓/— table for true matrix lookup (which secret does which component need?) — see the rst source.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

Comment threadairflow-core/docs/security/jwt_token_authentication.rst Outdated
potiuk added 3 commits May 25, 2026 08:40
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of apache#67435.
@potiuk
potiukforce-pushed the docs/security-model-v3.2-catchup branch from e6a0a7c to 5c989d2CompareMay 25, 2026 06:40
@vatsrahul1001
vatsrahul1001 merged commit 0a506b1 into apache:mainMay 25, 2026
145 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-2-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-2-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 0a506b1 v3-2-test

This should apply the commit to the v3-2-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@potiuk@kaxil@vatsrahul1001
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Docs: refresh JWT and security model for v3.2 with mermaid diagrams - #67435

Merged
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup
May 25, 2026
Merged

Docs: refresh JWT and security model for v3.2 with mermaid diagrams#67435
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup

Conversation

@potiuk

Copy link
Copy Markdown
Member

Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
and re-aligns prose with the current code.

What changed in the docs

airflow-core/docs/security/jwt_token_authentication.rst:

airflow-core/docs/security/security_model.rst:

Tooling change

Registers sphinxcontrib-mermaid>=1.0.0 as a new dependency in
devel-common[docs] and adds sphinxcontrib.mermaid to
BASIC_SPHINX_EXTENSIONS so every Airflow Sphinx build (core,
providers, chart, docker-stack) can use .. mermaid:: directives.
uv.lock is regenerated.

Verification

  • breeze build-docs --package-filter apache-airflow
    "Documentation build is successful", 0 build errors, 0 spelling errors.
  • All six mermaid containers render in the produced HTML.
  • prek run --from-ref upstream/main --stage pre-commit and
    --stage manual both pass.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

@potiuk

Copy link
Copy Markdown
MemberAuthor

@potiuk

Copy link
Copy Markdown
MemberAuthor

Also cc: @vatsrahul1001 -> we should include it in rc2

@potiuk

potiuk commented May 24, 2026

Copy link
Copy Markdown
MemberAuthor

Below are the six mermaid diagrams introduced in this PR (updated for higher contrast and an easier-to-read credential matrix), rendered inline via GitHub's native mermaid support. They are identical to what breeze build-docs produces in the published HTML.


jwt_token_authentication.rst — overview of components and flows

flowchart LR
subgraph Clients
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph Internal["Internal Airflow components"]
WORKER[Worker / Task]
DFP[Dag File Processor]
TRG[Triggerer]
end
APISVR[API Server]
EXECAPI[Execution API]
UI -->|JWT cookie / Bearer| APISVR
CLI -->|Bearer| APISVR
EXT -->|Bearer| APISVR
WORKER -->|Bearer<br/>workload &rarr; execution| EXECAPI
DFP -. in-process<br/>JWT bypassed .-> EXECAPI
TRG -. in-process<br/>JWT bypassed .-> EXECAPI
classDef internal fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WORKER,DFP,TRG internal
Loading

jwt_token_authentication.rst — symmetric vs asymmetric signing

flowchart TB
subgraph Sym["Symmetric (HS512)"]
direction LR
S1[Scheduler / API Server]
S2[Shared secret<br/>jwt_secret]
S3[Token validator]
S1 -->|sign| S2 -->|same secret<br/>also validates| S3
end
subgraph Asym["Asymmetric (RS256 / EdDSA)"]
direction LR
A1[Scheduler / API Server]
A2[Private key<br/>jwt_private_key_path]
A3[Public key /<br/>JWKS endpoint]
A4[Token validator]
A1 -->|sign| A2
A2 -. derives or<br/>publishes .-> A3
A3 -->|verify only| A4
end
classDef secret fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pub fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class S2 secret
class A2 secret
class A3 pub
Loading

jwt_token_authentication.rst — two-token sequence (workload → execution)

sequenceDiagram
autonumber
participant SCH as Scheduler
participant EXE as Executor<br/>(Celery / K8s / Local)
participant WRK as Worker
participant API as Execution API
Note over SCH: Task ready to dispatch
SCH->>SCH: generate workload token<br/>scope=workload<br/>exp = task_queued_timeout
SCH->>EXE: workload JSON<br/>(includes token)
Note over EXE: Task waits in queue<br/>(can be minutes)
EXE->>WRK: dispatch (workload JSON)
WRK->>API: POST /run<br/>Bearer: workload token
Note over API: validates workload scope<br/>checks TI in QUEUED/RESTARTING<br/>409 if not
API-->>WRK: 200 OK<br/>Refreshed-API-Token: execution token<br/>(scope=execution, ~10 min)
WRK->>WRK: BearerAuth swaps to<br/>execution token
loop For all subsequent calls (heartbeats, XComs, ...)
WRK->>API: Bearer: execution token
alt token expiring (less than 20% left)
API-->>WRK: 200 OK<br/>Refreshed-API-Token: new execution token
WRK->>WRK: BearerAuth swaps again
end
end
Loading

jwt_token_authentication.rst — Execution API request-time validation pipeline

flowchart TD
REQ([Incoming request<br/>Authorization: Bearer ...])
REQ --> CACHE{Cached on<br/>request.scope?}
CACHE -->|yes| RET([Return cached TIToken])
CACHE -->|no| SIG[JWTValidator:<br/>verify signature]
SIG -->|fail| F1([403 Forbidden])
SIG -->|ok| STD[Verify exp / iat / nbf<br/>aud / iss]
STD -->|fail| F1
STD -->|ok| SCOPE[Default scope to<br/>'execution' if absent]
SCOPE --> SCHEMA[TIClaims:<br/>typed Pydantic schema]
SCHEMA -->|ValidationError| F1
SCHEMA -->|ok| TYP{require_auth:<br/>scope in<br/>route.allowed_token_types?}
TYP -->|no| F1
TYP -->|yes| SELF{ti:self scope<br/>declared?}
SELF -->|no| OK([Return TIToken])
SELF -->|yes| MATCH{token.sub ==<br/>task_instance_id?}
MATCH -->|no| F1
MATCH -->|yes| OK
classDef fail fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pass fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class F1 fail
class OK,RET pass
Loading

security_model.rst — component trust boundaries

flowchart LR
subgraph users["Users (untrusted by default)"]
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph dataplane["Worker plane (no metadata DB access)"]
WRK[Worker / Task]
end
subgraph controlplane["Control plane (metadata DB access)"]
APISVR[API Server]
SCH[Scheduler]
DFP[Dag File Processor]
TRG[Triggerer]
end
DB[(Metadata DB)]
UI -->|JWT| APISVR
CLI -->|JWT| APISVR
EXT -->|JWT| APISVR
WRK -->|JWT<br/>Execution API| APISVR
APISVR -. SQL .-> DB
SCH -. SQL .-> DB
DFP -. SQL .-> DB
TRG -. SQL .-> DB
DFP -. in-process<br/>JWT bypassed .-> APISVR
TRG -. in-process<br/>JWT bypassed .-> APISVR
classDef untrusted fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef trusted fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef data fill:#fff9c4,stroke:#f57f17,stroke-width:2px,color:#000
class UI,CLI,EXT,WRK untrusted
class APISVR,SCH,DFP,TRG trusted
class DB data
Loading

security_model.rst — credential-distribution (least → most privileged)

flowchart LR
subgraph WRK["Worker (least privileged)"]
direction TB
W1[Fernet key]
W2[Worker secrets backend credentials]
W3[Remote log handler kwargs]
end
subgraph TRG["Triggerer"]
direction TB
T1[DB connection]
T2[Fernet key]
T3[Non-worker secrets backend credentials]
T4[Remote log handler kwargs]
end
subgraph DFP["Dag File Processor"]
direction TB
D1[DB connection]
D2[Fernet key]
D3[Non-worker secrets backend credentials]
end
subgraph SCH["Scheduler"]
direction TB
S1[DB connection]
S2[JWT signing key]
S3[Fernet key]
S4[Non-worker secrets backend credentials]
S5[Remote log handler kwargs]
end
subgraph API["API Server (most privileged)"]
direction TB
A1[DB connection]
A2[JWT signing key]
A3[Fernet key]
end
classDef wrk fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef ctrl fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WRK wrk
class API,SCH,DFP,TRG ctrl
Loading

The doc also includes an explicit ✓/— table for true matrix lookup (which secret does which component need?) — see the rst source.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

Comment threadairflow-core/docs/security/jwt_token_authentication.rst Outdated
potiuk added 3 commits May 25, 2026 08:40
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of apache#67435.
@potiuk
potiukforce-pushed the docs/security-model-v3.2-catchup branch from e6a0a7c to 5c989d2CompareMay 25, 2026 06:40
@vatsrahul1001
vatsrahul1001 merged commit 0a506b1 into apache:mainMay 25, 2026
145 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-2-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-2-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 0a506b1 v3-2-test

This should apply the commit to the v3-2-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@potiuk@kaxil@vatsrahul1001
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Docs: refresh JWT and security model for v3.2 with mermaid diagrams - #67435

Merged
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup
May 25, 2026
Merged

Docs: refresh JWT and security model for v3.2 with mermaid diagrams#67435
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup

Conversation

@potiuk

Copy link
Copy Markdown
Member

Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
and re-aligns prose with the current code.

What changed in the docs

airflow-core/docs/security/jwt_token_authentication.rst:

airflow-core/docs/security/security_model.rst:

Tooling change

Registers sphinxcontrib-mermaid>=1.0.0 as a new dependency in
devel-common[docs] and adds sphinxcontrib.mermaid to
BASIC_SPHINX_EXTENSIONS so every Airflow Sphinx build (core,
providers, chart, docker-stack) can use .. mermaid:: directives.
uv.lock is regenerated.

Verification

  • breeze build-docs --package-filter apache-airflow
    "Documentation build is successful", 0 build errors, 0 spelling errors.
  • All six mermaid containers render in the produced HTML.
  • prek run --from-ref upstream/main --stage pre-commit and
    --stage manual both pass.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

@potiuk

Copy link
Copy Markdown
MemberAuthor

@potiuk

Copy link
Copy Markdown
MemberAuthor

Also cc: @vatsrahul1001 -> we should include it in rc2

@potiuk

potiuk commented May 24, 2026

Copy link
Copy Markdown
MemberAuthor

Below are the six mermaid diagrams introduced in this PR (updated for higher contrast and an easier-to-read credential matrix), rendered inline via GitHub's native mermaid support. They are identical to what breeze build-docs produces in the published HTML.


jwt_token_authentication.rst — overview of components and flows

flowchart LR
subgraph Clients
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph Internal["Internal Airflow components"]
WORKER[Worker / Task]
DFP[Dag File Processor]
TRG[Triggerer]
end
APISVR[API Server]
EXECAPI[Execution API]
UI -->|JWT cookie / Bearer| APISVR
CLI -->|Bearer| APISVR
EXT -->|Bearer| APISVR
WORKER -->|Bearer<br/>workload &rarr; execution| EXECAPI
DFP -. in-process<br/>JWT bypassed .-> EXECAPI
TRG -. in-process<br/>JWT bypassed .-> EXECAPI
classDef internal fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WORKER,DFP,TRG internal
Loading

jwt_token_authentication.rst — symmetric vs asymmetric signing

flowchart TB
subgraph Sym["Symmetric (HS512)"]
direction LR
S1[Scheduler / API Server]
S2[Shared secret<br/>jwt_secret]
S3[Token validator]
S1 -->|sign| S2 -->|same secret<br/>also validates| S3
end
subgraph Asym["Asymmetric (RS256 / EdDSA)"]
direction LR
A1[Scheduler / API Server]
A2[Private key<br/>jwt_private_key_path]
A3[Public key /<br/>JWKS endpoint]
A4[Token validator]
A1 -->|sign| A2
A2 -. derives or<br/>publishes .-> A3
A3 -->|verify only| A4
end
classDef secret fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pub fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class S2 secret
class A2 secret
class A3 pub
Loading

jwt_token_authentication.rst — two-token sequence (workload → execution)

sequenceDiagram
autonumber
participant SCH as Scheduler
participant EXE as Executor<br/>(Celery / K8s / Local)
participant WRK as Worker
participant API as Execution API
Note over SCH: Task ready to dispatch
SCH->>SCH: generate workload token<br/>scope=workload<br/>exp = task_queued_timeout
SCH->>EXE: workload JSON<br/>(includes token)
Note over EXE: Task waits in queue<br/>(can be minutes)
EXE->>WRK: dispatch (workload JSON)
WRK->>API: POST /run<br/>Bearer: workload token
Note over API: validates workload scope<br/>checks TI in QUEUED/RESTARTING<br/>409 if not
API-->>WRK: 200 OK<br/>Refreshed-API-Token: execution token<br/>(scope=execution, ~10 min)
WRK->>WRK: BearerAuth swaps to<br/>execution token
loop For all subsequent calls (heartbeats, XComs, ...)
WRK->>API: Bearer: execution token
alt token expiring (less than 20% left)
API-->>WRK: 200 OK<br/>Refreshed-API-Token: new execution token
WRK->>WRK: BearerAuth swaps again
end
end
Loading

jwt_token_authentication.rst — Execution API request-time validation pipeline

flowchart TD
REQ([Incoming request<br/>Authorization: Bearer ...])
REQ --> CACHE{Cached on<br/>request.scope?}
CACHE -->|yes| RET([Return cached TIToken])
CACHE -->|no| SIG[JWTValidator:<br/>verify signature]
SIG -->|fail| F1([403 Forbidden])
SIG -->|ok| STD[Verify exp / iat / nbf<br/>aud / iss]
STD -->|fail| F1
STD -->|ok| SCOPE[Default scope to<br/>'execution' if absent]
SCOPE --> SCHEMA[TIClaims:<br/>typed Pydantic schema]
SCHEMA -->|ValidationError| F1
SCHEMA -->|ok| TYP{require_auth:<br/>scope in<br/>route.allowed_token_types?}
TYP -->|no| F1
TYP -->|yes| SELF{ti:self scope<br/>declared?}
SELF -->|no| OK([Return TIToken])
SELF -->|yes| MATCH{token.sub ==<br/>task_instance_id?}
MATCH -->|no| F1
MATCH -->|yes| OK
classDef fail fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pass fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class F1 fail
class OK,RET pass
Loading

security_model.rst — component trust boundaries

flowchart LR
subgraph users["Users (untrusted by default)"]
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph dataplane["Worker plane (no metadata DB access)"]
WRK[Worker / Task]
end
subgraph controlplane["Control plane (metadata DB access)"]
APISVR[API Server]
SCH[Scheduler]
DFP[Dag File Processor]
TRG[Triggerer]
end
DB[(Metadata DB)]
UI -->|JWT| APISVR
CLI -->|JWT| APISVR
EXT -->|JWT| APISVR
WRK -->|JWT<br/>Execution API| APISVR
APISVR -. SQL .-> DB
SCH -. SQL .-> DB
DFP -. SQL .-> DB
TRG -. SQL .-> DB
DFP -. in-process<br/>JWT bypassed .-> APISVR
TRG -. in-process<br/>JWT bypassed .-> APISVR
classDef untrusted fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef trusted fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef data fill:#fff9c4,stroke:#f57f17,stroke-width:2px,color:#000
class UI,CLI,EXT,WRK untrusted
class APISVR,SCH,DFP,TRG trusted
class DB data
Loading

security_model.rst — credential-distribution (least → most privileged)

flowchart LR
subgraph WRK["Worker (least privileged)"]
direction TB
W1[Fernet key]
W2[Worker secrets backend credentials]
W3[Remote log handler kwargs]
end
subgraph TRG["Triggerer"]
direction TB
T1[DB connection]
T2[Fernet key]
T3[Non-worker secrets backend credentials]
T4[Remote log handler kwargs]
end
subgraph DFP["Dag File Processor"]
direction TB
D1[DB connection]
D2[Fernet key]
D3[Non-worker secrets backend credentials]
end
subgraph SCH["Scheduler"]
direction TB
S1[DB connection]
S2[JWT signing key]
S3[Fernet key]
S4[Non-worker secrets backend credentials]
S5[Remote log handler kwargs]
end
subgraph API["API Server (most privileged)"]
direction TB
A1[DB connection]
A2[JWT signing key]
A3[Fernet key]
end
classDef wrk fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef ctrl fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WRK wrk
class API,SCH,DFP,TRG ctrl
Loading

The doc also includes an explicit ✓/— table for true matrix lookup (which secret does which component need?) — see the rst source.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

Comment threadairflow-core/docs/security/jwt_token_authentication.rst Outdated
potiuk added 3 commits May 25, 2026 08:40
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of apache#67435.
@potiuk
potiukforce-pushed the docs/security-model-v3.2-catchup branch from e6a0a7c to 5c989d2CompareMay 25, 2026 06:40
@vatsrahul1001
vatsrahul1001 merged commit 0a506b1 into apache:mainMay 25, 2026
145 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-2-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-2-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 0a506b1 v3-2-test

This should apply the commit to the v3-2-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@potiuk@kaxil@vatsrahul1001
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Docs: refresh JWT and security model for v3.2 with mermaid diagrams - #67435

Merged
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup
May 25, 2026
Merged

Docs: refresh JWT and security model for v3.2 with mermaid diagrams#67435
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup

Conversation

@potiuk

Copy link
Copy Markdown
Member

Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
and re-aligns prose with the current code.

What changed in the docs

airflow-core/docs/security/jwt_token_authentication.rst:

airflow-core/docs/security/security_model.rst:

Tooling change

Registers sphinxcontrib-mermaid>=1.0.0 as a new dependency in
devel-common[docs] and adds sphinxcontrib.mermaid to
BASIC_SPHINX_EXTENSIONS so every Airflow Sphinx build (core,
providers, chart, docker-stack) can use .. mermaid:: directives.
uv.lock is regenerated.

Verification

  • breeze build-docs --package-filter apache-airflow
    "Documentation build is successful", 0 build errors, 0 spelling errors.
  • All six mermaid containers render in the produced HTML.
  • prek run --from-ref upstream/main --stage pre-commit and
    --stage manual both pass.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

@potiuk

Copy link
Copy Markdown
MemberAuthor

@potiuk

Copy link
Copy Markdown
MemberAuthor

Also cc: @vatsrahul1001 -> we should include it in rc2

@potiuk

potiuk commented May 24, 2026

Copy link
Copy Markdown
MemberAuthor

Below are the six mermaid diagrams introduced in this PR (updated for higher contrast and an easier-to-read credential matrix), rendered inline via GitHub's native mermaid support. They are identical to what breeze build-docs produces in the published HTML.


jwt_token_authentication.rst — overview of components and flows

flowchart LR
subgraph Clients
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph Internal["Internal Airflow components"]
WORKER[Worker / Task]
DFP[Dag File Processor]
TRG[Triggerer]
end
APISVR[API Server]
EXECAPI[Execution API]
UI -->|JWT cookie / Bearer| APISVR
CLI -->|Bearer| APISVR
EXT -->|Bearer| APISVR
WORKER -->|Bearer<br/>workload &rarr; execution| EXECAPI
DFP -. in-process<br/>JWT bypassed .-> EXECAPI
TRG -. in-process<br/>JWT bypassed .-> EXECAPI
classDef internal fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WORKER,DFP,TRG internal
Loading

jwt_token_authentication.rst — symmetric vs asymmetric signing

flowchart TB
subgraph Sym["Symmetric (HS512)"]
direction LR
S1[Scheduler / API Server]
S2[Shared secret<br/>jwt_secret]
S3[Token validator]
S1 -->|sign| S2 -->|same secret<br/>also validates| S3
end
subgraph Asym["Asymmetric (RS256 / EdDSA)"]
direction LR
A1[Scheduler / API Server]
A2[Private key<br/>jwt_private_key_path]
A3[Public key /<br/>JWKS endpoint]
A4[Token validator]
A1 -->|sign| A2
A2 -. derives or<br/>publishes .-> A3
A3 -->|verify only| A4
end
classDef secret fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pub fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class S2 secret
class A2 secret
class A3 pub
Loading

jwt_token_authentication.rst — two-token sequence (workload → execution)

sequenceDiagram
autonumber
participant SCH as Scheduler
participant EXE as Executor<br/>(Celery / K8s / Local)
participant WRK as Worker
participant API as Execution API
Note over SCH: Task ready to dispatch
SCH->>SCH: generate workload token<br/>scope=workload<br/>exp = task_queued_timeout
SCH->>EXE: workload JSON<br/>(includes token)
Note over EXE: Task waits in queue<br/>(can be minutes)
EXE->>WRK: dispatch (workload JSON)
WRK->>API: POST /run<br/>Bearer: workload token
Note over API: validates workload scope<br/>checks TI in QUEUED/RESTARTING<br/>409 if not
API-->>WRK: 200 OK<br/>Refreshed-API-Token: execution token<br/>(scope=execution, ~10 min)
WRK->>WRK: BearerAuth swaps to<br/>execution token
loop For all subsequent calls (heartbeats, XComs, ...)
WRK->>API: Bearer: execution token
alt token expiring (less than 20% left)
API-->>WRK: 200 OK<br/>Refreshed-API-Token: new execution token
WRK->>WRK: BearerAuth swaps again
end
end
Loading

jwt_token_authentication.rst — Execution API request-time validation pipeline

flowchart TD
REQ([Incoming request<br/>Authorization: Bearer ...])
REQ --> CACHE{Cached on<br/>request.scope?}
CACHE -->|yes| RET([Return cached TIToken])
CACHE -->|no| SIG[JWTValidator:<br/>verify signature]
SIG -->|fail| F1([403 Forbidden])
SIG -->|ok| STD[Verify exp / iat / nbf<br/>aud / iss]
STD -->|fail| F1
STD -->|ok| SCOPE[Default scope to<br/>'execution' if absent]
SCOPE --> SCHEMA[TIClaims:<br/>typed Pydantic schema]
SCHEMA -->|ValidationError| F1
SCHEMA -->|ok| TYP{require_auth:<br/>scope in<br/>route.allowed_token_types?}
TYP -->|no| F1
TYP -->|yes| SELF{ti:self scope<br/>declared?}
SELF -->|no| OK([Return TIToken])
SELF -->|yes| MATCH{token.sub ==<br/>task_instance_id?}
MATCH -->|no| F1
MATCH -->|yes| OK
classDef fail fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef pass fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
class F1 fail
class OK,RET pass
Loading

security_model.rst — component trust boundaries

flowchart LR
subgraph users["Users (untrusted by default)"]
UI[UI / browser]
CLI[CLI]
EXT[External REST clients]
end
subgraph dataplane["Worker plane (no metadata DB access)"]
WRK[Worker / Task]
end
subgraph controlplane["Control plane (metadata DB access)"]
APISVR[API Server]
SCH[Scheduler]
DFP[Dag File Processor]
TRG[Triggerer]
end
DB[(Metadata DB)]
UI -->|JWT| APISVR
CLI -->|JWT| APISVR
EXT -->|JWT| APISVR
WRK -->|JWT<br/>Execution API| APISVR
APISVR -. SQL .-> DB
SCH -. SQL .-> DB
DFP -. SQL .-> DB
TRG -. SQL .-> DB
DFP -. in-process<br/>JWT bypassed .-> APISVR
TRG -. in-process<br/>JWT bypassed .-> APISVR
classDef untrusted fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
classDef trusted fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef data fill:#fff9c4,stroke:#f57f17,stroke-width:2px,color:#000
class UI,CLI,EXT,WRK untrusted
class APISVR,SCH,DFP,TRG trusted
class DB data
Loading

security_model.rst — credential-distribution (least → most privileged)

flowchart LR
subgraph WRK["Worker (least privileged)"]
direction TB
W1[Fernet key]
W2[Worker secrets backend credentials]
W3[Remote log handler kwargs]
end
subgraph TRG["Triggerer"]
direction TB
T1[DB connection]
T2[Fernet key]
T3[Non-worker secrets backend credentials]
T4[Remote log handler kwargs]
end
subgraph DFP["Dag File Processor"]
direction TB
D1[DB connection]
D2[Fernet key]
D3[Non-worker secrets backend credentials]
end
subgraph SCH["Scheduler"]
direction TB
S1[DB connection]
S2[JWT signing key]
S3[Fernet key]
S4[Non-worker secrets backend credentials]
S5[Remote log handler kwargs]
end
subgraph API["API Server (most privileged)"]
direction TB
A1[DB connection]
A2[JWT signing key]
A3[Fernet key]
end
classDef wrk fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
classDef ctrl fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
class WRK wrk
class API,SCH,DFP,TRG ctrl
Loading

The doc also includes an explicit ✓/— table for true matrix lookup (which secret does which component need?) — see the rst source.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

Comment threadairflow-core/docs/security/jwt_token_authentication.rst Outdated
potiuk added 3 commits May 25, 2026 08:40
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of apache#67435.
@potiuk
potiukforce-pushed the docs/security-model-v3.2-catchup branch from e6a0a7c to 5c989d2CompareMay 25, 2026 06:40
@vatsrahul1001
vatsrahul1001 merged commit 0a506b1 into apache:mainMay 25, 2026
145 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-2-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
v3-2-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 0a506b1 v3-2-test

This should apply the commit to the v3-2-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…67435) (#67466)
* Docs: refresh JWT and security model for v3.2 with mermaid diagrams
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:
- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
/api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).
Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
* Docs: improve security-diagram readability and add credential matrix
- Replace the arrow-spaghetti credential-distribution mermaid with a
component-grouped layout (least- to most-privileged left-to-right)
plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
medium-saturation fills with explicit black text and 2px strokes, so
labels stay readable in both light and dark mode renderers.
* Fix HTTP verb in JWT auth mermaid diagram (PATCH, not POST)
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of #67435.
(cherry picked from commit 0a506b1)
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@potiuk@kaxil@vatsrahul1001