[fix](fe) Fix Ranger column-level privilege bypass when CTE combined

[smith1000]

What problem does this PR solve?

Issue Number: close#61631

Problem Summary: When a CTE (WITH ... AS) is referenced multiple times in a
JOIN query and is not inlined (due to inlineCTEReferencedThreshold), the
CheckPrivileges rule does not traverse the CTE producer subtree because
LogicalCTEConsumer is a leaf node in the plan tree. This means column-level
privileges on the CTE's underlying tables are never checked, allowing users
without proper column access to bypass Ranger authorization.

The fix adds a visitLogicalCTEConsumer override in CheckPrivileges that
explicitly retrieves the CTE producer plan (stored by RewriteCteChildren)
and traverses it for privilege checking. The privChecked flag remains on
StatementContext to preserve the view permission passthrough mechanism.

Release note

Fixed a security issue where Ranger column-level privileges could be bypassed
when using CTE (WITH ... AS) combined with JOIN queries. Users without proper
column access permissions could read restricted columns through CTE+JOIN
patterns.

Check List (For Author)

• Test: Unit Test / Manual test (verified with Ranger 2.7.0 + Doris 4.0.2 environment)
• Behavior changed: No
• Does this need documentation: No

[Thearas]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[morrySnow]

add test please

[smith1000]

sure

[smith1000]

@morrySnow Test cases added. Please take a look

[924060929]

Hi, @smith1000,

It is expected that the entire query statement checks permissions only once, which helps prevent redundant permission checks. In addition, placing the permission flag in the StatementContext is intentional.

Consider this scenario: a user has access to view_a, but does not have access to table_b inside view_a. In this case, two CascadeContexts are generated—one for the scope outside view_a, and one for the scope inside view_a. The expected behavior is that if the user has access to view_a, they can bypass the permission check on table_b and query it through the view.

This requirement implies that the permission information needs to be shared in a common place (i.e., StatementContext).

Therefore, for this issue, the expected behavior is to traverse the outermost query. If a CTE reference is encountered, the traversal should continue into the CTE to perform a complete permission check. Once the entire traversal is finished, the StatementContext should be marked as having completed the permission check.

[924060929]

/review

[github-actions]

Code Review Summary

Verdict: No issues found. The change is correct, minimal, and well-tested.

Change Overview

This PR fixes a security bug (Ranger column-level privilege bypass with CTE+JOIN) by moving the privChecked flag from StatementContext (shared across all CTE subtrees) to CascadesContext (per-subtree). This ensures each CTE subtree gets its own independent privilege check.

Critical Checkpoint Conclusions

• Goal and correctness: The goal is to fix a privilege bypass when CTE is referenced multiple times via JOIN (preventing CTE inlining). The fix correctly moves privChecked to CascadesContext, since RewriteCteChildren creates a new CascadesContext per CTE subtree (via newSubtreeContext()), each starting with privChecked = false. Tests prove the fix works.

• Modification minimality: The change touches 4 files with +41/-14 lines. It is focused solely on moving the flag and adding tests. No unnecessary changes.

• Concurrency: privChecked is a non-volatile boolean on CascadesContext. This is safe because each CascadesContext is created and used within a single thread during the rewrite phase. No concurrent access concerns.

• View privilege regression risk: The original comment ("Only enter once, if repeated, the permissions of the table in the view will be checked") referred to avoiding re-checking after InlineLogicalView exposes base tables. This is still safe because: (1) CheckPrivileges runs before InlineLogicalView in the same topic block in Rewriter.java:497-507, (2) the per-CascadesContext guard prevents re-execution within the same context, and (3) CheckPrivileges appears only once in the entire job list (CTE_CHILDREN_REWRITE_JOBS_BEFORE_SUB_PATH_PUSH_DOWN).

• Other callers: isPrivChecked/setPrivChecked are only called from CheckPrivileges.rewriteRoot(). The CreateMaterializedViewCommand.validate() also invokes CheckPrivileges but uses its own fresh CascadesContext, so the fix is compatible.

• Parallel code paths: No other parallel code paths affected. The change is isolated to the Nereids privilege checking mechanism.

• Incompatible changes: None. No storage format, function symbol, or protocol changes.

• Configuration items: None added.

• Test coverage: Four new CTE+JOIN test cases added covering: (1) fully-privileged CTE+LEFT JOIN succeeds, (2) restricted column CTE+LEFT JOIN denied, (3) allowed-columns-only CTE+LEFT JOIN succeeds, (4) restricted column CTE+INNER JOIN denied. Tests exercise the per-CascadesContext privilege checking path since multi-reference CTEs trigger RewriteCteChildren with separate subtree contexts. Existing view and base table privilege tests continue to validate no regression.

• Performance: No performance impact. The privChecked flag is a simple boolean check.

• Other issues: None identified.

[smith1000]

hi, @924060929 ,
Thank you for the detailed review and explanation. I will submit a new commit later

[smith1000]

hi, @924060929

I've reworked the fix based on your feedback. The privChecked flag now stays on StatementContext to preserve view permission passthrough. Instead, CheckPrivileges.visitLogicalCTEConsumer explicitly traverses the CTE producer plan for privilege checking.
Local verification was done with Ranger 2.7.0 + Doris 4.0.2 (patched JAR):
Setup: A column-level Ranger policy grants test_limited user SELECT on restricted_table(id, name) only (no access to salary). A separate view-level policy grants test_limited SELECT on restricted_view (which includes salary).

#	Scenario	Expected	Result
1	SELECT id, name FROM restricted_table	OK	PASS
2	SELECT id, name, salary FROM restricted_table	Denied	PASS
3	SELECT * FROM restricted_table	Denied	PASS
4	SELECT * FROM restricted_view (view passthrough)	OK	PASS
5	CTE + LEFT JOIN with SELECT * on restricted_table (original bug)	Denied	PASS
6	CTE + LEFT JOIN with allowed columns only	OK	PASS
7	CTE + LEFT JOIN explicitly referencing salary	Denied	PASS
8	CTE referencing restricted_view + JOIN	OK	PASS
9	Single-ref CTE with SELECT * (inlined by optimizer)	Denied	PASS
10	Single-ref CTE with allowed columns	OK	PASS
11	Nested CTEs + JOIN with allowed columns	OK	PASS
12	Nested CTEs + JOIN with restricted column	Denied	PASS

All 12 scenarios passed. The fix correctly denies CTE+JOIN bypass while preserving view permission passthrough.

[924060929]

/review

[github-actions]

Code Review Summary

PR: fix Fix Ranger column-level privilege bypass when CTE combined with JOIN

Overall Assessment

This is a well-targeted security fix for a privilege bypass vulnerability (Issue #61631). The approach is sound: when a CTE is materialized (not inlined due to inlineCTEReferencedThreshold), LogicalCTEConsumer is a leaf node, so CheckPrivileges (which inherits ColumnPruning's traversal) never reaches the underlying tables in the CTE producer subtree. The fix explicitly traverses the producer plan when visiting a CTE consumer.

Critical Checkpoint Conclusions

1. Goal and correctness:
The goal is to prevent column-level privilege bypass when CTEs are used in JOIN queries (causing materialization). The fix correctly stores the CTE producer plan in StatementContext during RewriteCteChildren.visitLogicalCTEAnchor and retrieves it in CheckPrivileges.visitLogicalCTEConsumer to traverse for privilege checks. Tests prove the fix works for both positive (allowed) and negative (denied) cases.

2. Modification minimality:
The change is focused and minimal: 4 files, 68 additions. The approach reuses existing infrastructure (StatementContext, PruneContext, visitor pattern) without introducing new abstractions.

3. Concurrency:
No concurrency concerns. CheckPrivileges is instantiated fresh per use (CheckPrivileges::new in Rewriter.java:505). The checkedCteIds instance field is thread-local to each instance. StatementContext is per-statement and not shared across threads during the rewrite phase.

4. Lifecycle management:
The cteProducerPlansForPrivCheck map on StatementContext stores immutable plan references (Nereids plans are immutable trees). No lifecycle issues or circular references. The map lives as long as the StatementContext (per-statement scope).

5. Execution ordering:
Verified correct. RewriteCteChildren.visitLogicalCTEAnchor stores the producer plan at lines 83-84 before recursing into the consumer side. CheckPrivileges runs as part of CTE_CHILDREN_REWRITE_JOBS_BEFORE_SUB_PATH_PUSH_DOWN inside the consumer-side rewriting. The producer plan is guaranteed to be in the map when visitLogicalCTEConsumer looks it up.

6. Pre-rewrite producer plan correctness:
The stored producer plan is cteAnchor.child(0) before the producer undergoes further rewriting. This is correct because: (a) Nereids plans are immutable, so the reference remains valid, and (b) the pre-rewrite plan contains all LogicalCatalogRelation nodes (the post-rewrite version can only restructure or eliminate relations, never add new ones). Checking all referenced tables is the correct security posture.

7. producerPlan.getOutputExprIdBitSet() as requiredSlotsIds:
Passing all output columns as "required" is correct. The CTE definition explicitly references those columns, and the user needs privilege for every column they reference, regardless of downstream usage. This is consistent with how inlined CTEs behave (column pruning propagates all CTE-body columns to the scan).

8. Edge cases verified:

• Nested CTEs: The recursive accept call naturally discovers nested LogicalCTEConsumer nodes. Anchor ordering ensures outer CTEs' producers are stored first.
• Partial inlining: Inlined CTEs have tables directly in the plan tree (normal check). Non-inlined CTEs use the cteProducerPlansForPrivCheck map.
• Recursive CTEs: LogicalWorkTableReference (self-reference) is not a LogicalCatalogRelation, so it's safely skipped.
• checkedCteIds guard: Correctly prevents redundant checking when multiple consumers share the same producer.

9. Functionally parallel code paths:
No other code paths need this fix. The privChecked flag on StatementContext correctly prevents double-checking in the view permission passthrough scenario.

10. Configuration items:
No new configuration items added. None needed.

11. Incompatible changes:
None. No storage format changes, no function symbol changes.

12. Test coverage:
Tests cover 4 scenarios: CTE+LEFT JOIN with full privilege (pass), CTE+LEFT JOIN with restricted column (denied), CTE+LEFT JOIN with only allowed columns (pass), CTE+INNER JOIN with restricted column (denied). Tests reuse the existing test infrastructure and privilege mock framework.

13. Observability:
Not applicable for this change (privilege denial throws AnalysisException with descriptive messages from the existing checkColsPriv implementation).

14. Performance:
Negligible impact. The producer plan traversal happens once per CTE (guarded by checkedCteIds), and only for materialized (non-inlined) CTEs.

Minor Observations (Non-blocking)

• Dead code in ColumnPruning: The base class ColumnPruning has private methods computeUsedColumns (line 556) and checkColumnPrivileges (line 572) that appear to be dead code -- they are never called from within ColumnPruning itself. CheckPrivileges has its own copies of these methods. This is a pre-existing issue, not introduced by this PR, and could be cleaned up separately.

Verdict

No issues found. The fix is correct, minimal, well-tested, and handles edge cases properly. The security vulnerability is properly addressed.

[924060929]

I think you should reuse StatementContext.cteIdToProducer and StatementContext.getCteProducerByCteId()

[morrySnow]

@smith1000 are you still dealing with this issue?

[seawinde]

run buildall

[hello-stephen]

FE UT Coverage Report

Increment line coverage 94.44% (17/18) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 100.00% (18/18) 🎉
Increment coverage report
Complete coverage report

[smith1000]

Hi @morrySnow, yes I'm still working on this. Sorry for the delayed response — I had some personal changes going on recently and was away for a while.

I'm back now and will submit a new revision soon based on @924060929's review feedback. Thanks for keeping this PR alive.

[smith1000]

Hi @924060929,

Thanks for the detailed review feedback. I've reworked the fix accordingly:

Changes in this revision:

• Removed the custom cteProducerPlansForPrivCheck map and its accessors from StatementContext.
• CheckPrivileges.visitLogicalCTEConsumer now uses the existing getCteProducerByCteId() method (introduced by commit a9bd84c) to retrieve the CTE producer plan. This reuses the cteIdToProducer map that RewriteCteChildren already populates via setCteProducer(), so no additional storage is needed.
• The privChecked flag remains on StatementContext to preserve view permission passthrough, as you suggested.
• The fix is now a single-file change to CheckPrivileges.java (plus the test file).

Local verification:

We are currently running Doris 4.0.2 in production. Since getCteProducerByCteId() does not exist in 4.0.2 (it was introduced after the 4.1.0-rc01 tag), we will apply the earlier CascadesContext-based fix (moving privChecked to CascadesContext) to patch our local doris-fe deployment. That approach has been verified with 12 end-to-end Ranger integration test scenarios (all passed). If we discover any issues during local use, we will report them to the community.

Note on dev/4.0.x cherry-pick compatibility:

The getCteProducerByCteId() method used in this PR was introduced by commit a9bd84c60d3 (2026-03-11). It exists on master and the latest branch-4.1 code, but is not present in any released version (4.0.2 ~ 4.0.5, 4.1.0). When cherry-picking to branch-4.0, the cherry-picker will need to either:

1. Also cherry-pick commit a9bd84c60d3 (and its dependency chain), or
2. Adapt the fix to use the CascadesContext approach (moving privChecked from StatementContext to CascadesContext).

[morrySnow]

run buildall

[github-actions]

PR approved by at least one committer and no changes requested.

[github-actions]

PR approved by anyone and no changes requested.