Uh oh!
There was an error while loading. Please reload this page.
feat(sql): Support RANK/DENSE_RANK for unified SQL - #5720
Conversation
PR Reviewer Guide 🔍(Review updated until commit 544e467)Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Latest suggestions up to 544e467 Explore these optional code suggestions:
Previous suggestionsSuggestions up to commit 77ae6e0
Suggestions up to commit 747d1b6
Suggestions up to commit 352ffbd
|
352ffbd to
747d1b6ComparePersistent review updated to latest commit 747d1b6 |
747d1b6 to
77ae6e0ComparePersistent review updated to latest commit 77ae6e0 |
Both were accepted by the SQL grammar and declared as BuiltinFunctionName constants, but missing from WINDOW_FUNC_MAPPING, so visitWindowFunction rejected them with "Unexpected window function". Register both, extend the ROW_NUMBER bypass of aggregate signature validation since they likewise take no arguments, and lower them to SqlStdOperatorTable.RANK and DENSE_RANK. WINDOW_FUNC_MAPPING is shared, so PPL eventstats/streamstats now resolve these functions as well. Related to opensearch-project#5168 Signed-off-by: Chen Dai <daichen@amazon.com>
77ae6e0 to
544e467ComparePersistent review updated to latest commit 544e467 |
RANK/DENSE_RANK in unified SQLRANK/DENSE_RANK for unified SQLWINDOW_FUNC_MAPPING (used by eventstats/streamstats) never supported rank/dense_rank, but PPL's scalarWindowFunctionName grammar rule still accepted the tokens, so `eventstats rank()` reached CalciteRexNodeVisitor#visitWindowFunction and failed there with the "not supported" message this PR improves. Meanwhile SQL's grammar already accepts RANK()/DENSE_RANK() OVER (...), and opensearch-project#5720 is adding real support for them on the SQL side via the same shared visitor. Remove RANK/DENSE_RANK from scalarWindowFunctionName so PPL rejects them at parse time instead of falling through to the shared SQL/PPL visitor - this keeps the language separation at the parser rather than relying on a WINDOW_FUNC_MAPPING check in shared planner code, and avoids PPL silently gaining rank/dense_rank as a side effect of opensearch-project#5720 registering them for SQL. Updates the eventstats/streamstats tests added earlier in this PR to expect a SyntaxCheckException (parse-time) instead of the semantic "not supported" error, and switches the unrelated visitWindowFunction-rejection unit test from rank() to percent_rank(), which remains unsupported and still exercises that code path. Signed-off-by: Radhakrishnan P <gingeekrishna@gmail.com>
RyanL1997
left a comment
There was a problem hiding this comment.
LGTM. Just would like to double check do we need to add doc test/doc for this?
RyanL1997
commented
Aug 28, 2026
In addtion, I found out that
|
Uh oh!
There was an error while loading. Please reload this page.
WINDOW_FUNC_MAPPING (used by eventstats/streamstats) never supported rank/dense_rank, but PPL's scalarWindowFunctionName grammar rule still accepted the tokens, so `eventstats rank()` reached CalciteRexNodeVisitor#visitWindowFunction and failed there with the "not supported" message this PR improves. Meanwhile SQL's grammar already accepts RANK()/DENSE_RANK() OVER (...), and opensearch-project#5720 is adding real support for them on the SQL side via the same shared visitor. Remove RANK/DENSE_RANK from scalarWindowFunctionName so PPL rejects them at parse time instead of falling through to the shared SQL/PPL visitor - this keeps the language separation at the parser rather than relying on a WINDOW_FUNC_MAPPING check in shared planner code, and avoids PPL silently gaining rank/dense_rank as a side effect of opensearch-project#5720 registering them for SQL. Updates the eventstats/streamstats tests added earlier in this PR to expect a SyntaxCheckException (parse-time) instead of the semantic "not supported" error, and switches the unrelated visitWindowFunction-rejection unit test from rank() to percent_rank(), which remains unsupported and still exercises that code path. Signed-off-by: Radhakrishnan P <gingeekrishna@gmail.com>
…layer opensearch-project#5720 added rank/dense_rank to the WINDOW_FUNC_MAPPING shared by both SQL's RANK()/DENSE_RANK() OVER (...) and PPL's eventstats/streamstats, which silently enabled them for eventstats/streamstats too: testRankingWindowFunctionsUnsupportedInEventstats/InStreamstats (added earlier in this PR) started failing with "expected ResponseException to be thrown, but nothing was thrown" once opensearch-project#5720 merged, since eventstats rank() now builds a real window call instead of hitting the "not supported" check. That's a real gap, not just a test artifact: PPL's eventstats/ streamstats grammar has no ORDER BY syntax at all, so ranking has no defined ordering to rank by there - unlike SQL's OVER(), which at least has (optional) ORDER BY in its own clause. A prior commit on this branch tried fixing this by removing RANK/ DENSE_RANK from the PPL grammar entirely, rejecting them at parse time. That broke a different, cross-repo contract: OpenSearch- Dashboards' PPL linter (validated by the "PPL grammar compatibility" CI check) expects `eventstats rank()` to parse successfully and be flagged by a semantic-layer diagnostic instead, so it could no longer produce that diagnostic once the query stopped parsing. That commit was reverted. Fix this at the semantic layer instead, where it belongs: PPL only ever reaches CalciteRexNodeVisitor#visitWindowFunction through eventstats/streamstats (no other PPL syntax builds a WindowFunction node), so context.queryType == PPL is an exact, unambiguous signal for "this is an eventstats/streamstats call". Filter rank/dense_rank out of the WINDOW_FUNC_MAPPING lookup specifically when queryType is PPL, so they fall through to the existing "not supported in eventstats/ streamstats" error - exactly the pre-opensearch-project#5720 behavior - while leaving SQL's RANK()/DENSE_RANK() OVER (...) handling (and everything else) completely untouched. Signed-off-by: Radhakrishnan P <gingeekrishna@gmail.com>
Description
RANK()andDENSE_RANK()were already accepted by the SQL grammar and declared as built-in function names, but were never registered as window functions, so planning failed withUnexpected window function: RANK. This PR registers both and maps them to Calcite's standardRANKandDENSE_RANKoperators. No grammar, lexer or AST changes were needed.Related Issues
Part of #5248
Check List
--signoffor-s.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.